新闻 · arXiv cs.CL

Language Identification via Compositional Data Analysis: A Linear-Time Classifier Based on Log-Ratio Geometry

Language identification is commonly addressed using either neural architectures or statistical n-gram models. Neural approaches typically require substantial computational resources, whereas classical frequency-based methods offer efficient linear-time performance, but rely on distance metrics that are not always appropriate for compositional data. This work models character and bigram frequency distributions as compositional vectors constrained to the simplex and mapped via the centered…