First draft of linguistic typology write-up. To be included in COLING submission.
We approach our analysis of this language modeling disparity from a linguistic perspective, and as such break our discussion into three primary areas: word order typology, morphological typology, and etymology. For further research, it may help the reader to know that the former two lie within the subfield of linguistic typology (which examines languages based on their structural features), whereas the latter (which studies the history of words and languages) is a different subfield of linguistics.
Word order typology is the study of ordering within a sentence. In our discussion of word order, we will focus chiefly on constituent word order, where a constituent is defined as a stand-alone unit of language (for example, a word or phrase); however, within word order typology there do exist other points of analysis, such as modifier order, that we will not consider due to their perceived limited effect on modeling efficacy. We will also discuss word order flexibility.
Nearly all languages have a dominant word order that is the most frequently found ordering of a sentence's subject, verb, and object. Given the flexibility of word order in a given language (see section [section number] below), a sentence in that language might vary from the dominant order because of situational constraints, to place emphasis, or to convey emotion. Given that our dataset is composed of formal news stories, we expect our samples to adhere to each language's dominant word order more than is perhaps representative of the language in other contexts, e.g. informally.
The constituent orders present in our dataset are subject-verb-object (SVO) and subject-object-verb (SOV). There are other word ordering schemes present in the world's languages, such as verb-subject-object (VSO) and object-verb-subject (OVS) that are not represented in our corpus.
We note two language-specific details here, and further details can be seen in [figure here]. Although most languages can be classified into a single dominant word order, the degree to how dominant that word order is changes on the language's word order flexibility. Although we will discuss word order flexibility shortly, it is important to note that Russian, as a language with flexible word order, is still considered an SVO language despite that ordering being less strict than other languages [cite source]. However, German varies between SVO (in main clauses without an auxiliary verb) and SOV (in clauses with an auxiliary verb and in subordinate clauses) equally, and thus is categorized as lacking a dominant verb order [cite source].
For clarity of analysis, we utilize two metrics, typicality and similarity indices, that are calculated according to known features of each language in our dataset. We have adopted these two metrics from the related linguistic literature [cite Comrie here], and summarize their calculation below. A language's features are referenced from the World Atlas of Language Structures, or WALS, where a feature is defined as a structural property of language that differs across languages [cite WALS here].
Comrie (2016) presents a typicality index that intuitively can be understood as the degree to which the given language's features match a hypothetical "average" of the world's languages. Thus the typicality index ranks languages relative to this average language, as opposed to any existing baseline. This metric also lends itself to viewing typicality as a spectrum, with higher scores implying higher typicality and lower scores implying higher atypicality.
The typicality index is calculated by averaging the percentage of the world's languages that share the same value of that feature, for each feature. For example, Japanese is categorized as having an SOV constituent order, just as 41.03% of the world's languages also are. For every other feature WALS contains on Japanese, this percentage is found; the typicality index is the average of these percentages. A full list of the typicality indices of the languages present in our dataset can be seen in [figure].
It is important to take note of a few points in this calculation. First, no single feature is intrinsically weighted higher or lower than the others; this is because there is no set of features that the linguistics field has determined contributes more to language typicality than others. However, WALS contains more entries on variations of constituent order than any other feature; thus, constituent order may be a disproportionally higher factor in a language's typicality index. Another characteristic of WALS that may affect our typicality index results is that features are not mutually exclusive, and indeed some sets of features have high levels of correlation, which may lead to over-counting those features if each feature is weighted equally. Lastly, to avoid having a language's typicality index be dependent on a few features if little were included in WALS, Comrie (2016) only considers languages for which WALS catalogues 100 or more features. Although most of the languages in our dataset meet this requirement, Italian only has 81 documented features in WALS; thus its typicality index should perhaps be treated a bit more skeptically.