Мы в СМИ
The second version of the frequency list
From this page you can access the frequency list for modern Russian. Up to now Chastotnyj slovarj russkogo jazyka (Zasorina, 1977) provided the most widely used frequency list for Russian. However, the corpus used in Zasorina is relatively small according to modern standards (about 1 million words). It is outdated: mostly it covers uses from 1920s to 1960s and includes a high proportion of ideological sources, like texts by Lenin and Khrushchev and Soviet newspapers, thus, word frequencies in it are severely biased, e.g. Soviet and comrade are in the first hundred of Russian words on a par with function words. Finally, the list of (Zasorina, 1977) is not available electronically.
The list accessible from this page includes about 32000 words with frequency greater than 1 ipm (one instance per million words). A shorter selection of 5000 most frequent words is also available. Lists use utf8 encoding for Cyrillic and are compressed by lemma.al.zip - lemmas sorted in the alphabetical order
Lists of 5000 most frequent words
Some data about uses of words in modern Russian
The exact information on the mapping of frequency to coverage is available from here.
The list is compiled on the basis of a corpus of modern Russian. It contains a selection of modern fiction, political texts, newspapers, and popular science (about 40 million words, MW, fiction allocates for about half of the corpus). All texts were written originally in Russian between 1970 and 2002; the majority of them between 1980 and 1995, the newspapers corpus is from 1997-1999.
It is widely known that large texts present a problem for frequency lists, since a large text that contains many instances of a rare word can boost its frequency. If the corpus is based on fiction, large texts are quite frequent. As an example, the corpus contains a huge sequel to Tolkien's "The Lord of the Rings" written by a Russian author (Nick Perumov). In spite of the fact that the length of the sequel is about 250 kW, less than one percent of the whole corpus, the frequency of uses of the word hobbit in that book puts the word in the first thousand of most frequent Russian words, if no precautions against large texts are made. Out of this reason, the frequency list is calculated under the condition that no single text from the corpus contributes more than 10 kW and no author contributes more than 100 kW to the count. Thus, the subset of the whole corpus used for frequency count is about 16 MW.
Words are not uniformly distributed in texts. Some of them (like prepositions) occur in many texts with predictable rate, some (like pronouns or mental verbs) are significantly more frequent for certain writers or genres, while some are "contagious": if a word (e.g. a proper name, a title of nobility or a technical term) occurs once in a text, it tends to be repeated, thus boosting its frequency in a document. The variation can be measured in a variety of ways (Church, K. and Gale, W. (1995) here. The structure:
The corpus, tools for working with it, as well as an aligned parallel English-Russian corpus are discussed in the following publication:
Sharoff, Serge, (2002). Meaning as use: exploitation of aligned corpora for the contrastive study of lexical semantics. Proc. of Language Resources and Evaluation Conference (LREC02). May, 2002, Las Palmas, Spain. PDF file.
Three frequency lists for word classes are also available:
|Все права защищены. © 2001 РосНИИ ИИ||Copyright. © 2001 RRIAI|