A Lexical Analysis of the ELAN Slovene-English Parallel Corpus
Špela VINTAR
University of Ljubljana, Faculty of Arts, Dept. of Translation and Interpreting
Aškerčeva 2, 1000 Ljubljana, Slovenia
spela.vintar@guest.arnes.si

Abstract 
In compiling parallel corpora, it is generally difficult to achieve the principles of representativeness and balance. The LJU1/ELAN corpus consists of texts that were selected primarily on the grounds of their availability, hence the need for analysis. The paper presents the results of a survey covering various aspects of the corpus composition and structure, including word frequencies, the keywordness of individual texts, collocations and multi-word units. We also explore the notion of translation equivalence and outline some possibilities for further exploitation of the corpus, particularly for the creation of lexical resources.

1	Introduction
A parallel corpus can be a valuable resource for linguists, computational linguists, lexicographers, translators, FL teachers and learners, to all those interested in studying, comparing and contrasting linguistic phenomena in aligned texts of two or more languages. The compilation of a corpus is usually a long and complex process, where the principles for text selection, encoding and annotation have been previously laid down in accordance with the general concept of the project and its specific needs. However, the compilation of parallel corpora tends to be slightly more problematic for several reasons, most notably because large quantities of electronic parallel texts are not that easy to come by. 

With the ELAN/LJU1 corpus [1], the only principle for text selection was their availability and the chief goal was to compile a parallel corpus of 1 million words in the shortest possible time. Clearly, any illusions about its representativeness would have been out of place, nevertheless we are convinced that the corpus – apart from being the first step to a possibly larger project of this kind – will prove a valuable resource at least in certain subfields of linguistic research. Thus a preliminary analysis is seen as a necessary stage, or in other words, we would like to know just how unbalanced this corpus is.

2	Tools
General statistics, frequency wordlists and keywords were obtained using Mike Scott’s Wordsmith Tools [2]. Concordances, collocations, clusters and translation equivalents were studied with the help of Michael Barlow’s MonoConc and ParaConc [3] as well as the IMS CQP and XKWIC [4]. For the first attempts in automatic lexicon extraction we used the Twenty-One word alignment software developed by Djoerd Hiemstra [5]. Simple Perl scripts were used for corpus pre-processing and the post-processing of query results.
 
3	General corpus statistics
The corpus consists of 15 parallel Slovene-English texts comprising 1.142,751 words, of which 510,533 Slovene and 632,218 English (see Table 1). The first considerable difference between the Slovene and the English part is the type/token ratio, as the number of types in Slovene (46,192) is over the double of English. This is, however, largely due to the fact that neither of the texts has been lemmatized, so that the types here are word forms.

N	1	1	
Text File	ELANSLO	ELANEN	
Bytes	3,375,576	3,716,919	
Tokens	510,533	632,218	
Types	46,192	19,587	
Type/Token Ratio	9.05	3.1	
Standardised Type/Token	50.48	35.66	
Ave. Word Length	5.36	4.7	
Sentences	25,572	24,993	
Sent.length	19.88	25.13	
Sd. Sent. Length	19.16	42.62	
1-letter words	38,095	55,143	
2-letter words	113,328	110,040	
3-letter words	34,970	119,337	
4-letter words	40,064	79,067	
5-letter words	46,243	57,553	
6-letter words	49,401	46,169	
7-letter words	47,510	45,813	
8-letter words	46,122	39,276	
9-letter words	35,621	30,290	
10-letter words	23,553	21,381	
11-letter words	16,341	12,928	
12-letter words	9,593	7,131	
13-letter words	5,191	4,660	
14-letter words	2,465	2,458	
15-letter words	1,087	673	
16(+)-letter words	452	173	

Table 1

Slovene words tend to be slightly longer that English on average, and there is also a higher number of 6- through 16-letter words, though in turn the average number of words in English sentences is higher, and also the standard deviation is twice as high.

Similar statistics were obtained for each “half” of the 15 texts and compared with the above data for the whole corpus. Yet before any further observations can be made about individual texts, perhaps the texts themselves should be briefly characterised. While realising that this might be an over-generalisation and highly questionable from other points of view, for the purposes of this analysis we divide the texts into six subgroups according to the text type or the domain they adhere to. We use 4-letter abbreviations to refer to individual texts, full titles may be obtained from the text headers. 

1.	legislation, texts pertaining to the Slovenian accession to the EU (ANX2, EKOL, EKON, KMET, PARL, SPOR, STRA, USTA, VINO). Domains: law, economy, ecology, agriculture. (170kW, 33%)
2.	political speeches of the Slovenian president Mr Kučan (KUCA). Domain: politics, diplomacy. (30kW, 6%)
3.	Linux user’s guide, glossary of computer terms (LIGS, GNPO). Domain: computers. (92kW, 18%)
4.	“1984”, novel by George Orwell (ORWL). Domain: literature. (91kW, 18%)
5.	Lek Vademecum – catalogue of medical products and drugs (VADE). Domain: pharmacology. (11kW, 2%)
6.	Slovenian Economic Mirror – a periodical publication (ECMR). Domain: economy, press. (116kW, 23%)

One of the initial assumptions was that legal, scientific or technical texts, i.e. terminologically “rich” texts, would exhibit a relatively high type/token ratio compared to non-technical texts. This was only partly confirmed by the statistics – texts (4) and (6), the novel and the periodical, are indeed significantly below the average, but so are some of the texts in (1) and the Linux manual in (3). It would seem that the type/token ratio cannot be taken as a reliable indicator of terminological richness, especially considering the fact that the texts were not lemmatized and that the ratio itself, though being calculated every 1,000 words anew, varies considerably according to text length. 

Comparing sentence length, the only text with notably shorter sentences is Orwell’s 1984. The largest difference between the Slovene and the English version can be found with the President’s speeches, where the translator perhaps felt that the political impact of his words would be better achieved through longer sentences in English.  The number of sentences in the Slovene and English versions of the texts shows little variation, proving that in a large majority of cases translators abide to the original sentence boundaries.

4	Word frequencies

In Slovene, approximately half of the word forms occur only once, in English only a third, which is again probably due to the inflectional nature of Slovene. As an attempt to evaluate the representativeness of the corpus, the word frequency list for the English part was matched against the BNC word list [6]. Among the first 50 items of both lists, as many as 39 matched, but moving further down there is little if any correspondence. Words like Slovenia, Linux, EU,  file and products are found among the first 100 in the ELAN list, whereas you won’t find Slovenia in the BNC word list until line 18,285, and a word like system has a frequency of 0,22% in ELAN but only 0,0047% in the BNC. 

Using a provisional stoplist for Slovene and English, frequency lists of lexical words were made for the whole corpus as well as for individual texts. In Table 2 we list 30 most frequent lexical words together with number of occurrences.

Word	Freq.	%	Word	Freq.	%	
LETA	988	0.19	SLOVENIA	1,872	0.3	
EU	844	0.17	SYSTEM	1,401	0.22	
LETU	754	0.15	YEAR	1,206	0.19	
VSE	751	0.15	ARTICLE	1,119	0.18	
SLOVENIJE	630	0.12	LINUX	984	0.16	
SKUPNOSTI	599	0.12	PRODUCTS	916	0.14	
SLOVENIJA	590	0.12	EU	912	0.14	
LINUX	589	0.12	USE	873	0.14	
ČLENA	588	0.12	EUROPEAN	840	0.13	
SISTEM	581	0.11	TIME	809	0.13	
DRŽAVE	572	0.11	NATIONAL	805	0.13	
ŠT	554	0.11	ECONOMIC	769	0.12	
SISTEMA	537	0.11	FILE	768	0.12	
SKLADU	527	0.1	PRICES	742	0.12	
RAST	484	0.09	COUNTRIES	732	0.12	
PODROČJU	469	0.09	MARKET	714	0.11	
ČLEN	467	0.09	COMMUNITY	691	0.11	
DATOTEKE	449	0.09	LAST	690	0.11	
SLOVENIJI	417	0.08	STATE	689	0.11	
DELA	392	0.08	NEW	682	0.11	
DRŽAV	388	0.08	DEVELOPMENT	671	0.11	
REALNO	387	0.08	PUBLIC	628	0.1	
UPORABO	386	0.08	REAL	617	0.1	
CENE	384	0.08	GROWTH	601	0.1	
DEJAVNOSTI	383	0.08	TERMS	598	0.09	
WINSTON	378	0.07	NUMBER	589	0.09	
EVROPSKE	376	0.07	EVEN	583	0.09	
L	358	0.07	PERIOD	574	0.09	

Table 2
5	Keywordness of individual texts
By matching a word frequency list of a single text against the reference list, i.e. the corpus frequency data, a list of text-specific keywords may be obtained. These are the words that characterise the domain of the text through their high occurrence relative to the whole corpus. 

Such lists can prove useful when studying the lexical specificity of a text or its terminological scope. Especially parallel keyword lists may represent the basic set for further identification of multi-word units or for multi-lingual concordancing. In the course of our analysis, a Slovene and English keyword list was produced for each individual text. Below we list just a selection from the first twenty items for three of the texts:

The Slovenian Economic Mirror
Slovene: letu, rast, realno, meseci, delež, plač, bruto, mio, EU, prihodki, proizvodnje, predelovalnih, BDP, povečal, povprečju,…
English: year, growth, compared, real, terms, increased, average, months, prices, total, rate, rise, period, manufacturing, wage, percentage,… 

Lek Vademecum
Slovene: mg, r, tablete, g, dan, zdravljenje, zdravila, Lekovit, kapsule, otrokom, trikrat, kapljic, kisline, kožo, bolnikih, izvlečka,…
English: mg, tablets, children, skin, acid, g, gel, disorders, patients, daily, vitamin, drops, capsules, drug, r, treatment, medical, Lekovit,...

Linux Installation and Getting Started
Slovene: Linux, sistem, datoteke, ukaz, IP, PPP, morate, ime, etc, DOS, namestitev, strežnik, diskete, imenik, uporablja,…
English: Linux, system, file, command, use, boot, software, installation, partition, drive, kernel, name, shell, disk, server,…

6	Multi-word units, collocations and clusters
Although most corpus workbenches provide numerous possibilities to search for frequent word combinations, this in itself does not suffice for efficient recognition and extraction of multi-word terms even from a monolingual corpus. Despite the fact that this is a widely acknowledged problem reaching back to the beginnings of computational-linguistic research, the idiomatic nature of language remains one of the basic issues in corpus linguistic still to be tackled. Clusters, idioms, set phrases or expressions, multi-word units and collocations all refer to the more or less fixed relations between individual words in a lexicon, and their proper treatment is crucial to many branches of linguistics, e.g. lexicography, syntax, translation.

The ELAN corpus consists mainly of texts that could be characterised either as legal or technical, and such texts tend to contain many multi-word expressions and terms. The structure of these units is frequently embedding, meaning that a multi-word term is composed of a multi-word term modified by a word or several words, or of several single or multi-word units. For example, the term exchange rate is embedded in floating exchange rate regime, or system installation and installation files are joined into system installation files. When compiling term databases or glossaries, it is often difficult to decide which of these combinations are indeed fixed and should be treated as a unit. 

Once a multi-word expression has been identified, parallel concordances can provide possible translations of the term (see also the section on translation equivalence). Some concordancing and translation memory tools (e.g. ParaConc, Atril’s DejaVu [7]) offer the possibility of automatic translation by searching through parallel concordances for the most frequent items, however for a multi-word unit like European Economic Community they would usually suggest individual words (Evropska, gospodarska, skupnost). Needless to say, this possibility is useless if a multi-word expression occurs only once in the whole corpus, and for ELAN it must unfortunately be assumed that the majority of them do. 

As it would have been impossible to analyse the idiomatic inventory of the corpus in its entire complexity, we decided to investigate just eight selected Slovene words and their behaviour in context, four of them with a frequency over 200 and the other four below. The following words were chosen: program, zakon, razvoj, proizvod, rast, varstvo, mreža, čustva.  For each word we examined the concordances and counted all the instances of the word occuring in a multi-word unit or fixed expression, which is represented as the percentage relative to the number of hits in the table below (Table 3). We also give the most frequent collocates or multi-word units. The table can be seen to indicate that multi-word units are all but rare in the ELAN corpus, or at least that some words tend to appear mainly in more or less fixed units or contexts. 

Word	Freq.	multi-word units (%)	Collocates	Most freq. Translation	
program	236	108 (45)	namestitveni~, nacionalni~, akcijski~, ~PHARE, ~varstva okolja	programme	
zakon	151	102 (67)	carinski ~, 
ustavni ~, ~določa, ~ureja, volilni~	statute, law	
razvoj	251	119 (47)	Agencija za kmetijske trge in ~ podeželja, gospodarski ~, Min. za ekonomske odnose in~, kulturni~, trajnostni~, ~programja, ~Linuxa	development	
proizvod	97	58 (57)	družbeni bruto~, bruto domači~, mlečni~, živilski~, ~po poreklu iz Skupnosti	product	
rast	445	294 (66)	gospodarska~, realna~, ~plač, ~produktivnosti, ~bruto plače na zaposlenega, ~izvoza, ~cen življenjskih potrebščin	growth	
varstvo	127	86 (67)	~okolja, ~človekovih pravic, ~osebnih podatkov, ~rastlin, ~konkurence, ~potrošnikov, zdravstveno~	protection	
mreža	21	7 (33)	računalniška~, lokalna~, merilna~	network	
čustva	14	2 (14)	čustva do Velikega brata	feelings	

Table 3
7	Translation equivalence
One of the primary motives for compiling parallel corpora is the study of possible translations of a word or phrase in a specific context. Although the notion of translation equivalence in terms of a symmetric mapping between languages no longer holds water in our contemporary understanding of translation, it is nevertheless a necessary and inevitable feature when it comes to terminological consistency and precision. 

For a translator, a parallel corpus can be a treasury of valuable information, joy and fun, providing numerous examples of successful and unsuccessful translation solutions, and thus cannot always be taken as the ultimate normative and reliable source, especially if it contains translations that were produced by non-native speakers of the language. 

To illustrate just one of the possible applications of a parallel corpus we focused on a certain group of highly frequent words which are taken to have a “fixed” translation, and looked for the cases where the word had not been translated with its fixed counterpart. Such words are cognates or internationalisms like system, information, culture or society, but also others that seem to have a fixed translation equivalent.

We found that this situation tends to occur when the word is being used as part of a multi-word term and indeed often indicates that the expression is a fixed terminological unit. Below we list some examples:

skupnost 	community
lokalna skupnost 	local government body
	~ authority
skupnost občin 	 association of municipalities

sistem 	 system
bančni sistem 	 banking sector
plačni sistem 	 wages policy
zdravstveni sistem 	 health service
sistem drsečega deviznega tečaja  
	floating exchange rate regime
sistem privatizacije 	 privatisation concept
uvozno-izvozni sistem 	import/export regimes
strukturni sistem za ribištvo  
	Financial Instrument for Fisheries Guidance
pravni sistem 	 legal framework
	legislation
prijava v sistem 	login
zagon sistema 	boot

8	One step further: automatic lexicon extraction

One of the intended applications of the ELAN corpus is the creation of bilingual lexical and terminological resources for translators, lexicographers and others. The first step towards this aim is the word-to-word alignment of the parallel texts and the lexicon extraction. Several interesting projects related to this issue are currently under way or have recently been finished, for example MARK-AlistEr [9], Plug [10], Twenty-One, Xerox etc. Among the software solutions that had been developed at various places, the one most readily available was the Twenty-One lexicon extraction tool developed by Djoerd Hiemstra, which can be downloaded from his homepage [5]. The extraction is performed by three different statistical algorithms. 

So far the software has only been tested on segments of the corpus, but already yielding promising results. The accuracy of suggested translations was between 45 and 65% for individual texts, but in interpreting these results it should again be noted that the texts were not lemmatized and thus all word forms were treated as distinct words. Several methods are being considered to improve the extraction of multi-word units, but this is already an issue of the following section.

Conclusion

As pointed out above, possible exploitations of the corpus will remain limited until it has been annotated with additional linguistic information. The first step will be the lemmatization, which should already facilitate tasks like lexicon extraction and the search for translation equivalents. The following phase would include POS-tagging, in order to be able to perform more complex queries, explore syntactic patterns etc.

Furthermore, we nurture the hope that the one million words is just a grain in what is to become a much larger pile of sand. Hundreds of pages are being translated in Slovenia every day, thus creating a next-to- endless pool of texts to be potentially included either in this corpus or perhaps within another similar project.

References 
1.	Erjavec, T. (1999) 
2.	Mike Scott's Web, Wordsmith Tools. http://www.ndirect.co.uk/~lexical/
3.	Barlow Monoconc
4.	IMS CQP
5.	Hiemstra, D. (1998) Multilingual domain modeling in Twenty-One: automatic creation of a bi-directional translation lexicon from a parallel corpus, In: Peter-Arno Coppen, Hans van Halteren and Lisanne Teunissen (eds.) Proceedings of the eightth CLIN meeting, pages 41-58. http://wwwhome.cs.utwente.nl/~hiemstra/
6.	Kilgarriff, A.: BNC Database and word frequenc lists. http://www.ndirect.co.uk/~lexical/
7.	Atril Translation Tools, http://www.atril.com/
8.	Telri Translation equivalence
9.	Markalister
10.	Plug
11.	Xerox
12.	Erjavec tm eacl