Eesti lastekeele korpuse morfoloogilise märgendamise kitsaskohtadest

Abstract

Artikli eesmark on anda ulevaade sellest, mis raskendab avalikult kattesaadava eesti lastekeele korpuse automaatset morfoloogilist margendamist ning anda soovitusi, kuidas tulevaste korpuste margendamist ja standardiseerimist analuusi tarbeks paremaks muuta. Analuusisime korpust kirjakeelele moeldud morfoloogiaanalusaatori abil. Automaatse analuusi jarel vaatasime, kui suur osa sonadest sai analuusi voi jai analusaatorile tundmatuks nii alamkorpuste kui lapse- ja hoidjakeele loikes. Selgus, et analuusi saanud sonade osakaal igas alamkorpuses varieerus hoidjakeeles 94–98% ja lastekeeles 57–96% vahel. Suurt rolli mangib lindistuste uleskirjutamisviis: hoidjakeelt kirjutatakse ules kirjakeelele sarnaselt, kuid lastekeeles lahtutakse kuuldeortograafiast. Koik alamkorpused kull jargivad CHILDES-i transkriptsioonisusteemi ettekirjutusi, ent iga alamkorpus on koostatud erinevaid eesmarke silmas pidades ja on erineva transkribeerimisstiiliga, millest jarjepidevalt kinni ei peeta. Margenduse tulemust hindasime kasitsi labivaatamise teel. Artiklis toome valja, millised olid nii tundmatuks jaanud kui vale analuusi saanud sonade sagedasemad probleemid ja pakume voimalikke lahendusi. "Issues in morphological annotation of the Estonian child language corpus" This article presents the results of an initial attempt to automatically annotate the currently existing, publicly available Estonian child language corpus morphologically. CLAN software is not suitable for morphological analysis of Estonian, but Estonian language technology resources are available for written language and can be adapted to spoken language and specific genres. The automatic parser provided annotation for 92–98% of words in the child-directed speech and 57–96% of the child speech, with the results for child speech varying across corpora. A manual analysis was also conducted of words which were automatically annotated in a random selection of transcriptions from each corpus. Across corpora, 63–96% of annotated words were correctly annotated. Reasons for the variation are discussed, and obstacles to automatic annotation are identified at various levels. First, the corpora have been collected and transcribed with various goals and according to differing principles, hence the style and detail of transcription vary greatly across the corpora. Second, even within a single corpus, discrepancies appear in coding which need to be uniformly resolved in order to ensure accurate morphological annotation. Finally, for flagging non-standard or idiosyncratic forms, the implementation of metacodes available for use in the child language corpora would greatly assist the task of automatic morphological parsing. For each corpus, a user dictionary adapted to the particular genre and the particular corpus would need to be developed, including proper names and idiosyncratic words. The marking of errors is a crucial area which needs to be standardised in order to enable automatic annotation. Additionally, five groups of words which received inaccurate annotation were identified, and suggestions are made for transcription of child language corpora in order to ease the task of morphological annotation in the future.