Nxaʔamxcín (Moses) Dictionary Blog

Met to work through the spec which SMK has created for the morpheme index sorting and nesting; it's extremely complicated, so it will have to wait till I get back before I start implementing it, but looks rigorous and doable.

SMK adds:

The latest spec is in moses/trunk/docs/Action/PrintDictionary/SpecsForRootBasedIndex.odt. A sample sorting of entries with the root ḥawˀiy is in SampleOrderOfSubentries.odt.

MDH can test the process on the files with status values of complete, edited, and light-edited.

Reminder to self on how to generate dictionaries:

• In Oxygen, select all the XML files in the dictionary directory and run the transformation moses_auto_orthography. This generates all the files with automatic orthographical forms in dictionary_test.
• Open the file utilities/generate_names-only_dictionary.xsl, and run it on itself. This uses the files from dictionary_test to create the smaller XML files in the names subfolder.
• Select the XML files in the names subfolder and run the transformations moses_xml_to_pdf_LEARNER_include_names and moses_xml_to_pdf_LINGUIST_include_names. This generates the complete set of pdfs in the names/pdfs subfolder.

A couple of days ago I ran this to get a candidate list of erroneous defs which should have been quotes:

xquery version "3.0";
declare default element namespace "http://www.tei-c.org/ns/1.0";
declare namespace util="http://exist-db.org/xquery/util";

for $t in //TEI
for $e in $t//entry
let $quoteSegs := for $s in $e//quote/seg where
string-length(normalize-space($s)) gt 0 return translate(normalize-space($s), '*', ''),
$defs := for $d in $e//def/seg where string-length(normalize-space($d)) gt 0 return translate(normalize-space($d), '*', ''),
$dupes := distinct-values($quoteSegs[.=$defs])
where count($dupes) gt 0
order by util:document-name($t), $e/@xml:id
return concat(util:document-name($t/@xml:id), ' / ', $e/@xml:id, ' : ',
string-join($dupes, ' | ')) 

I've now written XSLT (utilities/flag_superfluous_defs.xsl) to insert a comment before each of these candidates.

ECH and SMK noticed a few more issues with rendering of <note>s in the Names dictionaries generated on 2013-10-17. See for examples names_only_with_related_linguist.pdf.

1) For entries with defs including the following,

<def>
<seg>
<note type="editorial">Meaning unclear.</note>
</seg>
</def>

... "Meaning unclear" is showing up twice in the pdf file - apparently once as the contents of the <seg>, and again as the contents of the <note>. It only needs to appear as a <note>.

2) Please indent entry-final notes to parallel the rest of the entry.

3) Please add more space between an entry-final note and the beginning of the next entry. At present, it's not clear in the printout whether an entry-final note belongs with the preceding entry or the following one.

Met to discuss the sequencing and grouping of related items in the print dictionary. Complicated but doable; SK will construct a large example tree, and we'll derive the rules from that. There's grouping and nesting, and then sorting within groups.

In some cases, the person entering data in the Lexware days used the same band name for glosses of example sentences as for defs, so we have some situations in which a def:seg duplicates data in a quote:seg. I've used this XQuery to identify candidate duplicates for SK to look at:

xquery version "3.0";
declare default element namespace "http://www.tei-c.org/ns/1.0";
declare namespace util="http://exist-db.org/xquery/util";

for $t in //TEI
for $e in $t//entry
let $quoteSegs := for $s in $e//quote/seg where string-length(normalize-space($s)) gt 0 return normalize-space($s),
$defs := for $d in $e//def where string-length(normalize-space($d)) gt 0 return normalize-space($d),
$dupes := distinct-values($quoteSegs[.=$defs])
where count($dupes) gt 0
return concat(util:document-name($t/@xml:id), ' / ', $e/@xml:id, ' : ', string-join($dupes, ' | '))

I ran some queries on where we have placed <note> elements thus far, and concluded that they should be rendered as follows in the print dictionary output.

1) Cultural, comparative, and editorial notes should appear in the print dictionary, and on the website. (Internal, referToEwa, and referToElders notes should NOT appear in the print dictionary, nor on the website.)

2) Render these elements immediately followed by all the (immediate child) <note>s they contain:

entry (some editorial notes that begin with a non-English word begin with a small letter)
form (some editorial notes that begin with a non-English word begin with a small letter)
sense
def
def:seg
cit
quote:seg

3) A <note> inside a <pron> should be rendered after its preceding-sibling <seg> and <bibl> pair. If there is no <bibl>, render the note after its preceding-sibling <seg>.

Similarly, a note inside a <quote> should be rendered after its preceding-sibling <seg> and <bibl> pair, or <phr> and <bibl> pair.

4) <note>s should never appear within these elements. (I have added this reminder to the Dictionary Editing Manual.)

div
pron:seg
hyph
xr

5) I also tried to standardize the formatting of <note>s to begin with a capital letter and end with a period. However ...

-Most comparative notes have the form <note type="comparative" resp="psn:MDK">CA. {a Coeur d'Alene word}</note>. They do not currently end with a period. MDH, can you please do this globally if you think it is necessary?

-There are 4036 editorial notes in the database, of which 1688 are "Meaning unclear." and 1560 are "Root entry added based on attested complex forms." MDH, can you help narrow down the finding and fixing of initial capitals and final periods in the other 788 editorial notes??

6) I have not tried to formalize or summarize MDK or EB's language in the cultural notes. I think we should render them in smaller type, and mention in the introduction that these are reproduced exactly as collected / contributed. This is consistent with what other dictionaries have done.

7) Delimit <note>s with « ».

Had long productive meetings today looking at a range of different print dictionaries and deciding that we like the organization and structure of the Klallam dictionary best; it also accords well with what we've done so far; all entries appear individually, while there are various other lookups (English, as well as morphologically-organized nested structures). SK is creating a detailed spec for generating the list of nested items organized by morphology. I have a list of minor tasks too:

• Regenerate name dictionaries to check they're now complete (done and confirmed).
• In print dictionary, show only notes which are comparative, cultural or editorial (basically done, except that there are too many contexts in which notes appear; we need to rationalize that, and we're in the process of doing it).
• Replace Search on main menu (done).
• Figure out why "Other entries" link doesn't appear when sense element is empty.
• When there are multiple form elements, add a border and margin to the bottom so they're clearly delimited.
• In the print rendering, where there are multiple forms, any after the first are fully outdented; they should only be partially outdented.
• These are the complete contents for the introductions for name dictionaries: Animal Names, Plant Names, Personal Names, Place Names, Tribal Names, Story People. Each of these needs to be turned into a complete TEI document ready for XInclude.

I just happened to notice that the hyphs for compound lexical suffixes were commented out along with the hyphs of monomorphemic entries. There are only five compound lexical suffixes, so I fixed them manually. They should look like this:

<hyph>=<m corresp="m:qin m:wil">qnwíl</m></hyph>

They were commented out because they only have one <m> tag within their <hyph>s, but they are actually bimorphemic, with two @corresp values.

Note to selves that "monomorphemic" doesn't mean just "hyph with a single <m> child", but rather "hyph with a single <m> child with only one @corresp value".

The audio files for the pronunciation guide were horribly noisy, so I've cleaned up all the mp3s and then regenerated the ogg versions. Then I rearranged the file structure to get the aup files and other detritus out of the way of the bits required by the website. Tedious job, cleaning up audio.