Nxaʔamxcín (Moses) Dictionary Blog

Much discussion over the last few weeks regarding the placing of gloss tags for generating the Eng-Nx wordlist. I attempt to summarize our conclusions here for future reference.

1) Why do we place inferred glosses (<gloss subtype=”i”>)?

At various times, we have placed inferred glosses for augmenting the search engine on the website, and for generating the English word list.

We concluded that from here on, we ONLY need to place gloss tags for generating the English word list. Inferred glosses do sometimes enhance the web search engine, but now that the stemming analyzer is in place, we don't need to do any further markup to help it out.

2) How should we tag inflected English words?

Until last week, we had been inferring the root word (or stem where relevant) when a def is an inflected or derived form of an English word, e.g.

<def>
<seg>he is <gloss>fattening</gloss> it up</seg>
<bibl corresp=“psn:JM”>JM 1.2.3</bibl>
<seg><gloss subtype=“i”>fatten</gloss></seg>
<bibl corresp=“psn:ECH”>ECH</bibl>
<seg><gloss subtype=“i”>fat</gloss></seg>
<bibl corresp=“psn:ECH”>ECH</bibl>
</def>

This encoding means that this entry will show up three times in the English-Nxa’amxcin wordlist: under fat, under fatten, and under fattening. This seems like overkill, especially when these three words will sort one after the other in the English wordlist anyway.

ECH and SMK decided we would like to see the “fat” entries as follows in the print dictionary:

fat: fat

fatten: fatten, fattened, fattening

fatty: fatty

To accomplish this, we need to reduce the number of gloss tags we place in each entry. Inflected English forms (-ed, -ing) should not be gloss tagged; only their root or stem should be gloss tagged.

So “fattening” would now be gloss-tagged as:

<seg>he is <gloss>fatten</gloss>ing it up</seg>

MDH confirmed that the search engine is ignoring gloss tags, so the stemmer will operate on <gloss>fatten</gloss>ing the same as it would on <gloss>fattening</gloss>. (That is, it will continue to return all results with the stem “fatten” when someone searches for fatten, fattened, or fattening.)

MDH has created two sample Eng-Nx word lists based on the 6 files with “complete” status, one using all the gloss tags, and one omitting the inferred gloss tags. They are in moses/trunk/docs/glosses. We concluded that we don't want to programmatically ignore the inferred glosses, because many of them – especially the synonyms – are worth including. But we can refer to these lists to identify the inflected English words whose gloss tags need to be revised.

3) How should we tag English phrasal verbs?

Where appropriate, English phrasal verbs will be enclosed in a single gloss tag - e.g, <gloss>go after</gloss>. This will allow us to organize the headwords in the Eng-Nx word list as follows:

go

go after

go down

go up

, etc.

4) How can we distinguish English homophones in glosses?

English homophones in glosses will be distinguished with a secondary word (or phrase) in an @n attribute on the <gloss> tag, e.g.<gloss n="conflagration">fire</gloss>, <gloss n="back of boat">stern</gloss>. These will then be rendered as follows in the print dictionary:

fire (conflagration):

stern (back of boat):

We decided not to use parts of speech for @n values. We will always use synonyms. We need to select synonyms that will be clear to readers in the community.

I have now disambiguated the English homophones listed here, and updated the Notes on Definitions and Gloss Tagging document accordingly. Where one homophone was far more common in the data than the other, I only added an @n value on the less common one - e.g. watch (wristwatch).

Did this through XSL with some cunning language-detection code based on content and context, and it seems to have worked pretty well. The Names page now uses the @xml:lang attribute instead of its own cruder detection code to build output.

As planned last week.

Meeting to review the presentation -- my task now is to collapse six slides which begin with the picture of the filecard box into a single stepped diagram illustrating the old encoding process and the horrible binary result.

Section 2 is now down to 6 slides, with more detail and more extensive notes.

Following Sarah's post, I've done the following:

• Added a language filter so you can view names only in English or Nxaʔamxcín. This is a crude regex, but it works because English names always begin with caps, and Nxaʔamxcín names never do.
• Turned off the traffic light display in the names page.
• Added more processing to the path, to handle rendering of e.g. choice elements inside names.
• Excluded lexical suffix entries.
• Elaborated the captions and links a bit.

Here are a few requests for the Names page on the website:

DONE -exclude Lexical Suffix entries

DONE -fix the display of sic/corr, so that only “Wenatchi” displays, not “WenatcheeWenatchi” (See for example the entry for “Sam George”.)

DONE -put flora (plants) and fauna (animals) in the link text at the top of the page

DONE -separate out the sorting into Nx-Eng and Eng-Nx pages. Ideally, users should be able to view the complete list, or any of the six lists by name type, sorted either by Nxa'amxcin name or by English name. The present setup with Nx and Eng names mixed together in the Name column is somewhat confusing. Continue to sort the Nx-Eng lists based on name tags in prons. Sort the Eng-Nx lists based on name tags in defs.

-For the present, exclude name tags in orths when generating these lists.

We've been running the live db with open access since the last time I rebuilt it, so in the process of doing other updates (such as rolling out the Java sorting collations) I've also added back the protection that we had before. In the process of doing this, I got bitten by the horrible eXist bug which enables you to lock yourself out of the admin account if you edit the admin user and forget to retype the password into the two password boxes (the effect is that you end up with a random admin password that you can never discover). As a result, I had to remove the server version of the app and replace it with a refreshed version of my local copy. This failed the first few times -- Tomcat tries to auto-deploy the app before it's completely uploaded the dbx files, so the uploaded .filepart files can not be renamed to overwrite the ones created by the live startup. It took two or three shots to get this problem solved. The only way seems to be to let it deploy, but stop it immediately in the Tomcat manager; then delete all the dbx, lock and log files; then upload them again; then restart it in the manager.

1) For the linguists' dictionary, we would like to see:

first phonemic representation in bold <orthography in angle brackets> [narrow transcription(s) in square brackets], for both forms and cits - e.g.:

ʔáyx̣ʷt <ʔáyx̌ʷt> [ʔáyəx̣ʷt]
√ʔáyx̣ʷ-t
1. be tired
2. tired, worn out

• √ʔáyx̣ʷ-tl kɬʔámnc
<√ʔáyx̌ʷ-tl kɬʔámnč>
[√ʔáyəx̣ʷ-t ləkɬəʔámənč]
he is tired of waiting (for you / me)

2) On the website, we would ultimately like things sorted by orthography.

This morning we decided that a simple and quick way to distinguish between homographs with different meanings is required to make the English lookup part of the dictionary less confusing. This will be achieved by adding a clarificatory word or phrase in the @n attribute of a gloss. Glosses will then be presented in the E-to-M view with this clarification in parentheses. Processing on the website will need to be changed to take account of this, and the print dictionary rendering will also have to be written with this in mind.