Nxaʔamxcín (Moses) Dictionary Blog

Delighted to hear that the affix file is not too large to be worked on.

Worked on the affix.xml file today. I am now at about line 24,000 out of 29,000 lines.

Completed several large entries, including applicatives and reciprocals. Checked Willett (2003) for some definitions and rewrote some definitions and added some notes explaining interpretations. Some merging of forms from the applicative entry into other entries (causative and L-applicative) needed to be done.

Next task: to merge -u and -tu, and to work on -ul.

Hi Ewa,

It turned out the file problem was simply my fault; I was uploading a file which wasn't valid because of some old code being left in it. I was thrown by the bad error message from eXist -- it seems that whenever it has any kind of problem indexing a file, it reports a network error. Once I'd fixed the file, it went into the DB OK, so the latest complete affix entries should now be available on the Entries page.

May 7, 2007
Martin, as you point out the affix file is big. The reason for the size is the following: at the moment in going through the affix file, I am just focusing on two things 1) filling in the feature structures, and 2) fixing up big editing problems like the cross-reference problem and the merging of entries.

When you wrote the conversions for the files, you left in unconverted forms with a comment that says: “The following list of elements is copied from the source doc for information and checking purposes.” Because I am not doing most of the detailed editing on the entries at this stage (since that would take a very long time, and we though that just focusing on feature structures would be sufficient for now), I am not deleting the unconverted forms as I go through the affix file. Taking out the unconverted forms would make the affix file much smaller. So for your purposes maybe we should make a copy of the affix file which has all the unconverted forms deleted?

I will be finished the current pass through the affix file by the end of this week, I hope. At that point we could make a shorter version of the affix file for you to work with.

After I finish the affix file, I thought I would return to working on the phargyngeal file since it is fairly small, and has already been worked on a bit. If I could finish that fairly soon, you would have a second non-affix file to work with. The editing work is more time-consuming than we had expected it would be because it is turning out that I need to do quite a lot of manual input of phonemic representations and file-merging, cross-references, etc. We had not anticipated this before I began working on the affix file.

ps Thanks for the AutoP info.

3. Editing Decisions concerning Conflation of Entries in Affix.xml file: As I have been going through the Affix.xml file, I have been making editing decisions about entries. Specifically, throughout this file there are entries which Kinkade listed as separate suffixes on his filecards, but which in fact are either 1) allomorphs of one affix, rather than affixes in their own right, or 2) combinations of two affixes which Kinkade analyzed as a separate affix.

In the first case, an example is the ‘inchoative’ affix has two allomorphs -?- and -p. These were listed as separate affixes in Kinkade’s affix filecards, and were therefore typed into the computer database as separate affixes. However, Kinkade himself did consider these to be allomorphs of one affix. Similarly with ?ac-/c-. With cases of this kind, I have listed both forms of the affix under one xml:id entry, but have indicated that there are allomorphs of the affix. And I have also placed all the illustrations of all the allomorphs together into one entry. Thus, for instance, all -?- and -p cases are in one xml:id entry.

In the second case, an example is words containing the causative morpheme -stu. In addition to the fact that this morpheme has different stressed and unstressed allomorphs, it is also found in combination with other suffixes. For example, -stu-m occurs in the data, as does -stu-n(n). Kinkade listed both -stu-m and -stunn as separate suffixes in the filecards, but his later analyses indicate that he perceived these sequences as combinations of affixes. With cases of this kind, I have moved the illustrations that Kinkade had for -stum and -stunn into the -stu xml:id entry. I have noted that I did this in the database itself, just to keep track, but these notes can be erased eventually.

I don't think this requires any action from me, although we'll need to look closely at what's happening to those entries in the Web page. To that end, I just downloaded the latest affix.xml file in order to push it into the database, but it has some problems. One is this:

<dicteg>
 <cit>əxʷ [TEXT IS NOT ALLOWED HERE OUTSIDE THE quote TAG]
 <quote>(s)‐n̩‐√mèy=ap‐m̩én‐ct<gloss>confess</gloss>
  </quote>
  <bibl><!--[No source]--></bibl>
 </cit>
</dicteg>

It looks like this pretty much shows the point you've reached in your editing, so I created a temporary file including only the entries up to that point, and tried uploading it to the database. I hit a file size limit problem when doing that -- an issue we've occasionally encountered before, and which Greg and I are working on -- so at the moment I can't put the file in the db. It does seem to me that this file is getting quite big, though, so I'm wondering if it would make things more practical if we split this file up into three or four sections?

One last thing I've been meaning to mention about posting to the blog. If you look to the bottom right of your screen in the blog entry editing page, you'll see a section headed "Text Renderers". In there is a checkbox labelled "Auto P". If you check that checkbox, you'll find that your carriage returns turn into actual paragraph breaks, so your text is not all run together. When I see your entries which are run together without breaks, I've been clicking on Edit, checking that box, and saving again, to introduce the page breaks. If you're wondering why that's off by default, it's because most of us using the blog tend to use HTML tags as we type, and do more complicated formatting, and the Auto P would interfere with that.

1. Cross-references: I went back and checked cross-reference examples. There is no difference between a link introduced by xr ‘See’ and one introduced by ‘cf’. So the second cross-reference type should be re-encoded using an xr tag. Do you do this, Martin, or do I?

I don't think there's any way to do this programmatically, because the <note> elements which use "cf." are discursive; there's no way for any algorithm to parse them and figure out what entry is being referred to. So it's a markup task, and as such I think it falls to you. You can assign it to me if you'd prefer, but it will take me a while to get to it, and I'll probably just have to ask you what to do in each case.

There's a simple way to find all <note> elements containing the text "cf" in oXygen:

1. Open the XML markup file in oXygen.
2. In the top left of the screen, you'll see "XPath 1.0" or XPath 2.0". Use the dropdown list to select "2.0".
3. Type this into the text box next to it:

   //note[contains(., 'cf')]

4. Press Return.
5. You should see a list of items in at the bottom of the oXygen screen. Click on each one to show it in the editor.

Doing this shows 8 items in the affix.xml file, and none at all in the s-rtr.xml, c-rtr.xml or phar-w.xml files.

Incidentally, this raising a question from me: which files are "finished" to the point where they should be uploaded into the database?

2. Martin’s question about lack of documentation of phr type=”phonemic/narrow” which are found in dictegs: We discussed this in December. See Blog of 18/12/06 Decisions. In Kinkade’s database some of the illustrations are transcribed phonetically and some are transcribed phonemically. So we decided to add phr type=”phonemic/narrow” in order to allow for phonemicization of the dicteg forms when necessary.

Mea culpa, and thank god for the blog. I've now updated the documentation to reflect this. I've also done a quick update to the rendering code so these are rendered between slashes and square brackets as in the case of <seg> elements in the <pron>.

May 3, 2007

1. Cross-references: I went back and checked cross-reference examples. There is no difference between a link introduced by xr ‘See’ and one introduced by ‘cf’. So the second cross-reference type should be re-encoded using an xr tag. Do you do this, Martin, or do I?

2. Martin’s question about lack of documentation of phr type=”phonemic/narrow” which are found in dictegs: We discussed this in December. See Blog of 18/12/06 Decisions.
In Kinkade’s database some of the illustrations are transcribed phonetically and some are transcribed phonemically. So we decided to add phr type=”phonemic/narrow” in order to allow for phonemicization of the dicteg forms when necessary.

3. Editing Decisions concerning Conflation of Entries in Affix.xml file:
As I have been going through the Affix.xml file, I have been making editing decisions about entries. Specifically, throughout this file there are entries which Kinkade listed as separate suffixes on his filecards, but which in fact are either 1) allomorphs of one affix, rather than affixes in their own right, or 2) combinations of two affixes which Kinkade analyzed as a separate affix.

In the first case, an example is the ‘inchoative’ affix has two allomorphs -?- and -p. These were listed as separate affixes in Kinkade’s affix filecards, and were therefore typed into the computer database as separate affixes. However, Kinkade himself did consider these to be allomorphs of one affix. Similarly with ?ac-/c-. With cases of this kind, I have listed both forms of the affix under one xml.:d entry, but have indicated that there are allomorphs of the affix. And I have also placed all the illustrations of all the allomorphs together into one entry. Thus, for instance, all -?- and -p cases are in one xml:id entry.

In the second case, an example is words containing the causative morpheme -stu. In addition to the fact that this morpheme has different stressed and unstressed allomorphs, it is also found in combination with other suffixes. For example, -stu-m occurs in the data, as does -stu-n(n). Kinkade listed both -stu-m and -stunn as separate suffixes in the filecards, but his later analyses indicate that he perceived these sequences as combinations of affixes. With cases of this kind, I have moved the illustrations that Kinkade had for -stum and -stunn into the -stu xml:id entry. I have noted that I did this in the database itself, just to keep track, but these notes can be erased eventually.

Now the site has been fully ported over to Lettuce, the old URL on Mustard is obsolete. I added a redirect to the Mustard sitemap pointing to the new site on Lettuce.

Nx has a specific sort order for its alphabet, which doesn't follow the instant results you would get from a normal alphabetical sort. For instance, the glottal stop comes at the beginning, and the reverse glottal at the end; a normal default XSLT alpha sort, as well as the Unicode codepoint sort, place the glottal after the y.

In order to get sorting to work the way we need it to, we'll have to define a custom sort collation. Since we're using XSLT 2.0 and Saxon 8, we can do that following the instructions on the Saxon site for implementing a collation sequence. What we have to do is to write a Java class which implements the java.util.Comparator interface. This IBM page has more details of the interface.

Because Java isn't my thing, I'm now looking around for a simple example I can modify. So far I've only found this, which is too basic to be much use. This shows a simple example of a case-insensitive comparator:

public class CaseInsensitiveComparator 
 implements java.util.Comparator {
  public int compare(Object o1, Object o2) {
    String s1 = o1.toString().toUpperCase();
    String s2 = o2.toString().toUpperCase();
    return s1.compareTo(s2);
  }
}

but it shells out to the default string comparator to do the real work. What we need is an example of an efficient way to compare individual characters. One possibility is to have a string comparator which also embodies a character comparator; the character comparator would tell you which of two characters sorts first, and the string comparator would call that for each character in the strings until it found an inequality. I have no idea if I'm on the right track here, though, so I'll keep researching for the moment. This is something that will be called frequently, so it has to be pretty fast.