Carrier dictionary sequencing
Posted by mholmes on 28 Jan 2011 in Activity log
My original approach to sequencing the Carrier dictionary was to create a Java collation, but it occurred to me when I looked at the actual data and the proposed sequence that this could easily be done in XSLT, using replacements. My strategy has been to remove all accents, then replace key pairs of characters with other pairs that will sort in the right location relative to existing data, and then to remove all apostrophes. This, as far as I can tell, gives the right results, although I'm still waiting for confirmation from the FV and Carrier folks. Here's the XSLT that does it:
<xsl:function name="mdh:tweak" as="xs:string">
<xsl:param name="inString" as="xs:string"/>
<xsl:variable name="output" select="$inString"/>
<xsl:choose>
<xsl:when test="string-length($inString) gt 0">
<!-- Replace all accented vowels with their unaccented equivalents. -->
<xsl:variable name="accentsGone" select="translate(normalize-space(lower-case($inString)), 'áéíóú', 'aeiou')"/>
<!-- Get rid of all combining underscores (u+0331 and u+0332).-->
<xsl:variable name="underscoresGone" select="replace($accentsGone, '̱|̲', '')"/>
<!-- Now before we remove the apostrophes, we need to replace some pairs of letters that need to sort as if they were one.
We can use a character following z to replace a character that needs to sort after all the rest. For example:
kh needs to sort after ka, kb, kc, kz, so we can replace kh with k{
So we do:
g becomes {
h becomes }
l becomes ~
o becomes ¥
s becomes ¦
w becomes §
z becomes ©
-->
<xsl:variable name="gReplaced" select="replace($underscoresGone, 'ng', 'n{')"/>
<xsl:variable name="hReplaced" select="replace($gReplaced, '(c|g|k|l|s|w)h', '$1}')"/>
<xsl:variable name="lReplaced" select="replace($hReplaced, '(d|t)l', '$1~')"/>
<xsl:variable name="oReplaced" select="replace($lReplaced, 'oo', 'o¥')"/>
<xsl:variable name="sReplaced" select="replace($oReplaced, 'ts', 't¦')"/>
<xsl:variable name="wReplaced" select="replace($sReplaced, '(g|k)w', '$1§')"/>
<xsl:variable name="zReplaced" select="replace($wReplaced, 'dz', 'd©')"/>
<xsl:variable name="aposGone" select="replace($zReplaced, '''', '')"/>
<xsl:value-of select="$aposGone"/>
</xsl:when>
<xsl:otherwise>
<xsl:variable name="noTerm">__NO DAKELH TERM IN THIS ENTRY.</xsl:variable>
<xsl:value-of select="$noTerm"/>
</xsl:otherwise>
</xsl:choose>
</xsl:function>
I can sort the actual records on the Dakelh Term passed through this function.