Map Of London

Various tweaks to XSLT and CSS

(Mins: 180)

Following a meeting at which we discussed strategy, and decided to focus for now on the Mayoral Pageants, worked with KMF on a range of minor display and rendering issues for primary source documents, including bylines, marginal labels, and text indents.

Beginning of tutorial for primary source encoding

(Mins: 180)

Started a tutorial based on SNOW1 (for the moment), and in the process of writing the first bit of it, came up against many annoyances in the rendering of egXML blocks; fixed those rendering issues (in three places, site, redesign, and codesharing. Grrr).

Added handling for dramatic text tags

(Mins: 30)

Added rendering handling for sp, speaker, and p within sp. The stage tag isn't handled yet. Rolled out changes both to site and to redesign codebases.

Troubleshooting: encoded title page of SNOW1, found and fixed rendering bug

(Mins: 120)

Since SNOW1 was a bit of a mess at the beginning, because of the encoders following obsolete examples, I've manually encoded the title page as an example.

Also found a problem with METR1 which was not really a bug, nor an encoding invalidity: a body element which goes straight to content (e.g. a head) with no intervening div is not invalid, but it triggered rendering problems because it was completely unexpected. As it happens, the encoding should not have been that way -- other divs appear later in the body -- but it wasn't technically wrong, so it would be good to figure out a way to prevent this through the schema or more likely through Schematron. We could change the content model of body so that it can only have divs, of course.

Fixes and updates

(Mins: 120)

Did some tasks from yesterday and some new ones:

Files that used <group> have now been converted to <div>s. (The only exception is stow_1633, which probably does need <group>.)
XSLT rendering has been updated to handle this.
Extra stray copies of METR1 have been identified in the db and removed. These were causing errors in the redesign pipeline.

Meeting and tasks

(Mins: 120)

I have these tasks coming out of the team meeting today:

DONE: Fix rendering of org popups.
DONE: Add Schematron constraint for malformed Julian dates.
DONE: Fix rendering of persNames with genName and roleName in them.
DONE (for group elements): Make a list of files containing group elements, and other bad old code.
DONE: Transform files with group elements into nested divs.
Add an attribute value parameter to the CodeSharing interface (will have to be done after July, probably).
Add handling for @style on list, along with documentation for it, change existing usage of list/@type to @style, then remove list/@type from schema.
DONE: Look at forme works in SNOW1 and figure out why they're not rendering properly.
Collapse the myth and fict personography types to a single type "lit". This will involve both data and rendering and must be done simultaneously.
Add rendering for sp, speaker and stage for SNOW1.
In redesign (with Pat): make page credits work like page TOC (pop-out rather than long list).

eXist build script

(Mins: 420)

I've spent the whole day working on getting a more flexible and successful build system for eXist. This is what I've added to Greg's script:

It now checks for the presence of Saxon and warns if it's not available.
It checks for three XSLT files, and in each case, if the file is there, it transforms a target file in the build tree. These are for conf.xml.tmpl, mime-types.xml.tmpl, and controller-config.xml. This should allow us to set up build environments for each of our specific projects.
It excludes XML Calabash and includes FOP. The former was blocking the build because its download location is down.

Found a number of problems with eXist, which I've reported, including a bad one once the webapp is running: you can no longer call transform:transform with a relative path to the XSLT file, otherwise you get an error. A full path from /db seems to work.

Wrote an eXist module for similarity metric comparisons

(Mins: 240)

I've now figured out how to create an extension module for eXist, following the instructions here. These are some things I've learned:

The only practical way to do this is to work with your module code in the context of the eXist tree, in $EXIST_HOME/extensions/modules/src/org/exist/xquery/modules.
You can use a non-eXist namespace -- I'm using http://hcmc.uvic.ca/ns/usm -- but it seems safest to use the eXist package structure, so my package is in org.exist.xquery.modules.unisimmetric.
All the extension modules are built together into a single jar called exist-modules.jar. You can build this jar alone, using build.sh extension-modules, then drop that jar into an existing eXist instance (although if the new jar was built with a substantially different version from the rest of the code, there could well be problems).

To turn on your module, you add a line to the conf.xml file like this:

<module uri="http://hcmc.uvic.ca/ns/usm"                        class="org.exist.xquery.modules.unisimmetric.UniSimMetricModule" />

along with the other modules.

I'm not yet happy with my module, and I'm still working on it. In particular, I'm not happy with the scores it's generating, and I think this might be something to do with other bits that get included in the GZIP stream, such as a header; if I can figure out how big those are, I can remove them from the calculation. The highest difference I seem to get is around 0.53 with completely dissimilar strings, so it seems as though the results are being compressed into a range much smaller than 0-1.

Meeting and work on page-image-linking

(Mins: 240)

Team meeting, at which we discussed the use of ISE's facsimile viewer in MoEML (which will be easy enough to do, although it's based on a traditional db, and we'll have to replace that with proper TEI facsimile encoding).

People also asked me to clarify how the EEBO linking works, so I've done that in the transcriptions documentation file, and I've also implemented the display of little page-images linking to the EEBO pages. Also, during today, <address> and <addrLine> were added to the schema, with some basic display rendering.

Implemented a crude similarity metric in XQuery

(Mins: 180)

Lucene-based fuzzy matching seems to be very broken in the build of eXist I'm using, and in any case it's based on Levenshtein distance, so I've implemented a crude version of the USM/NCD algorithm in XQuery. It's a long way from ideal, though, because it's using base64 versions of strings rather than compressing the actual strings (this is all I can do with eXist's exposed gzip access); using zip seems to be punitive because it would require creating a file on the filesystem or in the db and compressing that. I think a simpler approach would be to take my Java class and strip out all the command-line stuff it contains, then call that directly from XQuery (see the xqSearchUtils java project and the way it's called from the Despatches XQuery for an example). A jar file with a simple XQuery module interface might be very handy indeed.