Map Of London

Woke up at 4am with a brainwave about improving the speed of the CodeSharing XQuery: build the queries as strings and use util:eval to run them. It took a lot of rewriting, because I now have to do much better sanitizing of inputs, but I was able to do some of that by using "castable as xs:NCName" and similar strategies, as well as regexes and quote-escaping, and I think it's good now. The eval approach works absolute wonders for speed; it's now usable on the live site.

I also found and fixed some bugs with namespace handling which were a little obscure.

My implementation of CodeSharing is too slow. I've built a brand-new eXist and put MoEML into it, and I'm working with it locally to test improvements.

After much pain and suffering, I eventually discovered that doing the transformation inside the XQuery with transform:transform is WAY faster than passing it on to a view in the controller. It's still nowhere near as fast as I'd like, but it's better; I now have a problem to solve related to the relative location of the stylesheet, but I'll get to that.

Added some more traps to catch usage of styles in born-digital docs, and fixed a whole pile of old encoding horrors from way back when.

Our data/utilities/diagnostics.xsl file was written prior to the decision to allow multiple pointers in pointer attributes, so it wasn't expecting them. I've tweaked it to remove that problem. Running it now shows a huge number of bad internal links, which I'll report to the team.

It turns out that the document root namespace does not get passed to the XPointer parser that is invoked as part of the expansion of XIncludes, so specific TEI element names were being read as in the empty namespace. Added a tei: prefix definition to the XInclude elements and used the prefix before every element, and now they work. Hurrah. Discovered this by looking at exist.log on my local version of the db, at AR's suggestion on the eXist list.

Altered the schema to allow XInclude in encodingDesc, and added an XInclude to every file with XSLT to point to the includes.xml listPrefixDef, so all XML file have it. This is part of splitting out the XML variants to allow a pure original XML version, a project-linked version, a standalone, and a TEILite. Fixed some validation problems in the process.

Managed to exclude a bunch of unwanted category info, and also to let eXist handle the XInclude expansions. Still to do: molgls expansion, inclusion of listPrefixDef in all files. Note to self: copy-namespaces="no" on xsl:copy in the identity transform template gets rid of unwanted xmlns declarations.

Even the egXMLs, which is something, EXCEPT that they trigger unwanted namespace prefixes -- not invalid, just annoyingly unnecessary. I think this is really a Saxon bug, but I can't pin it down or work around it so far.

The mod rewrite on Apache which handles the old render_page.php URLs is occasionally sending horribly invalid "page names" such as CAST2&title=Castle Baynard/.htm to our webapp. We're working with RE on how/whether to remove or reconfigure the mod rewrite, which is hindering as much as it's helping these days, because we really should be giving 301s for those pages; in the meantime, I'm trying to figure out a way to avoid the 500 server error generated by these URLs, but it seems to be raised before the data actually gets as far as my controller.xql code.

I also noticed a bug in the rendering which was providing a "Documents mentioning this topic" link even on the missing.xml 404 page, which is ridiculous. I've now fixed that.

Continued work on TEILite, which is close to complete (haven't tried Stow yet!). In the process, found an oddity in encoding which I fixed a couple of examples of, and passed on to KMF on Asana to fix the rest and document.