Map Of London

Quick fix to a simple bug.

Added a range of dating elements and attributes to the schema, and wrote a section for the markup documentation explaining how to use them. Then wrote rendering code for handling them; this is already slightly complicated, but will get more so as we work out how we're going to handle Julian versus Gregorian dates.

Bi-weekly timesheet round-up.

Implemented the request for bibl items to provide a link which pops up a copy of themselves which includes a list of the articles which mention them.

Wrote some XSLT to do the following:

• Replace all instances of U+0027 in text() nodes with U+2019, following our editorial decision to that effect yesterday.
• Transform old oxygen processing instructions to xml-model PIs for the RNG schema.
• Add a PI pointing to the Schematron schema.

I've transformed all the documents with this, and validated them all. Flow has instructions for creating a Code Template in Oxygen to simplify typing of the U+2019 character. I've also added a constraint to the Schematron schema which enforces this.

The XSLT and Schematron were fiddly to get right because of the difficulties escaping the single-quote -- use-mention was hard to get around. For the record, these are the only approaches which seem to work (different for XSLT versus Schematron):

  <xsl:template match="text()">
    <xsl:value-of select="replace(., '''', '’')"/>
  </xsl:template>

    <rule context="text()">
      <assert test='not(contains(., "'"))'>
        "Straight apostrophe" characters are not permitted. Please use the 
        Right Single Quotation Mark (U+2019) character instead.
      </assert>
    </rule>

Changes and decisions are being tracked both in SVN and in Flow, so I'm not going to go into as much detail as I'd like to on the blog, but basically today we:

• Implemented many XSLT changes, some relating to the handling of punctuation following quotation marks. This now applies to the <quote> and <title level="a"> elements, but will in future also apply to <soCalled>, <mentioned>, and possibly others.
• Made a decision to use the U+2019 exclusively for apostrophes. This means creating a way to type it, which I've done using Code Templates in Oxygen and documented on Flow; a transformation to add it into all the documents, which I'll do tomorrow; and a Schematron rule to enforce it, which I'll add tomorrow (along with adding Schematron to the head of every file).
• Fixed some TOC rendering issues and normalized some document structures for complex documents that need TOCs.
• Found a bug in (I think) eXist, or maybe Saxon, whereby the @xml:id of the final child::text of a <group> is apparently inaccessible to XSLT. I don't know how or why this is the case, but it's making TOC rendering problematic. One solution would be to update the version of Saxon in the web application to see if that fixes it; if not, try the same algorithm in XQuery to see if eXist is really having an issue.

Fixed a bunch of issues in XSLT, including adding handling for page breaks, and making TOC generation proceed one level further into the <div> nesting.

I've made a number of schema-related changes:

• XInclude elements are now integrated into the schema.
• The content models of <listBibl> and <listPerson> now allow a single <xi:include> element instead of other content. This makes pages like sources.xml valid, where they weren't before. The expectation is that we will extend the use of <xi:include> into other areas, to allow the centralization of editorial and encoding declarations etc.
• A Schematron schema has been created, which imposes a restriction which may or may not be permanent: <milestone> and <pb> are not allowed inside <hi>. This is to avoid HTML invalidity because of the inclusion of HTML <hr> elements (generated for page breaks) inside <span> elements (generated from <hi>s). Some instances of this have been fixed in a few files, which have had the Schematron schema PI added to their headers. Eventually this will be done for all files, and we will use Schematron for the sake of convenience.

The new TEI release fixed the problem of <closer> in <postscript>, so only <list> in <signed> remained. I've now XSLTed that away, using simply linebreaks. There are still lots of tweaks to do to 1633 before it can go to proofing.

Following that, at JJ's request, I've used oddbyexample to generate a single master-schema for the project (london_all), to which all documents are linked. The ODD file is in SVN too, and there will be many upcoming fixes as we uncover new needs.

Added a tweak to sort biblio entries in the main sources page by their text content. Also debugged a problem where a person who had no @type on their <persName> tag was not shown as having any mentions in articles; I should make @type required on <persName>.

Made various changes to the Stow ODD file and schema in response to the first proofing session by JJ and SM. This will continue to evolve. Also created a char definition for s with tilde above ("shillings") for SM to incorporate into the file.

Finished most of the tasks from the meeting the other day.

I've generated a temporary ODD file and schema for Stow, using oddbyexample against the 1598, and the 1598 validates against it. This will enable SM to get going on proofing and editing that file. Meanwhile, we still have the issue of the invalidities in 1633, which will need some thought.

EDIT: I'm adding DONE as I caomplete each task.

Tasks arising out of today's project meeting:

• DONE: The page of @xml:ids should have descriptive text appropriately formatted. This is (I hope) just a question of apply-templates rather than value-of.
• IN QUESTION: It's not clear how or if this is best achieved: Entries in the personography, when rendered are accompanied by a "mentioned in the following articles" bit. This should also be available in bibliographical entries.
• DONE: Whenever "AGAS Map" is mentioned with quotes, the quotes should be removed. Multiple-file search-and-replace, I think, but care required.
• CB is creating a decision tree to help people decide the correct formatting for citations; help by turning this into a Quandary exercise.
• Show CB how to use regular expressions to retrieve all quotations (which are currently just using quote marks), so he can examine them with regard to planning migration to proper tagging of quotes.
• DONE (in fact, the Harlow example was due to an error in encoding, not processing): In person entries, even where there's no content for a person, make sure the "mentioned in" list appears. It's currently not appearing for e.g. Harlow.
• DONE: Bibliographical entries in the PERS file need to show up as popups, not links to the bibliography page.

CB had some problems with MOL becoming unresponsive after uploading files using the eXist admin client. I restarted MOL using the tomcat manager (took forever, but it worked - without restarting tomcat itself!) and things seem to be back to normal, but if CB uploads another file we end up in the same unresponsive state. While in that state, tailing the exist.log shows numerous entries like this:
DEBUG (BrokerPool.java [get]:1408) - waiting for a broker to become available
A quick google finds this page http://sourceforge.net/apps/trac/exist/browser/trunk/eXist/src/org/exist/storage/BrokerPool.java?rev=15806 - not sure if this is current enough to be useful...

Thoughts for resolution:
CB has had permission to upload for several days, and has had no problem until today.
CB changed his password in exist today. The two may be connected.
Is the problem is that CB changed the password himself? Would the problem occur if the admin user makes password changes for non-admin users?

NOTE: tailing /usr/local/webapps-dev/mapoflondon/WEB-INF/logs/exist.log shows that every request/response has a debug entry. Is logging set too sensitively for production?