LEMDO

Met with JJ to discuss priorities, which was very helpful. She laid out some ideas for documentation and structure; she's going to get working on some texts to help flesh out some real life encoding.

To do so, she needs some stuff from the TCP, which we've done before. Since we've needed to do this a few times, I've decided to automate the process slightly:

• First, I downloaded the JSON version of the TCP catalogue, and converted it to a much smaller XML file, deleting fields that we don't need
• Then, wrote another XSLT that resolves an STC number to a TCP number, if available, retrieves that TCP document and then passes it through the (existing) set of templates for converting TCP to LEMDO

I needed to regenerate the schema, but the TEI outage was causing issues. A simple fix for this was simply to switch the defaultTEIServer parameter and the defaultTEIVersion parameter in the XSLT. I've now codified this in the ANT as a set of conditions, so that, by default, we use the TEI server, but if that's down for whatever reason, then use the TEI Jenkins.

Extensive mtg with MH about LEMDO; mostly just catching up and came up with a plan for various copies (for lack of a better term) in collations, and sent an email about that. Also started to work through the issue 328 on Github, but ran into some issues with the git pull stuff (which I'll figure out myself)

Then, met with JM twice, to work through encoding challenges and to resolve some issues; fixed some of the code template stuff in the process and he updated his oXygen (he was running on 16). Worked through some encoding problems and some errors, including some pretty printing problems; we really should see if we ca disable that by default in a project.

JM sent a bundle of files for KNDW edition--including a selection of annotated primary sources, an OS text, a modern text, and an essay--for conversion, which I've now done. It was an interesting experiment: most of these documents were in older forms of IML, which we didn't necessarily know how to handle. So I've modified the IML conversion slightly to allow for an AB element, so we can fix hierarchy issues more easily.

Over email, discuss Stylesheet work with MH; he has created a branch and we're going to do a dive into the odd2lite stuff. This will be particularly useful for LEMDO, but will be broadly useful for any of the Endings projects

Biweekly meeting with JM and worked through a number of issues. Came up with plan for the next two weeks

• JM's SVN access isn't working at the moment, so he's getting me to merge his Ado_M [DONE]
• Rom_Q2M was out of date; reconvert [DONE]
• Convert Err for editors and add linebreaks to the OS as test case for bare display
• Start thinking about metadata display (discuss with MH and PS)
• Add schematron to use em-dash instead of double dash [DONE]

Added some new schematron for common problems that arise after conversion that have to be fixed by hand, including when there is embedded verse in prose (which could be done programatically but is safer to do by hand to confirm lineation, etc) and other common errors (unescaped angle brackets et cetera). The idea here is that if the texts pass the basic TEI conversion (which they basically all do), the converter must then check the text and ensure that they are valid against the LEMDO schema; if not, then the texts must be modified slightly by hand to fix things that require human intervention. It's a bit more involved, but it stops absent minded conversion (which I am guilty of).

Fixed a ton of small issues (mostly embedded verse, but some other small things) and converted Mac and 1H4 as per JM.

Long meeting with JM to discuss progress; discussed plans and priorities going forward. He also asked for a better display of overlapping annotations, which I agreed with; he wants everything that crosses multiple lines to be displayed as a line on the side rather than multiple types of underline. This is, of course, a rendering issue primarily, but we do need to get the algorithm in place to get the annotations and collations embedded in the HTML so that rendering decisions are easily handled later.

So, reworked the code so that if something goes over two lines, the behavior is slightly different. One way to resolve this would have been to create container divs, but decided against that since it's a lot of manipulation of the HTML; instead, we'll just add an onclick() event to the div itself and a left-border, and make sure the events don't propogate. Seems to be working alright; more work will likely need to be done, but it's mainly aesthetic at this point

I realized that the annotation matching code was a bit too generous with its selection of nodes, due to the preceding:: and following:: axis. I've now reworked it into something much more efficient and cleaner. The problem is that there are a number of different ways in which an editor might select a bit to annotation; the best way would be to point directly to an element (say an l or an lg), but most often editors want to gloss particular words or phrases using an anchor, so we have to handle both (and both in combination: say something starts at a line but ends halfway through another). So the solution to this is to first generate all the nodes that are between the two boundary points (using the incredibly fast << and >> selectors, which I've just learnt about) and then group them based off of ancestor nodes that were in that original selection of nodes. (That's not a very elegant way to describe it, but it's all documented heavily in the XSLT.)

This new process shored up a number of issues with the annotations. Many of them are aesthetic, some are functional, and some are processual. We'll have to have a set of diagnostics to check whether or not something that was annotated with anchors could be better annotated by pointing to a line or a line group. That won't be trivial, so that'll take some time