Mariage

Eventually figured out that I had a corrupted exist-jars.xml file; after starting again from scratch, I got eXist and Cocoon to build. Then I followed these instructions to add Saxon (using the single-file SaxonHE release) and my own two jars, and to set up utf-8 handling. After that, Cocoon seemed to be running fine, but eXist is still MIA. Another note from WM instructed me to copy the ws-commons-util-1.0.2.jar from EXIST_HOME/lib/core into the generated COCOON/build/webapp/WEB-INF/lib. After doing that, I was able to get to the eXist admin web page, and create a collection; but the jars are still unsigned, so the webstart client won't run (although that also fails on a sitemap error before the signing error even happens), and trying to connect from a local admin client also fails. So nothing is usable yet.

I need to figure out how to sign the jars -- I think this requires a key, which I don't have, and I don't know what kind of key it is, or how to get one or generate one. If I can do that, then an alternative approach is available: I could use the repackaging strategy on eXist 1.2 with Cocoon 2.1.11, to get a working system with eXist 1.2. This would at least be a big step forward. 1.4 could wait on WM having the time to finish it.

WM reported that he'd made some changes in the source tree of eXist 1.4 to enable us to build Cocoon with it, and sent some instructions, which I tried to follow. It looks promising, but it's not quite working yet. Here are the steps I followed using Cocoon 2.1.11 and eXist 1.4rc:

1. Replaced the build.xml, exist-gump.xsl and exist-jars.xsl with the ones from here:
   
   http://exist.svn.sourceforge.net/viewvc/exist?view=rev&revision=10205
2. Downloaded a fresh Cocoon 2.1.11.
3. Patched the Cocoon blocks.properties with a local.blocks.properties which disables Lucene and querybean, per WM's instructions.
4. Tried to run the eXist build.sh:

       ./build.sh -f build/scripts/cocoon/build.xml

   I got several errors caused by 503s from the W3C server, which doesn't really want to deliver the XHTML 1.0 Transitional DTD which is referenced by the exist-jars.xsl file. This appears to be a standard response from the W3C servers, to avoid excessive traffic on those files.
5. Edited that file to point to a local copy of the DTD, and downloaded the DTD and .ent files into the same folder as the XSL file.
6. Tried to run a build again; this time it failed on a missing jars.xml file in the Cocoon tree, which I realized had been deleted by the previous attempt to build. I think it's initially read and removed, then replaced at the end of the process, but if the build dies in the middle you have to replace it.
7. Restored the Cocoon tree to its original state (leaving the local.blocks.properties in place).
8. Successfully built eXist.
9. Tried to build Cocoon, but it failed with this error:

    Warning!
     [ERROR] File core/avalon-framework-api-4.3.jar appears in the lib/ directory,
     but is not declared in lib/jars.xml.
   
     Please update the lib/jars.xml file to include the core/avalon-framework-api-4.3.jar
     file together with a description.
   
   : Fatal Error! Stylesheet directed termination
   home/mholmes/cocoon_with_exist/cocoon-2.1.11/tools/src/check-jars.xsl:161:40: Fatal Error! Fatal error during transformation Cause: Fatal error during transformation
   Failed to process /home/mholmes/cocoon_with_exist/cocoon-2.1.11/lib/jars.xml
   
   BUILD FAILED
   /home/mholmes/cocoon_with_exist/cocoon-2.1.11/tools/targets/validate-build.xml:72: Fatal error during transformation
   

Reported back to WM. I think we'll get there eventually, and I think this work is actually necessary in order to give us a Cocoon/eXist block with 1.4 (one of the standard distro formats).

This is similar to the errors I was getting previously. The file seems to be a standard part of Cocoon 2.1.11.

I've now marked up 27 items in the bibliography, and the patterns of markup are pretty much set. I've written a quick XQuery file to spit out the biblio from the db in XML format, with three sort options so far (date, author, and original order -- in the last, the original division headings are maintained). I need to add title sorting, which is going to require building in the link to my own Java sort library, and after that I can get into the rendering (which might just be automatic, using index.xsl, but will probably require one or two tweaks).

Starting about ten days ago, we began to experience mysterious errors when uploading XML data into the eXist db in the main Cocoon on Lettuce (this is the "cocoon" running in jakarta-tomcat, which hosts several major projects including Mariage, ScanCan, EMLS and the Devonshire MS. This is very worrying.

The problems appear to be caused by structural index corruption, a known issue with earlier eXist versions, but something that hasn't caused any problems for us before. The first symptom was that uploads to the db would fail with an error relating to proxy ids. The first couple of times it happened, we were able to restart Tomcat and solve the problem, but it's becoming more frequent. In addition, we now have a new problem, this one specifically affecting Mariage -- the search page fails to load because of this error: "Internal error: failed to store temporary doc fragment". Judging by posts on the eXist list, this is also related to corrupted indexes. Restarts haven't solved this so far.

This has sent Greg and I scrambling to figure out a strategy for migrating these projects to a more stable version of eXist. Unfortunately, there are a couple of stumbling blocks, which I encountered as I tried to build a new portable Mariage today. First of all, the current Cocoon+eXist distribution (with Cocoon 2.1.11 and eXist 1.2.6) has some unsigned jar files, so the admin client will not run; that makes it difficult to manage the database. An earlier version I used for ColDesp proves to have a bug which would also impoverish the Mariage search engine (when the ancestor axis is used in a search query, it fails to tag text matches with <exist:match> tags, which means the searches return only links to the documents, not the list of hits within the documents.

We're now pursuing another option, which is to build our own Cocoon 2.1.11 + eXist 1.4 package following the instructions here. If this is successful, we'll document any departures from the procedure described, and we'll also start building a new portable Mariage with it, and then try running it in the development Tomcat; if that works, we'll try putting that into the main Tomcat, and pointing mariage.uvic.ca at it. That would be a possible strategy for the highest-profile projects with their own domains (Mariage and ScanCan), and perhaps removing those from the old stumbling Cocoon will enable it to recover some stability.

I've spent all day fighting with this. I did, in the process, fix two small bugs in Mariage (find.xq needed to be updated to take account of the # (hash) characters preceding values of the @facs and @corresp attributes, and some of the style settings were pointing at old CSS files in one of Greg's own directories). But that's not much consolation for a completely broken search page.

LSPW pointed out that guillemets should have spaces between them and the quote itself, so I fixed that in the CSS. Also noticed that the <foreign> tag has (quite usefully) been used to mark foreign words, principally Latin and Greek, in the references, but that I wasn't handling it, so I've added some rendering for it (italics, basically), and modified the XSLT that handles the <head> tag in a reference to make sure it gets processed.

Spent a while figuring out the exact steps necessary to make quotations (<cit> / <q> in XML, rendered as <q> in XHTML) appear with guillemets instead of English-style quotes in the browser. For future reference, these are the steps I took:

• Checked that the <html> tag has xml:lang="fr" in it (it already had). We're not specifying any particular subset of French (such as -CA) because our documents span several centuries and at least two countries.
• Added this to the stylesheet:

  :lang(fr){ quotes: "«" "»" "‹" "›" "‘" "’" }
  

  Note: this will also work:

  q[lang |= fr] { quotes: "«" "»" "‹" "›" "‘" "’" }
  

• Added this to the XHTML <q> tag: lang="fr". Using xml:lang doesn't work. Note that you can't add lang="fr" to the root <html> element or to the <body> because it's invalid.
• This now seems to be working (in Firefox and Opera, at least; Arora (webkit) doesn't show the guillemets, so I'll need to check on Safari).

I've written the basic code for the collected references page on the site (XQuery, XSLT, CSS and JS). It shouldn't yet be linked from the menu, but the basics are there. I need to add some code for "find references to this in the text", but that will come later. In the meantime, LCC is working on cleaning up the existing references, and I'll be turning my attention to bibliographical data.

At CC's request, changed "Complet" to "Tous les documents".

When items were sorted by "type" on the TOC page, within each type they were in random order (or rather, apparently so; they were actually in db order). I've now added a second <xsl:sort> element in these cases, to sort the components of each type in title order.

There are 61 engravings, and the gallery pages them in sets of 15, but the 45-60 page lacked a "next" button to take you to the final item, #61. This was due to an off-by-one error in a line of XSLT in the gallery.xsl file.