Mariage

Worked on the search. Decided to take a different approach from EMLS, and write the find operation (in find.xq) in such a way that it pulls back full documents in a corpus, for the XSLT to deal with. Got the basic find working -- documents are retrieved based on the criteria in the dropdowns combined with any text search. Searches for any word and all words use the eXist functionality (&= and |=), and therefore result in <exist:match> tags which can be used for highlighting, but searches on exact phrase use the xpath contains function and therefore can't be highlighted; I don't know a way around that.

In the process of working on this, I began to re-think the problem with innerHTML, and it occurs to me that we could use the responseXML property of XMLHttpRequest instead of the responseText. The former is supposed to return a DOM document, so we could use standard DOM methods to move its children into the page DOM, thus avoiding innerHTML, which presumably means the operations should work on Safari even if the page is served as application/xhtml+xml. That would be grand. Additionally, I had a DOH moment when I realized that the display issues caused by that mime type are simply due to the CSS not having obvious properties set (such as display: block for p, ul and li tags). This opens the door to running the whole site as pure XHTML (served with the right mime type), which would be the best option by far, if we can make it work.

Following confirmation from Claire, I edited the <sourceDesc> tags of the three texts discussed in this post. This is the approach I've taken:

All three texts appear to have been published originally as pamphlets (i.e. standalone documents) in Paris in the 17th century, then collected and published in anthologies in the 19th. For each document, then, I've created two <biblStruct> elements, one for the original publication and one for the anthology, like this:

 <sourceDesc>
                <biblStruct>
                    <monogr>
                        <title>Sermon pour la consolation des cocus</title>
                        <author><name>anon.<reg>anon.</reg></name></author>
                        <imprint>
                            <pubPlace>Paris</pubPlace>
                            <date value="1624">1624</date>
                        </imprint>
                    </monogr>
                </biblStruct>
                <biblStruct>
                    <analytic>
                        <title>Sermon pour la consolation des cocus</title>
                        <author><name>anon.<reg>anon.</reg></name></author>
                    </analytic>
                    <monogr>
                        <title>Le Bibliophile fantaisiste ou choix de pièces désopilantes de rares</title>
                        <imprint>
                            <publisher>J. Gay et fils</publisher>
                            <pubPlace>Turin</pubPlace>
                            <date value="1869">1869</date>
                            <biblScope type="pp">359-70</biblScope>
                        </imprint></monogr>
                </biblStruct>
            </sourceDesc>

Please let me know if this is misleading or incorrect. The three documents are arrest_contre_les_chastrez response_des_servantes and remede_pour_le_cocuage.

After uploading all the updated images and image files, I was puzzled to find that one image markup file wouldn't display properly in the XHTML view. After some investigation, it turned out to be due to invalid XML. Many of the annotations included <l> tags not enclosed in <lg> tags, and <lg> tags with no content. After fixing all these errors, the file now works OK.

As I work on the search page, I'm using XQuery to harvest lists of data to use for drop-downs (author names, text types, and so on). In the process of doing this, I looked a bit more closely at some of the markup and found a few oddities, such as image file names with spaces, and links to old schema files. I've cleaned these up, and posted the resulting files back onto the server.

I also built test versions of the interactive images for each of the image markup files, in the process generating the smaller versions of the images required for the Web pages, then uploaded these, along with the original files, into the scans folder. Then I checked all of the XML files (for texts and images), and uploaded them into the database.

A problem has emerged with the <publisher> tag. Some texts seem to have much more information in the publisher tag that should really be there. These are the ones that I think need fixing:

• arrest_contre_lest_chastrez.xml shows <publisher>Iouxte la coppie imprimée à Bourdeaux pour Anthoine Barre, Avec permission.</publisher>. Should this just be "Anthoine Barre"? The full text shown here looks like it should be in a <respStmt> tag, or possible a <note>.
• response_des_servantes.xml has <publisher>Paris, 1636; Rpt. dans Variétés historiques et littéraires. Recueil de pièces volantes rares et curieuses en prose et envers. Tome III. Ed. E. Fournier. Paris : P. Jannet, 1855 : 101-08.</publisher>. This looks like the entire bibliographical information entered in the publisher tag instead of in the individual tags where it belongs.
• sermon_des_cocus.xml has <publisher>Dans Le Bibliophile fantaisiste ou choix de pièces désopilantes de rares. Turin : J. Gay et fils, 1869 : 359-70.</publisher>. Again, this looks like full publication info in the publisher tag.

I can fix these, but I'll need the go-ahead from Claire.

Created a basic search form. I still don't really have a clear idea of what type of things people will want to search for, so it's basic at the moment.

In the process, I had to revisit the xslt2/xhtml issues discussed earlier, because although these are (basically) solved for the display of documents and images from the collection, the site meta-pages (Home, TOC, Search, Instructions etc.) still use the old XSLT 1.0 system. The solution of constructing all HTML elements using the <xsl:element> constructor is not optimal here, because these pages have large blocks of raw HTML code which is included using an <html_block> wrapper element, the contents of which are copied to the output. Elements copied like this end up with the infuriating xmlns="" attributes.

I don't have a solution for this yet, but I do have a good, clear test case with this site, and because none of these XSLT stylesheets make any use (yet) of xpath-default-namespace, we can try using that, and figuring out what's really happening with namespaces and mime types.

Met with Claire to talk about the display of titles in the TOC. We agreed that, for the purposes of consistency, there needs to be a short, title-case title for every text, which is supplied by the editor. This can be added to the teiHeader as <title type="toc_display">Advis salutaire</title> or something similar. Then these can be used for the TOC page, but not for the documents themselves.

Solved a number of problems, leading to successful rendering of all documents on the new site (which is here).

The first problem was the non-display of any content when using the strip-namespaces stylesheet at the end of a pipeline. This stylesheet is, in any case, a kludge, arising from the difficulties we've had making various xslt engines behave reliably with regard to the output of unnecessary or unwanted xmlns attributes. I removed that from the pipeline, and found a way to stop the attributes from appearing in the first place:

• Use XSLT 2 and Saxon 8.
• Include xmlns="http://www.w3.org/1999/xhtml" in the root element of all stylesheet files included in the transformation process.
• Output all html elements using the xsl:element constructor; don't use literal output.
• Us xsl:output method="xhtml", with an xhtml doctype and xpath-default-namespace, like this:

  <xsl:output method="xhtml" doctype-public="-//W3C//DTD XHTML 1.1//EN" 
  doctype-system="http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"
   xpath-default-namespace="http://www.w3.org/1999/xhtml" />

• Construct the root element with a namespace attribute, like this: <xsl:element name="html" namespace="http://www.w3.org/1999/xhtml">
• Use xpath-default-namespace="http://www.tei-c.org/ns/1.0" wherever you want to avoid having to use the tei prefix. This is only slightly more convenient; actually, it might be better to be explicit about prefixes when matching, as I have been in most places. I should test the results of removing these calls.
• Serialize as "xhtml11_compat". Unfortunately, if we use the Cocoon xhtml11 option, even Firefox fails to render it quite right (ignoring the background image). This needs some diagnosis, but it isn't major. It's a little odd, because the rest of the site (not documents from the db) works fine serialized through xhtml11, but then again it's being produced by a different block of XSLT, which is 1.0 and processed by Xalan.

At this point, the whole site is working and I can go on to constructing the search system.

Got the permissions issue sorted out -- it was one directory which didn't have the group write bit set, so maybe the sticky bit isn't working.

Ported the stuff into the webapps folder, and began testing. All output which depends on XSLT 1.0 is working (the main site, contents, etc.). However, XSLT 2.0 transforms died because the saxon transformer wasn't set up and named consistently. With Greg, went back to the main Cocoon sitemap and another conf file (Greg, which one?) to make sure that the Saxon 8 transformer is consistently referred to (name and role) as "saxon".

Following this, transforms would run OK, but fail because of ambiguous matching in the teiGeneral.xsl file. Found some almost-duplicate templates and eliminated them. Now the transform runs, but there is no result. I've stopped at that point for today: no errors, but empty documents.

Then I tested the text-analysis output, and found that both XML and text were being served with the wrong encoding. Worked with Greg to make the default encoding for text and XML serializers UTF-8 in the main Cocoon sitemap. With this, I was able to comment out my extra serializers from the Mariage sitemap.

That's it on this problem for today; tomorrow we get stuck back into the XSLT for the documents.

Began by creating a new copy of all the data, so the old site can continue to function. The new one is mariage_2. Then I uploaded all the current xml files into the db without problems. Next, I went to create the webapps folder, but found I was unable to do anything in the mariage account on home1t because of permissions issues. This was what stopped me yesterday, and it was supposed to be fixed. So now I'm back negotiating with Ryan to figure it out. Grrr.