HCMC Blogs

Spent a lot of the day building, wrestling with, and testing a freshly-checked-out eXist. There are some detailed instructions here. After I'd got the webapp built and working, and the controller-config.xml settings right, I had some remaining problems caused by permissions (and even ownership) on some files being changed during the process of backup (from the old db) and restore (to the new one). Finally, all seems to be working well, and the problem of intrusive spaces caused by the HTML5 serializer has gone away, so we're ready to deploy first thing tomorrow.

Met with JJ to check that everything is good to go. She has outlined a new approach to categorizing contributors, contributions and responsibilities, which we're going to think carefully about. The objective is to ensure that different types of contribution and contributor are appropriately encoded in the data. This will be done with a combination of role identification in the personography and responsibility statements in the headers of files; the latter should also take care of capturing the status of a document (e.g. whether it's been reviewed or not). However, these features will be part of the next phase of the project, after we've worked through the experimental map port.

I'm still getting confused between the files dealing with the Search Website feature and the files dealing with the Search Database feature, so I've added some comments in to each to make it more obvious.
For the GUI of the site, I've:
- removed "Periodical Name" label and textbox from the advanced search form
- replaced "Publisher" with "Title" label in Periodical section of
advanced search form
- put the simple search interface (label, textfield and submit button)
above the advanced search form on the advanced_search page
- added javascript code in the form_advanced.php file so that if none of the elements in the advanced form have a value, then the user gets a popup alert and the query is not submitted, can extend that for any other kinds of input validation we may want
- replaced "Victorian Poetry Network Search Results" with "Victorian
Periodical Poetry Database Search Results" on the vpn-search-results page
- replaced "Search Results" with "Website Search Results" in the
search.php page
- At the bottom of the database search results, I've added a link back to the search page. I've also modified the database search results so that if you get 0 results the page just says "No results found" rather than "found 1 to 0 of 0 results".

If this is a new setup and you've never checked out eXist before, then:

• mkdir exist_trunk
• cd exist_trunk
• svn co http://exist.svn.sourceforge.net/svnroot/exist/trunk/eXist .

Otherwise just go into exist_trunk and do

• svn update

• cd eXist
• ./build.sh clean
• ./build.sh download-additional-jars
• cd extensions
• cp build.properties local.build.properties
• [Check local.build.properties to make sure FOP is turned on. It's turned on by default at the moment. **FYI: just checked out rev 15484 (Oct 29, 2011) and FOP is turned off in build.properties]
• cd ../
• ./build.sh

NOTE: if the build fails because of an error related to a Ant and Ivy, then download this file: http://repo1.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar and put it in eXist/tools/ant/lib/

• ./build.sh -f build/scripts/jarsigner.xml
• ./build.sh dist-war

Next, if you're going to store your XQuery and the rest of your application logic in the database, don't forget to edit WEB-INF/controller-config.xml to make sure requests in the webapp root are handled by your controller file. For example, I added this:

<!-- MDH: Application site configuration. -->
  <root pattern="/*" path="xmldb:exist:///db/site"/>

to tell eXist that my controller.xql lives in /db/site, and should be handling every request to the webapp from the root down.

One other fix proved to be necessary: my application serves images out of an /images/ subcollection, but there's a line in the default controller-config.xml which blocks the handler in my controller.xql. I had to comment it out:

<!-- MDH: Commented this out. -->
  <!-- <forward pattern="/images" servlet="ScaleImageJAI"/> -->

A detailed write-up of the information below has been added to the Guidelines document. For now, the following examples, where the attributes are emphasized, should suffice:

• For places: <placeName type="incomplete">Point Aitch Bee Cee</placeName>
• For people:<persName type="incomplete"> <surname>Andrews</surname>, <forename> J.</forename><forename> A.</forename></persName>
• For vessels: <name subtype="incomplete" type="vessel" key="archer">Archer</name>

Made a number of site changes and added new images on instructions from DR.

Staying on target to deploy MOL by the end of the week.

Today's progress:

• The "More Info" box is now more intelligent. It doesn't appear unless there's something to put in it.
• Obsolete JavaScript libraries and XML data files have been removed from the project and from SVN.
• A few new info pages have been added, including a new Technical Assistants page, which populates itself from PERS1.xml, with a new person category of "tech".
• The variant spellings AJAX feature is now working, and has been integrated into the search page.
• The Search Tips page has been fully updated to include information on all the Lucene-powered search options.
• The webapp has been tested successfully under Tomcat 6, which is what's running on the server. I'm using Tomcat 7.

Still to do:

• Approval by JJ.
• New checkout/build of eXist, and testing with that. It should solve the problem with spacing in HTML inline tag output, but we'll need to make sure it doesn't introduce any new problems.
• Create an editor user, and set permissions appropriately in the db.
• Creation of a sample full-scale teiHeader including responsibility statements, to act as a model for future page-creation.
• Creation of a stripped-down schema using oddbyexample.
• Completion of the site documentation.
• Detailed encoding guidelines.
• Instructions for using SVN (Windows, Mac and Linux).

Quick-and-dirty XQuery to generate stats for 1860:

xquery version "1.0";

declare default element namespace "http://www.tei-c.org/ns/1.0";

let $pbs := count(//TEI[substring(@xml:id, 2, 2) = '60']//pb[@n]),
$biblScopes := count(//TEI[substring(@xml:id, 2, 2) = '60']//biblScope[@type='startPageImage']),
$tot := ($pbs + $biblScopes)
return concat('Page-break tags: ', $pbs, '; biblScopes: ', $biblScopes, '; total: ', $tot)

xquery version "1.0";

declare default element namespace "http://www.tei-c.org/ns/1.0";

let $names := count(//TEI[substring(@xml:id, 2, 2) = '60']//name[@key])
return $names

Made some updates to the stats page (stats.htm) so that it displays more useful data about the state of completion of the documents.