TS will be joining us to work on the Despatches (in the Library position), starting tomorrow. He'll be working initially on learning TEI, and then on writing abstracts.
September 15, 2011
I have basic location display working, although I couldn't figure out a way to get locations to display as part of a subfolder using controller.xql; going back up to index.xql in the root proved impossible, as the controller always wanted to use an index.xql that was in the subfolder. I might not give up on this yet -- I'll ask a question on the eXist list about it.
It's becoming apparent that there are a number of problems in the XML that will need to be fixed. I'm trying to arrange a period of at least 2 weeks when no changes are made to the db, so that I can reorganize the structure, fix all the problems, and then get the new version of the webapp working. Two weeks will be very tight indeed, so hopefully I can get more, but I think we have to start from a thorough revision of the markup and the documentation of it.
On the plus side, it seems that the experimental map is based on layers of raster tiles; there's a single SVG file containing the base image and the other layers, and I think we can use Inkscape and ImageMagick at the command line to generate all the tiles we need for our zoom layers. We should then be able to use OpenLayers for the display, which obviates the need for any server-side code.
TIL that you can boost results by adjusting your lucene index to rank certain types of results higher than others.
Short version: add this: <text qname="placeName" boost="2.0"/> to your collection.xconf to double the relevancy of results marked up with 'placeName'.
Long version: see docs here.
A couple of minor updates done at DR's request.
After reading around a bit in the latest eXist docs, and finding this, I've written a page called backup.xql; when you hit the page, it will try to create an incremental backup and report success or failure. This is the key function:
declare function local:doBackup() as node()*{
try{
let $params :=
<parameters>
<param name="output" value="export"/>
<param name="backup" value="yes"/>
<param name="incremental" value="yes"/>
</parameters>
return
system:trigger-system-task("org.exist.storage.ConsistencyCheckTask", $params),
<p>Your backup has completed successfully.</p>
}
catch * ($errorcode , $description , $value ){
<p>Your backup failed with the following error information: {concat($errorcode, '. ', $description, '. ', $value)}</p>
}
};
The backup location ("output") is relative to WEB-INF/data, and it creates a zip file in there, named for the db, date and time.
After enabling XQuery 3 support in Saxon inside oXygen, I was hoping to see the XQuery above show as valid, but it doesn't; Saxon complains:
catch * ($errorcode , $description , $value ){ (:expected "{", found "(" :)
<p>Your backup failed with the following error information: {concat($errorcode, '. ', $description, '. ', $value)}</p> (: Variable $errorcode has not been declared. :)
Still, this stuff is a bit experimental, and it works OK, so I think it's a reasonable strategy to adopt. We should watch the documentation to see if it changes. It's going to replace util:catch, which is deprecated and will be removed for eXist 1.6.
September 14, 2011
Set up some more controller clauses to handle what I'm calling "info" pages; these are pages which were originally in PHP/HTML, but will now (after consultation with JJ) be in TEI XML like the rest of the data. They'll be stored in /db/data/info. I've written a Transformer script for converting the HTML, which is working pretty well; the only things I need to think about are the addition of nested divs to enable the proper use and rendering of <head>.
I've also written XSLT to convert these pages over to XHTML5, and spent quite a lot of time getting the page to validate. There were a couple of issues:
- The validator is unhappy with
<meta http-equiv="Content-Type" content="application/xhtml+xml;charset=UTF-8" />
at the moment; don't know if that's permanent or not, but I've commented it out. - The validator required this as charset decl:
<meta charset="utf-8"/>
- It rejects this as a valid
meta/@name:<meta name="resource-type" content="document"/>
. I've commented it out. - It gacked completely on the original code for a Facebook Share button, which is clearly invalid:
<fb:like href="" show_faces="false" width="450" action="recommend" font="arial" colorscheme="dark"></fb:like>
and also lacks an href, so I don't know what it would actually do. I replaced it with this alternative formulation which FB now offers:<div class="fb-like" data-href="mapoflondon.uvic.ca"></div>
So we now have working, valid pages being served out of the db, and also transformed inside eXist, all as part of index.xql, although every page gets its own filename ending in .htm, by doing this in the controller:
[...]
(: Regular site pages served from the root with .htm filenames. :)
else if (text:matches-regex($exist:path, '/[A-Za-z0-9]*.htm')) then
<dispatch xmlns="http://exist.sourceforge.net/NS/exist">
<forward url="index.xql">
<add-parameter name="subCol" value="../data/info/"/>
<add-parameter name="fileId" value="{substring-before($exist:resource, '.htm')}"/>
</forward>
</dispatch>
[...]
I'll be using similar patterns to serve other pages out of other subcollections, apparently from subfolders on the site, but all using the main index.xql file.
September 13, 2011
On late duty.
In Cocoon, I've always used the sitemap to simplify the linking of resources, so that (for instance) you can ask for a .css file from anywhere in the site structure, without worrying about the actual relative path, and the system knows where to find it and serves it up. I've implemented the same thing in eXist 1.5 using controller.xql. The code is below. The initial stanza, handling images, is slightly more complicated than the CSS and JS stanzas because there are subcollections inside /images/.
[...]
(: All images are to be served from the /site/images collection. :)
else if (contains($currentUri, 'images/')) then
<dispatch xmlns="http://exist.sourceforge.net/NS/exist">
<forward url="{concat('/images/', substring-after($currentUri, 'images/'))}"/>
</dispatch>
(: All CSS files will live in /site/css, so find them there. :)
else if (ends-with($currentUri, '.css')) then
<dispatch xmlns="http://exist.sourceforge.net/NS/exist">
<forward url="{concat('/css/', $exist:resource)}"/>
</dispatch>
(: All JS files will live in /site/js, so find them there. :)
else if (ends-with($currentUri, '.js')) then
<dispatch xmlns="http://exist.sourceforge.net/NS/exist">
<forward url="{concat('/js/', $exist:resource)}"/>
</dispatch>
[...]
Inside WEB-INF/controller-config.xml, I had to comment out this line, which is mysterious to me but which was sending requests for things in /images to I-know-not-where:
<!-- Comment out the following line or it will screw with images stored in your db. --> <forward pattern="/images" servlet="ScaleImageJAI"/>
I've also got the main webapp working, with nothing at all outside WEB-INF, so it's very clean and simple. I have an includes.xql module which handles the header, footer etc., and I'm now in a position to start figuring out how best to structure the app. I want to have proper .htm URLs if possible for every page that isn't a search, so I might use something like /pages/[filename].htm for every [filename].xml in the db. All of these can redirect to index.xql with an appropriate set of parameters, which would make things very simple.
Set up your lucene instructions in collection.xconf so that you're using the standard analyzer instead of the whitespace analyzer (which is the rather inflexible default). Here's what's inside my lucene structure:
<analyzer id="sa" class="org.apache.lucene.analysis.standard.StandardAnalyzer"/> <analyzer id="ws" class="org.apache.lucene.analysis.WhitespaceAnalyzer"/> <text qname="tei:div" analyzer="sa"/>
DW helped me sort out a range of issues with the banner in the French site. One was due to an HSD image rotator being inadvertently included in our banner setup; the other seems to have been my fault -- apparently I'd made changes to the site-styles.css file back in May, but I can't find any references to that on the blog, and can't remember what I was trying to fix when I was doing it. I took the first half of that file from the German and Slavic site, and it's working fine now. Meanwhile, History seems to have been making progress, and I've contacted TG about image processing and BD's instructions.