HCMC Blogs

On late duty, and was gifted the solution to a bug very late in the day, so stayed to get the Moses search working fully.

JW on the eXist list solved my problem, by pointing out a slight syntax change that makes util:expand() work as it should. This is the working code to retrieve tagged hits inside a modified container:

for $e in collection('/db/moses')//tei:entry[ft:query(., $find)]
    order by ft:score($e) descending
      return
      <tei:entry xml:id="{$e/@xml:id}"  n="{$e/ancestor::tei:TEI/tei:teiHeader/tei:revisionDesc/@status}">
        {util:expand($e)/*}
        </tei:entry>

It works really well, and all the normal search string tricks (plus and minus, quotes etc.) work perfectly. Stayed late to get it deployed to the server.

I have the search working well, but there's a dilemma: due to what seems to be a bug in eXist, I appear to have a choice between retrieving results with the hits marked up with <exist:match> tags, which enables me to highlight them, and retrieving the results with their status value attached to them as an attribute. In other words, if I massage the retrieved entries in any way, to attach the attribute to them based on the status of the file they come from, the <exist:match> tags are lost, and I can't highlight the hits on the page; but if I don't add the status value, I can't display the appropriate red, amber or green box next to each hit.

Because this looks like an eXist bug, I've spent half the afternoon trying to rebuild eXist and Cocoon using our script from last year, but that doesn't seem to work, unfortunately; I'm still trying to figure out why it fails.

It's not quite clear yet what will go on the cover, and the precise year is not clear (2010, or 2009-2010?), but I've made a start on the cover for volume 19. I'm basically working from these instructions, but I did find a much easier way to extract the TOC pages from the main PDF for insertion into the cover, using GhostScript on Linux:

gs -sDEVICE=pdfwrite -dNOPAUSE -dBATCH -dSAVER -dFirstPage=5 -dLastPage=5 -sOutputFile=master_vol_19_toc_1.pdf master_vol_19.pdf

gs -sDEVICE=pdfwrite -dNOPAUSE -dBATCH -dSAVER -dFirstPage=6 -dLastPage=6 -sOutputFile=master_vol_19_toc_2.pdf master_vol_19.pdf

I was then able to combine the two TOC pages into one, following my own instructions from before, and import it into the cover in Illustrator on Windows (which I'm still using, sadly, because my template is all set up there).

I calculated the spine width at 0.82 inches this time, much bigger than previously, because there are so many pages this time. I've re-jigged everything appropriately. HT's name has been removed from the cover because of the absence of reviews in this edition. The only thing still outstanding is a decision on the date; once that's fixed, I may have to re-do all the TOC stuff, because the date's included in there too, or possibly just the cover content if it's 2010. The former will be easier because the cover content has knockouts, which are tricky.

It's been a year since we built what seems to have been a remarkably stable Cocoon+eXist, and we've been using it for many projects. However, today, working on the Moses project, I came across what looks to me like a bug in the functioning of util:expand(), which is preventing me from making the search engine work exactly as I had hoped. So I've finally bitten the bullet, and I'm trying to use our Cocoon+eXist build script again. These are the hoops I've had to jump through so far:

• In the rebuild of my machine, a directory called build_files had unaccountably disappeared from my build tree. This contained all our key files. I restored it from Rutabaga.
• JAVA_HOME was not set on my machine. I set it to point to the Sun JRE, only to discover that I need the JDK, so I installed that.
• Next I got a Cocoon error due to a duplicate entry in the jars.xml file; I don't know why this error is suddenly appearing, but perhaps a new file got included into eXist which was previously already in Cocoon.
• However, the showstopper is a stack of errors that show up when building eXist itself, during the build of the Lucene extension. I can't see what's causing these, but tomorrow I'm going to do a full new checkout and try it all again.

Frustrating and annoying, but I guess necessary.

Bug: results containing anomalous citations
e.g.
Place: Pylus
Char: Abas (1)
Cite: Apollodorus 1.9.9

Problem: Abas doesn't actually show up in 1.9.9

Also, in tracking this down I notice that when I look at the view (in phpMyAdmin) I get data that's cut off. Are the fields too short? How? Why?

Here's the view:
CREATE VIEW `view_characters_places` AS select
`view_events_characters`.`character_name` AS `character_name`,
`view_events_characters`.`character_id` AS `character_id`,
group_concat(distinct `view_events_places`.`place_name` order by `view_events_places`.`place_name` ASC separator ', ') AS `place_name`,
group_concat(distinct cast(`view_events_places`.`place_id` as char(20) charset utf8) order by `view_events_places`.`place_name` ASC separator ', ') AS `place_id`,
group_concat(distinct cast(`Places`.`latitude` as char(20) charset utf8) order by `view_events_places`.`place_name` ASC separator ', ') AS `latitude`,
group_concat(distinct cast(`Places`.`longitude` as char(20) charset utf8) order by `view_events_places`.`place_name` ASC separator ', ') AS `longitude`,
group_concat(distinct `view_events_places`.`citation` order by `view_events_places`.`place_name` ASC separator ', ') AS `citation`

from (
(`view_events_characters` join `view_events_places`) join `Places`
)
where (
(`view_events_places`.`event_id` = `view_events_characters`.`event_id`)
and (`Places`.`place_id` = `view_events_places`.`place_id`)
)
group by `view_events_characters`.`character_name` order by `view_events_characters`.`character_name`;

Three to go.

The IALLT Journal is planning to move to a Drupal installation, but they want to maintain the XML versions of articles, so at their request I've used oddbyexample.xsl to generate a constrained schema and documentation for them. My own documentation, which is more intended for novice markup folks, will need to be updated to account for the changes in markup practice over the last year or two.

Added a link to the site on RR-R's instructions.

I now have the search returning results rendered into XHTML. The last stage is hooking up the JS to make the AJAX work.