Map Of London

Map of Early Modern London with Literary References
September 14, 2011

Controller patterns working, XHTML5 valid, and other progress

Posted by on 14 Sep 2011 in Activity log

Set up some more controller clauses to handle what I'm calling "info" pages; these are pages which were originally in PHP/HTML, but will now (after consultation with JJ) be in TEI XML like the rest of the data. They'll be stored in /db/data/info. I've written a Transformer script for converting the HTML, which is working pretty well; the only things I need to think about are the addition of nested divs to enable the proper use and rendering of <head>.

I've also written XSLT to convert these pages over to XHTML5, and spent quite a lot of time getting the page to validate. There were a couple of issues:

  • The validator is unhappy with 
    <meta http-equiv="Content-Type" content="application/xhtml+xml;charset=UTF-8" />
    at the moment; don't know if that's permanent or not, but I've commented it out.
  • The validator required this as charset decl: 
    <meta charset="utf-8"/>
  • It rejects this as a valid meta/@name: 
    <meta name="resource-type" content="document"/>
    . I've commented it out.
  • It gacked completely on the original code for a Facebook Share button, which is clearly invalid: 
    <fb:like href="" show_faces="false" width="450" action="recommend" font="arial" colorscheme="dark"></fb:like>
    and also lacks an href, so I don't know what it would actually do. I replaced it with this alternative formulation which FB now offers: 
    <div class="fb-like" data-href="mapoflondon.uvic.ca"></div>

So we now have working, valid pages being served out of the db, and also transformed inside eXist, all as part of index.xql, although every page gets its own filename ending in .htm, by doing this in the controller:

[...]
(: Regular site pages served from the root with .htm filenames. :)
else if (text:matches-regex($exist:path, '/[A-Za-z0-9]*.htm')) then
    <dispatch xmlns="http://exist.sourceforge.net/NS/exist">
    	<forward url="index.xql">
    	  <add-parameter name="subCol" value="../data/info/"/>
    	  <add-parameter name="fileId" value="{substring-before($exist:resource, '.htm')}"/>
    	</forward>
    </dispatch>
[...]

I'll be using similar patterns to serve other pages out of other subcollections, apparently from subfolders on the site, but all using the main index.xql file.

September 13, 2011

Using controller.xql to simplify resource linking, and other progress

Posted by on 13 Sep 2011 in Activity log

In Cocoon, I've always used the sitemap to simplify the linking of resources, so that (for instance) you can ask for a .css file from anywhere in the site structure, without worrying about the actual relative path, and the system knows where to find it and serves it up. I've implemented the same thing in eXist 1.5 using controller.xql. The code is below. The initial stanza, handling images, is slightly more complicated than the CSS and JS stanzas because there are subcollections inside /images/.

[...]

(: All images are to be served from the /site/images collection. :)
   
else if (contains($currentUri, 'images/')) then 
    <dispatch xmlns="http://exist.sourceforge.net/NS/exist">
    	<forward url="{concat('/images/', substring-after($currentUri, 'images/'))}"/>
    </dispatch>
    
(: All CSS files will live in /site/css, so find them there. :)
    
else if (ends-with($currentUri, '.css')) then 
    <dispatch xmlns="http://exist.sourceforge.net/NS/exist">
    	<forward url="{concat('/css/', $exist:resource)}"/>
    </dispatch>
    
(: All JS files will live in /site/js, so find them there. :)
    
else if (ends-with($currentUri, '.js')) then 
    <dispatch xmlns="http://exist.sourceforge.net/NS/exist">
    	<forward url="{concat('/js/', $exist:resource)}"/>
    </dispatch>

[...]

Inside WEB-INF/controller-config.xml, I had to comment out this line, which is mysterious to me but which was sending requests for things in /images to I-know-not-where:

  <!-- Comment out the following line or it will screw with images stored in your db. -->
<forward pattern="/images" servlet="ScaleImageJAI"/>

I've also got the main webapp working, with nothing at all outside WEB-INF, so it's very clean and simple. I have an includes.xql module which handles the header, footer etc., and I'm now in a position to start figuring out how best to structure the app. I want to have proper .htm URLs if possible for every page that isn't a search, so I might use something like /pages/[filename].htm for every [filename].xml in the db. All of these can redirect to index.xql with an appropriate set of parameters, which would make things very simple.

September 12, 2011

Working on application structure

Posted by on 12 Sep 2011 in Activity log

I've been looking at the URL Rewriting documentation, and writing some test libraries. I've also looked at Jamie's Francotoile work, and I'm learning from that, and deciding to do some things slightly differently from the way his site works. Things learned today:

  • If you deploy in Tomcat using a WAR file, don't ever delete the WAR file; Tomcat deletes the app irrecoverably. Nothing valuable lost, but it could have been.
  • Nothing outside of WEB-INF is actually required. The JNLP continues to function as long as it's configured in controller-config.xml (in WEB-INF).
  • The approach of putting the application logic inside a collection called site is a good one; relative links in output to e.g. CSS files in subcollections "just work", but they're served as application/xml or whatever rather than text/css, so I think it's more productive to have a configuration in controller-config.xml which calls an XQuery for ANY path ending in .css; that way the correct response headers can be sent, and CSS files are accessible from anywhere in the site structure, without worrying about relative paths.
  • For permissions, "update" doesn't really appear to mean "update" in the case of XQL files; to enable a guest to browse to an XQL file and see its output, it needs to have world "update". However, as long as world doesn't have update on the folder containing the XQL file, it's safe from modification by guest. This means you really can't be mixing data and logic.
September 7, 2011

Preliminary work on port to eXist 1.5

Posted by on 07 Sep 2011 in Activity log

Started this project by doing a rough analysis of the application as it currently is, and reading MK's original documentation. I've sent some requests for info to JJ. I then started looking at eXist documentation and figuring out what the best approach will be to the architecture of the application. The two choices are basically to store everything in the db, including controllers, binary data such as PDFs and images, XQuery, XSLT etc.; or storing only XML data in the db, and having controllers, transforms and binaries on the file system. The former is cleaner, and is what the other two eXist 1.5 projects have done, so it's favourite, but it will require careful separation of concerns and user management inside the db, because both JJ and research assistants manage data in the db, and we don't want logic files overwritten by mistake when uploading data.

August 29, 2011

Access problem

Posted by on 29 Aug 2011 in Activity log

JJ called up to report an access problem, which we eventually narrowed down to what looks like a rogue ACL:

  • mapoflondon.uvic.ca is accessible from everywhere.
  • The admin interface (lettuce port 8888) is not accessible from UVic WIFI or from wired connections in the English dept, but it IS accessible from our office, and from outside UVic (e.g. 3g phones).

I've put in a ticket to sysadmin -- waiting for responses.

June 20, 2011

eXist stopped

Posted by on 20 Jun 2011 in Activity log

Inexplicably, eXist just stopped running. No idea why. JJ reported it and I found nothing running when I ps aux|grep london.
So I started the service and everything appears normal now.

March 20, 2011
March 17, 2011

Kingsford Edition Experiment

Posted by on 17 Mar 2011 in Announcements

We have added an experimental page to the library: a transcription of a section of the Kingsford edition of John Stow's A Survey of London (1603 text). We have taken a brief section of the transcription made available on British History Online by the Centre for Metropolitan History, made a few minor corrections, and marked it up with site identifications. The section we have marked up is The Citie of London Divided into Parts, which describes the logic and history of the ward boundaries, a crucial structuring device for Stow. Our long-term goal is to provide a marked-up transcription of A Survey in a database that each street/site page will query.

September 14, 2010

missing stars due to empty shape and coords elements

Posted by on 14 Sep 2010 in Activity log

JJ reported that in some segments of the map, only a few stars were appearing. Checked the code quickly and it seemed fine, so then replaced the index1.xml file with a backup copy from May and the start reappeared. Compared May index1.xml to September index1.xml and noted differences.
Concluded that the code processing that XML file into stars for the map requires that if you specify a segment on the map in the section/title and section/sortKey elements (e.g. A6) then you must specify a shape (rect or poly) and provide coordinates (a minimum of four numbers, I'm not quite sure what each number represents, though they are obviously the number of pixels from the top-left corner of the image).
If you write a location element like this:
<location>
<title>The Curtain</title>
<sortKey>CURTAIN</sortKey>
<id>CURT2</id>
<type>site</type>
<section>
<title>A6</title>
<sortKey>A6</sortKey>
<shape/>
<coords/>
</section>
</location>
then a star for that entry doesn't appear, and some other stars in that segment of the map (e.g. A6) don't appear either (the code processing the stars for the map segment dies when it encounters the empty shape element and any subsequent entries aren't processed into stars)

If you include an entry like this:
<location>
<title>The Curtain</title>
<sortKey>CURTAIN</sortKey>
<id>CURT2</id>
<type>site</type>
<section>
<title>A6</title>
<sortKey>A6</sortKey>
<shape>rect</shape>
<coords/>
</section>
</location>
then a star for this entry appears on the left margin immediately below the map, and the stars for all the other entries appear properly. If you have more than one such entry in a section, then all of the stars for such entries are piled underneath each other and only the first star is visible. The rect requires at least 4 comma-delimited integers which are offsets from the top-left of the image (but I'm not sure of exactly what each value represents)

If the location has a sortKey value of OFF MAP, then apparently there are no problem with the display.