Category: Activity log


Permalink 09:57:47 am, by mholmes, 35 words, 57 views   English (CA)
Categories: Activity log; Mins. worked: 90

Completed first incarnation of document type taxonomy

Working from the plan KSW and I created last week, I've created a TEI taxonomy, filling out details and adding in a couple of things we hadn't discussed (such as document types for born-digital stuff).


Permalink 03:46:09 pm, by mholmes, 48 words, 74 views   English (CA)
Categories: Activity log; Mins. worked: 60

Beginning work on a document type taxonomy

Following the success of the MoEML document type taxonomy as a way of organizing, listing and searching resources, I've decided to do the same thing with the Coldesp documents, so I've started outlining a taxonomy. The project schema has also been updated to allow the appropriate header elements.


Permalink 02:57:42 pm, by mholmes, 151 words, 64 views   English (CA)
Categories: Activity log; Mins. worked: 240

OAI regeneration now fully automated

I now have a bash script, utilities/, which does the following:

  • Runs the XSLT transformation to generate the OAI records for all existing documents. This XSLT now also creates a fresh version of identify.xml which contains the right <earliestDatestamp> for the records.
  • Commits the new identify.xml to svn, and uploads it to eXist.
  • Compares the newly-generated files with the old ones, and:
    • Where a new file has been added, adds it to svn and uploads it to eXist.
    • Where an old file exists but no new one has been created, deletes it from svn and from eXist.
    • Where an old file exists, uploads it to eXist, and commits its changes to svn.

With this, we have a fully-automated process to regenerate the complete OAI record set whenever we need to. It takes about fifteen minutes to complete, and has been pretty thoroughly tested.


Permalink 04:48:59 pm, by mholmes, 140 words, 86 views   English (CA)
Categories: Activity log; Mins. worked: 180

OAI generation rewritten

The old XQuery-based way of generating OAI files was very time-consuming and required an offline Cocoon/eXist to run. I've now completely rewritten it as a single XSLT transformation, which takes about a minute to generate all the files. It has a couple of parameters we can tweak if we decide to change ids to remove the .scx, and it builds the files to a separate folder. I'm going to combine it with a shell script that will:

  • Note the time.
  • Generate all the files.
  • Copy them over into the original folder.
  • svn delete any files which have timestamps older than the start time, because that means their despatch files have gone.
  • svn add all files (in case there are new ones).
  • svn propset all files (ditto).
  • svn commit
  • Clean out the directory on the eXist server.
  • Repopulate the directory.


Permalink 04:09:00 pm, by mholmes, 235 words, 80 views   English (CA)
Categories: Activity log; Mins. worked: 180

Beginning work on a new eXist webapp for the Despatches

Today I've started work on a completely new web application for the Despatches project. The first stage has been to get a good working build script; I'm using the one Greg created in the svn/hcmc/utils repo, but I've tweaked it a bit (and recommitted the changes) so that it will include the Scheduler extension, which I think we may use for building OAI files.

I've decided to build this app (initially at least) in the form of an eXist .xar file, so I've been learning how xar files work and how they're built. I've configured the ant build.xml script so that it can update the svn copy of the data and copy that into the xar file from the original svn location, excluding certain files. This is a good start, and it means we should be able to create an easily-updated, easily-deployed version of the webapp for testing with new builds of eXist as we move forward with the rewrite. There's a lot of stuff from the standard xar setup that I'm not going to follow, and I definitely won't be using any of the templating stuff, but I see no reason not to use the basic xar deployment functionality.

I'm also going to reconfigure the SVN file structure a little too, to make it more friendly for XQuery targetting the despatches. I'll move the "year" directories into a single "despatches" directory.


Permalink 04:28:42 pm, by mholmes, 7 words, 75 views   English (CA)
Categories: Activity log; Mins. worked: 45

Review of grant app outline

Reviewed KSW's rewrite of the grant application.


Permalink 04:21:05 pm, by mholmes, 222 words, 92 views   English (CA)
Categories: Activity log; Mins. worked: 120

Meeting, reports and grant applications

KSW and I met with CP, LW and CW at the library to review the situation regarding grant applications. We completed the interim report for this year, which is now with CW and will be submitted next week. We also plan to write the final report for this year within the next two months, so that it's in before the grant application for next year goes in, and the latter has to be there by December. CW will send us a copy of last year's final report, so that we can start drafting this year's. Meanwhile, TB will be asked to read through the 1863 transcripts and relevant external sources to list the significant events from that year, to help with writing the justification for the new application.

1863 is about the same size as 1862 in terms of the transcripts we have; we would want to generate images from CO 60 / 15, 16 and 17, which we have (B-088, B-089, B-090); ditto CO 305 / 20 and 21 (B-244 and B-245); CO 398 / 2 is done, as is CO 410 / 1. CO 6 / 37 and 38 we'll have to get from LAC, and while we're at it I think we should get 39 through 43 (the last CO 6), so they're there when we need them. RG7 G8C vol 10 is already done and covers 1863; it runs to the middle of reel C15634, and vol 11 starts in the middle of that reel.

Permalink 04:05:20 pm, by mholmes, 57 words, 101 views   English (CA)
Categories: Activity log; Mins. worked: 20

Another fix for vessel name formatting

A previous attempt to format vessel names correctly was incomplete. What we need to do is to always format vessel names in italics when they're not in correspondence documents; but in correspondence, we stick with the formatting of the source document (of course). I've now tweaked the CSS and that seems to be the result we're seeing.


Permalink 11:29:43 am, by mholmes, 13 words, 66 views   English (CA)
Categories: Activity log; Mins. worked: 10

Grabbed the stats

Hadn't grabbed a copy of the stats for a few months. Now done.


Permalink 03:14:42 pm, by mholmes, 50 words, 109 views   English (CA)
Categories: Activity log; Mins. worked: 30

Update to schema and fixes for XSLT

It looks as though I'd never finished implementing and testing the use of @type on placeName and @subtype on label/name for vessels for incomplete or unavailable entries, so I updated the schema to allow these and include documentation, and fixed the XSLT which wasn't quite doing the right thing.

<< Previous Page :: Next Page >>

Colonial Despatches

The Colonial Despatches is an XML database project which is creating a digital archive containing the original correspondence between the British Colonial Office and the colonies of Vancouver Island and British Columbia. The project lives at, and the web application runs on the Pear dev Tomcat. The XML data is managed in SVN at



August 2014
Sun Mon Tue Wed Thu Fri Sat
 << <   > >>
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30

XML Feeds