Colonial Despatches

A new version of the Colonial Despatches site, edition 2.2, was published on 2nd December 2020, including the following new features:

• Added more organizations to index as well as some descriptions (currently 64 organizations).
• Tagged majority of organizations in the database (12761).
• Added a few hundred more abstracts.
• All places with a known location has a map.
• The initial version of the Indigenous glossary of terms has been completed.
• Fixed a couple encoding bugs (missing divs in transcribed enclosures, unwanted text edits).
• Removed titles and roles from Despatch citations as per request from Principal Investigator.
• "Mentions of [entity] in the documents" in the indexes are now sorted by date sent and date received when available.

This is the third completely static version of the site, in compliance with the Project Endings Principles for Digital Longevity.

A new version of the Colonial Despatches site, edition 2.1, was published on 9th July 2020, including the following new features:

• Several new sections have been added to the About page, including:
  
  • Colonial Office Staff and Consultants
  • The Despatches at Court
  • Decolonizing the Despatches
  • Glossary of Terms
• Many more biographies and abstracts have been added to the collection.
• The search page now allows the use of wildcards (* = anything and ? = zero or one character). This can be helpful when searching for names or terms that may be inconsistently spelled.
• The facsimile browser has been rewritten to improve its responsiveness.
• Encoding Documentation and Guidelines have been updated.

This is the second completely static version of the site, in compliance with the Project Endings Principles for Digital Longevity.

The static release site, which is on home1t/coldesp, has now been running for a few weeks without problems, so today I've brought down the old Cocoon version that was on tomcat-devel. After stopping it in the Tomcat manager app, I took a backup of it, which I zipped and stored in my home directory on Squash at Colonial_Despatches/cocoon_final_backup_2020/coldesp_old_peach.zip.

We could now undeploy the tomcat-devel app completely, freeing some space and resources on Tomcat; we should probably do that so that a restart of Tomcat doesn't inadvertently restart the old app. I'll wait for everyone's OK before doing that.

Regarding the new static site: I need to put together detailed instructions on how to deploy a new edition. Basically, you have to rsync the content from the build on Jenkins to the deployment folder (coldesp/www), but it's important to ensure that the rsync DOES NOT INCLUDE THE DELETE FLAG, since there is a lot of other material required by the site (such as hundreds of thousands of page-images) which are not part of the Jenkins build, and they must not be deleted. (They are backed up on Squash, though.)

This is a brief outline of the work I've done over the past two weeks, comprising the bulk of the conversion to static build:

• All entities and documents have unique URLs and complete pages.
• All content is rendering correctly as far as I know.
• Schedules have been hidden on the main menu, since they're a mess.
• The map gallery has been simplified into a table. I see no reason not to show all maps at once.
• Individual map rendering is very simple, and none of the tagged maps have any interactivity. This is for a future work block based on BreezeMap, I think.
• The browse tree is working, and is richer in terms of information.
• The static search is basically working well, but there are some tweaks that really need to be done at the level of the staticSearch codebase. There isn't (yet) a special search page for people or places or any of those vague things we've planned, but you should be able to search only biographies, or only place write-ups, so no worries there.
• The snippets are now handled via JSON and the home page snippet changes randomly every 20 seconds. There's remaining work to do to convert the old approach to snippets to the new one, but GL is doing that.
• Facsimile browsing is better, using the FacsBrowser codebase I wrote for DVPP, and now allows rotation of images. It's a bit slow to load when there's a huge collection of images, but short of splitting them into batches (which would involve a lot of extra work), I think we should just live with that.
• Placeography maps are working, having been re-coded using OL/OSM.
• There is still some work to do writing specific citation forms for e.g. maps.

As soon as possible, I propose to push a recent build of this to home1t/coldesp/www, and ask sysadmin to repoint the subdomain so that we can bring down the old Cocoon app.

I'm currently engaged in a very long and gnarly process to fix lots of encoding errors and oddities which are only now being revealed in the source XML, mostly because they give rise to invalid HTML5 in the output. I'm down to a small number of errors, but there are still some issues to be fixed which will take significant time. However, the basic rendering of despatch pages is working, and before the end of this week I hope to be able to get the build process to the point where I can add HTML validation to the Jenkins job and have it pass. The time required for the build process should also have dropped considerably as of today, because I've refactored a bit and I'm making use of the already-rendered AJAX fragments rather than re-rendering the same entity descriptions multiple times. Dozens of hours on this in the last few weeks.

I've now finished converting all the original templates from the Cocoon app into modern XSLT 3, with the exception of the stuff relating to the image browser and similar special pages. There are still some page-level modules that need to be done, such as rendering the abstracts and the correspondence headers, but we're close. The output is a long way from valid, though, so I'm now going through the process of debugging and fixing things like nested links. 480 minutes, today and yesterday.

274 new page-images (in three different sizes) from the following volume have been added to the Colonial Despatches collection: RG7 G8C Volume 18 (BC 1871 Despatches from London). These will now be linked into the transcription documents.

Post content lost by db error.

Post content lost through database failure.

Post content lost by db error.