HCMC Blogs

Let's take a moment of silence to mark the passing of culture.ca

According to a message on culture.ca's homepage, this online portal to Canadian culture has been discontinued since April 1.

Read more »

On late duty.

Having worked through all the BC 1858 notes to fix up the XML and add all the appropriate links among documents, I spent today working out the best system for integrating them into the actual despatch files, and then carrying it out. These are the detais:

• All operations on the XML files were carried out in a /test/ subcollection, parallel with the main correspondence collection, to protect the original data.
• First I uploaded a single document to the /test/ collection, so I could work out the code on a small sample.
• This is the XQuery code used to replace the footnote:

  declare default element namespace "http://www.tei-c.org/ns/1.0";
  
  for $t in collection('/db/coldesp/test/')//TEI
    for $r in $t//ref[@type='footnote']
      let $f := concat(substring-before($r/@cRef, ' '), '.scx'),
          $n := substring-after($r/@cRef, ' ')
    return 
      update replace $r with collection('/db/coldesp/notes/')//TEI[@xml:id=$f]/descendant::note[@xml:id=$n]:)
  

• This worked well on the single test file, so I uploaded all of BC 58 and ran it. It failed after a certain number of files, because of n anchor in a file which did not have a corresponding note to retrieve.
• I identified and fixed all such cases, using this code:

  declare default element namespace "http://www.tei-c.org/ns/1.0";
  
  for $t in collection('/db/coldesp/test/')//TEI
    for $r in $t//ref[@type='footnote']
      let $f := concat(substring-before($r/@cRef, ' '), '.scx'),
          $n := substring-after($r/@cRef, ' ')
      return 
  
        if (collection('/db/coldesp/notes/')//TEI[@xml:id=$f]/descendant::note[@xml:id=$n]) then
          ''
        else
          xs:string($r/@cRef)
  

• Suspecting, rightly as it turned out, that there would also be instances of the reverse situation, I used this code to discover and fix notes in the footnote file that did not have corresponding anchoring <ref> tags in the documents:

  declare default element namespace "http://www.tei-c.org/ns/1.0";
  
  for $note in collection('/db/coldesp/notes/')/TEI[@xml:id='B58NOTES.scx']//note
    let $id := $note/@xml:id,
        $n := $note/@n
  
    return
      if (collection('/db/coldesp/test/')/TEI/descendant::ref[ends-with(./@cRef, $id)]) then
        ()
      else
        concat($id, ' ', $n)
  

• Notes with no anchors were easily fixed because they have citation text which identifies where they belong in the document. Anchors with no notes were all the result of broken identifiers. All errors were fixed uncontroversially.
• When I ran the conversion code again, it completed successfully.
• I then backed up the /test/ subcollection to the local machine, validated the files and examined them, and then test-uploaded one to the main collection.
• Next, I had to edit the XSLT so that it correctly renders the new in-document notes, without breaking the rendering of the old notes system, because the latter is still in use in the VI files. This was fairly straightforward.
• Finally, I was able to upload all the changed files. We now have 491 working footnotes, in the right places, for BC 58.

Added an entry for John Lutz's new book to the book gallery page on the history site.

Used a graphics editor program to create two images of the book cover from the large (about 1500 by 1900 pixel) original. lutz_makuk_sm.jpg appears on the index page and is about 140 by 180 px; lutz_makuk_med.jpg appears on the details page and is about 280 by 360 px.
Uploaded the two images to the book_images folder on the site. Ensured the file permissions were correct.

Copied the file lutz_myth_memory.html from the site.
Renamed it lutz_makuk.html.
Edited that file with the new content and a pointer to the new image file lutz_makuk_med.jpg.
Uploaded the file to the site. Ensured the file permissions were correct.

Downloaded the book_index.html file.
Copied a td element to the start of the table.
Moved the /tr and tr tags so that each tr element contained three td elements.
Edited that element with the correct details for John's book and a pointer to the image file lutz_makuk_sm.jpg and a link on that image to the file lutz_makuk.html.
Uploaded the file to the server.

Checked everything in a browser.

Long project meeting with JEH, JL, CP and EGW. EGW will write up detailed notes. I got 99 new documents from JEH at the meeting, along with revised versions of the schedules, and a bibliography detail document which will be handy for SM (and PS when he takes over the bibliography).

It's not really clear what these 99 documents are. I had 106 RTF files which I thought were these documents, but clearly they're different by at least 7 files; furthermore, the 99 files we've received are in at least two formats (rtf and docx), and it seems likely that some are versions of others. It's also not clear whether any of them yet exist in the database, or whether their filenames overlap with documents that do exist in the database. The schedule revisions are even more confusing; JEH had planned thatwhen we insert new documents, we could happily rename/re-id the existing documents, but that's not feasible because the names/ids are used as anchors and targets, so I think we'll need to use a suffix on each inserted filename which serves to create the correct order without renaming any existing files. Since it's more than likely that other documents will appear in future and need to be inserted, we should also make the suffixes suffixable. This is my proposal:

• B585G03 [old file]
• B585G03_A [new file]
• B585G03_B [new file]
• B585G04 [old file]

with the proviso that we can later do this if necessary:

• B585G03 [old file]
• B585G03_A [new file]
• B585G03_A_A [newer file]
• B585G03_A_B [newer file]
• B585G03_B [new file]
• B585G04 [old file]

Not pretty, but it should sort correctly, and is infinitely extensible if required.

Also began planning some strategies for future presentations of the collection, and how we might demonstrate the scale and value of it. Coming back to the office, PS and I took some amusing pictures of the stack of original volumes, in an attempt to convey the inconvenience of the print format, and I began planning a system to represent the scale of the contents using stats retrieved from the db and demonstrated using SVG and JavaScript.

Made a couple of fixes to the BC 58 notes, especially relating to the Colonization Circular, and actually found and saved PDFs of the circular itself from an Australian site. It's probably too big to mark up (64 pages), so perhaps we can get permission to use the PDFs. Also migrated the site stylesheet onto the Lettuce server, along with some graphics that weren't there yet, and set up the document header to point to it. Added a new stylesheet link to lang02 which Pat can use to develop a print stylesheet.

Late duty...

First pass through the BC 1858 footnotes is now finished. That means they're all marked up and linked correctly (where target documents exist in the database). Many are incomplete from an academic point of view, and contain instructions to find documents or do more research. However, that's not the main concern from a programming point of view. Now I can attempt to merge the notes into the documents they're linked to, as typical TEI notes. Then it will be much easier for content editors to work on them. One thing we must to is to retain their ids, because they're used for pointing between notes.