Landscapes of Injustice

I now have an ant build file written, which grabs the XML dump from the db, processes it into the simpler XML my existing code is used to working with, and builds the Streets HTML file. It also validates all the existing XML before starting, something which threw up some horrible invalidities. There's a bunch of encoding that took place on the Haney prose data which abandoned our schema entirely and went for tei_all, which obviously caused much markup drift; I've now incorporated the needs of that encoding into our schema. The existing directory files also had lots of problems, so I've tightened up the schema and fixed those; everything now validates. I need to build a lot more Schematron into the ODD file.

But now we're good to go on building stuff like diagnostic checks and generational suggestions into the build process.

This wasn't dead trivial to set up, so I'm documenting it. What's needed, first, is a read-only user on the database. That user needs to have read permissions (I included views here), and also needs Lock Tables (presumably everything is locked during the dump process). Finally, that user needs to have explicit rights to "localhost"; the generic "%" will not do.

Once the user is set up, create a script on the db server which first deletes the old version of the file you want to dump to (otherwise the dump will fail); then runs the dump command. That command looks like this:

/usr/bin/mysqldump --xml --user=[readonlyuser] --password=[password] landscapes_live > /home1t/[user]/www/[outputfile].xml

In this case, we put it in www so that it can be retrieved easily by a Jenkins build process which makes use of it.

Finally, crontab -e to edit your crontab, and add the following line (this dumps every midnight):

0 0 * * * /home1t/user/[script].sh

At RS's request, changed the field size of ttl_notes from 1024 to 2048.

To generate the streets.html file from the xml data files:
- run compare.xsl on itself
I.e. a transformation configured as:
(XML URL = ${pdu}/xsl/compare.xsl , XSL URL = ${pdu}/xsl/compare.xsl , transformer = Saxon-PE 9.5.1.7
Save as = ${cfd}/../streets.html, check Open in Browser with Saved file, check Open in Editor )

To generated a csv file for each xml data file:
- run everything_to_csv.xsl on streets.html
I.e. a transformation configured as:
(XML URL = ${currentFileURL} , XSL URL = ${pdu}/xsl/everything_to_csv.xsl , transformer = Saxon-PE 9.5.1.7 ,
Save as = empty , show results as = XML , nothing else set)

HR is now able to upload materials to the shared file space, but file naming and permissions are still a bit of a problem; we're working on that. In the AtoM db, I created a user and group for her, and gave her permissions over her institution, so she's going to start creating archival records.

The directories markup has diverged into various dialects over time, so we're pulling it back into consistency. I've written some XSLT designed to standardize it and made some ODD changes, but I'm not certain we're on the right path yet; the current proposal is a bit tag-abusive of <roleName> inside <occupation>, so we may go back to the other things that was being used, <title>. This will have to be completed next week.

The copy db could not be used without core tables having content, so rather than have the students re-create ethnicities, census tracts, locations etc., I imported all that stuff including properties from the live db. We also discovered that the triggers were missing on the lawyers table, so I've now added those. Hopefully that makes the copy db functional.

Listed out my current top tasks in the new Basecamp project. Also got the SFU repository data cleaned up in AtoM, and published all the imported archival descriptions so that we can discuss the success/failure of the import. It looks quite good, but there are obvious problems such as repeated dates.

Apparently, my BEFORE INSERT trigger concatenating the owner info with their new id could never have worked (although no bug was ever seen or reported before). What it's trying to do is:

• when a new record is inserted
• concatenate the information about the owner name, address and the record id, and
• store that information in a description field (own_desc) in the record itself.

However, you can run a trigger BEFORE inserting the record (in which case the id is zero, because it hasn't been inserted yet); or you can run it AFTER inserting the record (in which case you cannot update the record itself, because it's already been inserted). The old trigger was doing the former, so the id showed as (0).

So my conclusion is that this either never worked at all, or if it did work, it was a lucky accident that was undone by a regular update to the database libraries, and it won't work in future. Curses.

More tweaks to presentation.