Capital Trials at the Old Bailey

Added the GUI controls and the javascript to allow users to include Jury type (Middlesex or London) and outcome (Pardon or Execution) attributes in controlling the visualization. The next thing to add is "Condition of Pardon" and I'm not sure if that means outcomes other than Death or Unknown or respites other than ToDie or Unknown - will have to check with SD. Other two categories are "Mode of Execution" and "Mercy Recommendations". I'm not yet extracting from the db to the javascript array on those fields. I've also got to create a way to allow user to specify a range of years to display. It may be that to accommodate all the capabilities desired by SD that I'll have to rewrite the grid array as an array of objects, where each object represents a trial. I can then have lots of fields on each object to control display etc.

removed all hard-coded references to colours in the javascript files (data.js, methods.js). Created 12 colour variables in data.js and used them in all the methods in methods.js. The colours need to have hex values lower than whatever hex value is used for colour_neutral to ensure that the records with value of unknown in the current sortby field will sort to the tops of each column.

MK asked to have five judges added to the list in the schema file she uses to validate the data (bailey_proofing.rng). I made the same change to the rng file I use for processing the xml into sql (bailey_for_sql.rng).

Check new structure in rng for mysql, then go back through the nine processing files and make sure that they generate the proper structure. Specifically ensure that 1) a trial_printed_sources and trial_other_documents element appear as siblings immediately following the trial_ref element and that any existing references to trial_publication and trial_correspondence refer instead to trial_printed_sources and trial_other_documents 2) a mercy_appeals element appears as a sibling after respites and before outcomes

Modified proofing and for mysql rng files, and 1790s data files by replacing the names of these two elements:
<trial_publication>some book</trial_publication>
<trial_correspondence>some other thing</trial_correspondence>

with these:
<trial_printed_sources>some book</trial_printed_sources>
<trial_other_documents>some other thing</trial_other_documents>

Also created a 1790s folder, put all the dev files into that. Created a 1800s folder and put copies of all the rng files into that. Will update those and the instructions for generating XML from word file.

Martin wrote some XSLT a year ago to export the then-current XML structure for the Old Bailey trials to mysql statements which we could use to create and populate tables. I began work on modifying that XSLT so that it would reflect changes in the structure of the XML in the data files and output based on current thinking of how we want the tables to be organized. Tried extracting list of current juries from trial records but ran into a stone wall when we tried to concatenate two strings (e.g. jury type and subtype) and then have that treated as an item. In addition, I realized that when we processed subsequent data files, there would be no way of preventing duplicate keys being generated. So, I decided that it would be simpler and safer to include in the data file all the fixed lists (e.g. judge names, jury types, outcomes, crime groups etc.) and then have the xslt process those into the appropriate sql statements to generate output and into xslt variables. That would also allow me to more easily write xslt in the trial processing code that could write in the appropriate id for e.g. the jury type by looping through the fixed list I'd generated previously and held in an xslt variable. I've now added those elements to a schema file (for_sql.rng) and example data file. I have yet to add the code in the trials processing section which will obtain the appropriate id for each of the fields (e.g. match the jury type and subtype against the the fields in each record in the lookup table and then return the id field for the matching record in the lookup table).

Set Marri up with Oxygen on Mac. Gave her the most recent data file (it lacks judge names).

Based on her first couple of hours, I edited the schema file to allow multiple (rather than one) outcomes and made a couple of other minor modifications.

Once the data set is cleaned up, I'll need to revisit whether the db engine should be relational or xml based.

Simon D and Marri came down to take a final look at the data and the oxygen editor. Marri will get in touch to set up times for her to come in an do the data correcting for the 1790s. Simon will be preparing the 1780s and 1800s. We'll ensure the 1790s data contains everything we want before processing any other decades. Once that's done, then I'll have to write the code to turn the XML structure into relational tables, and then start work on query and reporting php/mysql.

Confirmed with Simon that fixed list of judge names has been included in the rng file for the proofer (checked rng files, added a couple of new values and tested them). Still need to find out from him if an empty string is an acceptable value.
The existing 1790s data has no judge names in it. Asked SD if he wants that added by the proofer or how we wants those names included.

As far as I'm concerned, we're ready to have proofer go through records. It looks like about 250 of the 750 records in the 1790s set will need some attention. They have snippets of text that my battery of regexp's could not figure out how to deal with - typically extraneous bits of names, second charges or subsequent findings by the court. The XML structure accommodates those features, but the regexp provides for only the simplest cases.

Added choice element allowing empty or item from list of normalized names of judges. The list items can't include a space character (I don't know why - I researched for an hour and all documentation indicates the space character should present no problem). I've put those elements into the file "bailey_proofing.rng" which is the file that the proofer will use.

I still have to modify the regular expressions in the NameJudgeJuryAge4 section of howto_regular081017.txt to look for the text values and replace them with the normalized ones.