The one remaining problem was ironed out by an XSLT fix, and the discovery that one of the thorny bugs was caused by a flaw in the original data, where "back" was typed as "bac-k" in the page of a reference target. All now seems to be working. The bug in page number duplicates is also fixed (where the original document duplicated a page number, the output rendering created duplicate ids).
May 12, 2010
Finished javascript code in 8processOneToManys.html and make_inserts_from_csv.seq.xml
8processOneToManys.html now generates two outputs:
- the first contains sql comments on cross-reference and keywords that point to non-existent values, then sql statements to clear all records out of the four intermediary tables, then sql statements to populate those tables again
- the second returns the tab-delimited input, but in the cross-reference and keywords fields, only those values which are illegal remain, so it is easy to see where the problems still lie. That output (saved in the file 9_tabs_id_clean.txt) becomes the input for the last step in the Text Transformer.
make_inserts_from_csv.seq.xml is an instruction file used by the Text Transformer. It is designed to take as input 9_tabs_id_clean.txt and produce the sql statements to clear all records from the main table and then repopulate the main table again.
Am now waiting for BI to provide data with identified errors corrected.
If BI adds any new records to the lookup sheets (e.g. new keywords), then those will have to be added manually to the appropriate table in the database and to the javascript code in both the files mentioned here.
Full procedure to go from BI's spreadsheet to the sql files ready to populate the databases is in the file howto_xls_strings_to_sql_inserts.txt. It's kind of rickety and hands-on, but shouldn't have to be executed very often.
The editorial team has decided that the 1609 Sonnet document is actually a composite document containing two texts. This means it needs to be marked up in a more complex structure than the rest of our texts:
TEI teiHeader (header for the whole document) text (container text) front (frontispiece for main document) group (group element which contains constituent texts) text (first constituent text: Satire Menipée) front (first text frontispiece) body (first text body) text (second constituent text: Thimithélie) front (second text frontispiece) body (second text body) back (back matter containing references, created by XQuery)
This requires a rewrite of the XQuery and the XSLT that processes documents, because they're not expecting this complex structure. I've rewritten the XQuery successfully, so the complete XML with references is being retrieved; I've edited LSPW's source document to add the rendition="#layoutDesc" attribute to both of the constituent texts; and I've confirmed that rendering works OK on my local copy of the db. However, I'm now struggling with a validation issue caused by the need to number each reference link in the text. These get unique ids created by counting the preceding ref elements with the same target. However, in this structure, the preceding:: axis doesn't seem to be working properly, so I'm getting lots of duplicate ids in the output. Still puzzling over this.
JL of History dropped off a 2-page summary of what he and ML of library (?law) would like done to produce a Zotero / Aigaion based project to create a legal bibliography. Martin and I are not clear on what the work flows are (i.e. what is each of the two packages used for and what are the information flows between packages). We're also not clear how much editing labour is likely to be saved by the proposed modifications to Zotero, specifically whether any code could be written that would significantly reduce the amount of editor's work checking for duplicate submissions from the field. Wrote to JL and ML asking them to provide work-flow or information-flow descriptions or diagrams.
KL sent me the quote for the VM server, administration costs for five years and TSM backup bits and pieces. Reviewed specs and terms with Greg - they're fine and come in within budget, so emailed KL saying that AS of dean's office would be paying for it and to get account details from her and buy as soon as possible. Expect about 6 weeks before ready for use.
I'm currently re-reading all the material about model/view architecture in QT to figure out how best to approach the synchronization of two views which have two underlying models (one tree, one string list), and where the synchronization code should live (in the code for the dialog box holding the view controls, or in the HcmcImtDoc class). An added complication is that the QSortFilterProxyModel which is currently filtering the list view by regexp currently lives in the dialog box code, but I think it should almost certainly move to the HcmcImtDoc class.
May 11, 2010
Too much to do, and not enough time to do it. New but in AdaptiveDB had to be fixed, Properties DB had to get its last feature added asap, and we had to have graphics ready for the Coldesp launch meeting tomorrow.
By default, one-to-many fields, when retrieved in the form of a <select> element which is used to add one to a related table, are sorted by the text field in them, ascending. However, this project will be more efficient if the latest-added item is sorted to the top -- so they should be sorted by id, descending. I added this feature as a pair of settings in the MdhOneToManyField class, after fixing another bug in the Search interface (see the AdaptiveDB blog). While I was at it, I figured it would also make more sense for ascending sorting to be available to all tables, so I've made it a feature of the MdhRecord class, and turned it on for all the tables in this project. That means all new items show at the top of lists, which makes sense to me.
Dug out my original launch poster, and added a faded-out map in the background, then reworked it for the new launch at 36 x 48. Seems to work OK...
JW reported a bug in the new search interface, whereby if you edit a document from within a table which has specific selected fields visible in it, the retrieved updated record row contains the full set of columns instead of the restricted set. Fixed this in the context of the dev site on Properties, before testing on AdaptiveDB dev, then migrating the fix to both live sites.