started simple php page to test code that will take the three fields in the documents table that have one-to-many data in them (doc_keywords, doc_chapters, doc_cross_references) and eventually populate the docs_to_keywords, docs_to_chapters and docs_to_xref_images tables. Encountered problem with permissions, the www folder in the siberian account on lettuce is not group write. Martin volunteered to write sys-admins to correct that. I'm pretty well stuck until that's done, as I'm pretty sure security policy won't let me post php pages on some other server or account and address that database.
March 9, 2010
changed fieldtype of doc_cross_ref from int to varchar - involved undoing a foreign-key relationship in the relations view
finished uploading all 2167 records into the documents table - about 5 in the last 1000 required manual attention due to anomalous data values. I went back to the data files and made necessary changes. I did not make changes in Ben's original xls file.
March 8, 2010
SA had to pick up C, so there was no-one else in the office between 3.30 and 4.30.
The affix.xml file which has just been completed needed to be processed using the XSLT I'd previously written to convert the erroneous transcriptions of glottalizations resulting from our original conversion to the current system. This XSLT has been successfully run on all the other files in the system. However, when it was run on the affix.xml file, in oXygen it simply did nothing; the old transcriptions remained untransformed.
I could not figure out what the problem was. Running Saxon at the command line didn't help either. There may be something rather odd about that file, though. There are two symptoms of oddity: 1) when opening the file in oXygen, I got a warning about bidirectional features being turned off due to the file size (which may indicate that there's something oddly bidi about it, although searches for the bidi-change-trigger characters produced nothing); and 2) when copying and pasting the contents of the file from oXygen to Transformer running under Wine, only three characters were pasted: an x, a 5, and a control character, "device control 2", or Unicode u+0012. However, if you search for this character, it can't be found in the document.
In the end, I gave up and used Transformer to do a conversion using search-and-replace. There may be some problem simply with the size of the file (1.1 MB) -- it's larger than any of the others. But I'll keep an eye on that file.
I've asked Leanna to remove prefatory statements of the sort “On a trouvé l'article suivant dans le Dictionnaire de l'Académie française en ligne (1694):” from the reference file. Such comments have on occasion been added where direct quotations from sources are present in the reference file. When quotation marks already indicate direct citation, this seems superpluous and looks amateurish. The same source information is given after the quote in any case.
In response to request from dept, modified the htaccess file to change which netlink ids are allowed into a restricted folder on the site
Reviewed notes from NE on broken links etc. Fixed those that needed it. A number of broken links might not be an issue when the site moves from the dev server to the production server. On the other hand, it looks like PS's update of the production site for GUMICH involves a different naming convention for the files, so we'll have to test. Most likely I'll include the old filenames, but make them redirects rather than update URLs in zillions of pages on the 12 mystery sites.
Created new users on the blog system for NE and MF and made them members of this blog.
Also did the usual generating of a small handful of passwords etc. over the week
AWB asked me to present on the diversity of Digital Humanities and how technologies in academia are affecting even those with no specific research interest in Digital Humanities. Presentation is at http://hcmc.uvic.ca/presentations/engl500. Prepared over course of week from outline I did around Christmas and then presented 1 hour on Friday afternoon. Their questions were largely on Web 2.0 technologies, intellectual property rights and academic recognition for projects that are collaboratively produced and dynamic.
Unplugged all the computers etc. in preparation for shutdown over weekend. Shut down the servers as specified by Greg. The computer running the video system for the ETCL turned out to be plugged in to the wall by itself, I shut it down by holding the power button in until the computer shut down. On Monday morning all the computers came back up except the ETCL video one. Checked the cables, power bar etc. and it looks like it might be the power supply in the computer, as there is absolutely no sign of life when I try to start it up. Informed MC of ETCL of problem with that computer.
With Martin, created the various tables needed for the Siberian project back-end. Martin populated all the look-up tables. I wrote a bunch of GREPS and Transformer rules (including modifying a js written by Martin to turn tab delimited text into an SWL insert statement). The details of the 9-step process are in a file called xls_strings_to_sql_inserts.txt . To be safe I put individual explicit delimiters in place of the tabs separating the fields using GREP. Then ran that through a number of Transformer transforms, then back to text editor to check for any obvious errors manually and to run some more GREPS to ensure all data normalized. Finally the Transformer javascript to render the insert statements. The javascript inserts an integer value for the id field. The js includes a variable, so you can go in and assign the start value before running the transform to accommodate batch uploads of additional records.
BI had pointers to 5 locations and 7 collections that didn't exist in the relevant lookup tables, so I created those and asked him for the necessary text translations.
Discovered that the last field contains 0 to n comma separated strings, each being a reference to an image file name in another record in that table. The table structure expects an int. Only 3 of first 1000 records had the problem, but many in the next 1000 do. Will change the field type in the db to varchar, upload the strings and the I have to write php that will extract those values and populate an image-cross-reference table. I'll need to have similar code to handle two other one-to-many fields in the database already.