Dr. Kim Blank's widely-admired site Mapping Keats's Progress: a Critical Chronology has reached another milestone, with edition 3.4. This project is a collaboration with HCMC and is a fully Endings-compliant project.
Category: "Academic"
Dr. Kim Blank's widely-admired site Mapping Keats's Progress: a Critical Chronology has reached another milestone, with edition 3.2. This project is a collaboration with HCMC and is a fully Endings-compliant project.
Neat approach using CSS flex and an attribute value so that when you select text, you don't get line numbers along with it. 30 minutes.
The HotPot 7 version of the Wheelock site has been checked and approved by MN, so I've rebuilt it with the current release, cleared out a bunch of obsolete exercises that were still around, reworked the home page for hrd/latin, and published it. However, there are several hundred remaining errors found by the VNU validator that are almost all cases where tables have been used in JCloze, but the numbers of cells in each row are not consistent. These will need to be fixed, but it's not urgent. There's also an ANKI APKG file for Latin that was prepared by a student, but that needs to be tested and checked for copyright violations etc. before we can post it; I'm not sure how to go about that, without potentially sacrificing a device. There are binaries in the file, but I don't know what they are. 140 minutes
Added the last of the responsiveness features planned for the Close Reading site (hamburger menu thing). 30 minutes.
Doing a quick conversion of the Keats infrastructure to handle the Close Reading site for KB. Initial setup is half done. Should take a couple more hours to get to testing point.
Doing a re-crawl with deeper depth setting...
Met with AW (Ling) to discuss possible text-encoding project for First Nations texts. This will probably solidify into a project proposal for initial work leading to a SSHRC application next year.
Rebuilt the schema because new places have been added and linked to; rebuilt the site and uploaded the results; worked with GL to clear a machine and disk space to house our new RA, and scheduled her on the machine for the coming semester.
First we create a brand-new table for persons.
PHASE ONE: CHANGE AUTHORS TO PERSONS.
DROP TABLE IF EXISTS `persons`; CREATE TABLE `persons` ( `prs_id` int(11) NOT NULL auto_increment, `prs_displayName` varchar(45) collate utf8_unicode_ci default NULL, `prs_surname` varchar(45) collate utf8_unicode_ci default NULL, `prs_firstnames` varchar(45) collate utf8_unicode_ci default NULL, `prs_honorific` varchar(45) collate utf8_unicode_ci default NULL, `prs_quals` varchar(45) collate utf8_unicode_ci default NULL, `prs_gender` int(11) default NULL, `prs_birth` date default NULL, `prs_death` date default NULL, `prs_nationality` varchar(45) collate utf8_unicode_ci default NULL, `prs_links` varchar(2048) collate utf8_unicode_ci default NULL, `prs_notes` varchar(2048) collate utf8_unicode_ci default NULL, PRIMARY KEY (`prs_id`), KEY `idx_authors` (`prs_surname`,`prs_firstnames`,`prs_notes`(255),`prs_id`,`prs_quals`,`prs_honorific`,`prs_displayName`,`prs_nationality`), KEY `fk_prs_to_gender` (`prs_gender`), CONSTRAINT `fk_prs_to_gender` FOREIGN KEY (`prs_gender`) REFERENCES `genders` (`gn_id`) ON DELETE CASCADE ON UPDATE CASCADE ) ENGINE=InnoDB AUTO_INCREMENT=2546 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
Now we migrate the existing data into it.
INSERT INTO `persons` (`prs_id`, `prs_displayName`, `prs_surname`, `prs_firstnames`, `prs_honorific`, `prs_quals`, `prs_gender`, `prs_birth`, `prs_death`, `prs_nationality`, `prs_links`, `prs_notes`) SELECT `au_id`, `au_displayName`, `au_surname`, `au_firstnames`, `au_honorific`, `au_quals`, `au_gender`, `au_birth`, `au_death`, `au_nationality`, `au_links`, `au_notes` FROM `authors` WHERE 1;
Now we have to update foreign key constraints.
ALTER TABLE `poems_to_authors` DROP FOREIGN KEY `fk_pta_author`; ALTER TABLE `poems_to_authors` ADD CONSTRAINT `fk_pta_author` FOREIGN KEY (`pta_au_id`) REFERENCES `persons` (`prs_id`) ON DELETE CASCADE ON UPDATE CASCADE;
NOW UPDATE local_classes.php and test.
PHASE TWO: ADD ILLUSTRATORS.
Next, we can create the poems_to_illustrators table.
DROP TABLE IF EXISTS `poems_to_illustrators`; CREATE TABLE `poems_to_illustrators` ( `pti_id` int(11) NOT NULL auto_increment, `pti_po_id` int(11) default NULL, `pti_il_id` int(11) default NULL, PRIMARY KEY (`pti_id`), KEY `fk_pti_illustrator` (`pti_il_id`), KEY `fk_pti_poem` (`pti_po_id`), CONSTRAINT `fk_pti_illustrator` FOREIGN KEY (`pti_il_id`) REFERENCES `persons` (`prs_id`) ON DELETE CASCADE ON UPDATE CASCADE, CONSTRAINT `fk_pti_poem` FOREIGN KEY (`pti_po_id`) REFERENCES `poems` (`po_id`) ON DELETE CASCADE ON UPDATE CASCADE ) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
NOW UPDATE local_classes.php and test.
Next, we insert all of the distinct values for illustrators into the persons table.
This is the XQuery to generate the SQL:
declare namespace output = "http://www.w3.org/2010/xslt-xquery-serialization"; declare option output:method "text"; let $prefix := 'INSERT INTO `persons` (`prs_displayName`, `prs_surname`, `prs_firstnames`, `prs_notes`) ', $inserts := for $i in distinct-values(//field[@name='po_illustrator'][string-length(normalize-space(.)) gt 0]/normalize-space(.)) return let $displayName := $i, $surname := if (contains($i, ',')) then substring-before($i, ',') else $i, $firstnames := if (contains($i, ',')) then normalize-space(substring-after($i, ',')) else '' return concat('("', replace($displayName, '"', '""'), '", "', replace($surname, '"', '""'), '", "', replace($firstnames, '"', '""'), '", "Illustrator")') return concat($prefix, ' VALUES ', string-join($inserts, ',
'), ';')
Now the hard bit: create links in the poems_to_illustrators * table between poems and the new persons entries.
First, dump a new version of the database including the new rows in the persons table.
Next, run this XQuery against the dump.
declare namespace output = "http://www.w3.org/2010/xslt-xquery-serialization"; declare option output:method "text"; let $prefix := 'INSERT INTO `poems_to_illustrators` (`pti_po_id`, `pti_il_id`) VALUES ', $inserts := for $i in //field[@name='po_illustrator'][string-length(.) gt 0] let $po := $i/preceding-sibling::field[@name='po_id'], $prs := //field[@name='prs_displayName'][. = $i]/preceding-sibling::field[@name='prs_id'] return concat(' ("', $po, '", "', $prs[last()], '")') return concat($prefix, string-join($inserts, ', 
'), ';')
Next, run the resulting SQL against the db to insert * the new records.
Finally, manually tweak all the stored procedures which generate the search table. I'm not sure these are even running properly any more, and they're not even being used, but best to avoid an error for now.
I've made the changes to the database as planned, and as far as I can tell, everything is working well. Results:
- The "Authors" table is now a "Persons" table, but it works just the same way.
- The "Persons" table now includes entries for all the people who were previously identified in the "Illustrator" field of the "Poems" table.
- The original "Illustrator" field is now renamed to "OLD Illustrator field"; eventually it will go away entirely.
- Below it there is now a new "Illustrators" field, which is a one-to-many link to the "People" table, working just like the "Authors" field.
- People in the "People" table who were imported from the old "Illustrator" field have the word "Illustrator" in their Notes field. That's just for information and convenience, and can be removed if necessary.
- Information on the new people in the "People" table (i.e. illustrators) is sparse, because we had very little info about them in the "Poems" table before. Someone will have to edit each of those entries. For instance, they all have unknown gender at the moment.
- It's possible that if the same person is both an author and an illustrator, they now appear in the "People" table twice; CE will be checking through all the illustrators and merging records where necessary.
Detailed explanation of the code steps follows in the next post.
I have four to prep, so I'm starting early. :-) The Keats one, which is part TEI part Endings, is perhaps the easiest.
The problem conflict file in svn is now fixed (svn resolve [file] --accept theirs-full
) and we have a good idea of where we're going after a discussion.
Fixed the remnants of a conflict in a file committed by KB. Combined the two files with the two parts of Lamia into one, because KB has been linking to a single file that doesn't exist, and it's not horribly big. Added four new images to the gallery.
Credits for proofers are now included in the output where they're available in the XML as respStmts. There is XSLT which can harvest most proofer info automatically from the HOCR, so with the exception of about 500 pages, where the proofer is only known personally to DH, that info can be added as collections are completed, and will appear on the site.
I'll document this on the Endings blog, because it's really part of that project.
Documentation in the ODD file is coming along nicely.
An ongoing list ePly information.
Full documentation here, comprehensive: http://help.eply.com/general-settings/form-status
Rather than using the hcmc@uvic.ca account, create a new user with the role of ‘client administrator’ - for full powers. Other users will not get this level, usually.
Only client administrators and client power users can refund credit cards, all other roles cannot.
Creating a form creates a new event.
Now up on the site.
Meeting to prioritize bugs; fixed a bug with footnotes pointed out by GL; tried to fix another quickly but the fix seems to be incomplete. More time for this after the Keats is handled.
I think this is working, but it needs thorough testing, and then it can be run as part of an ant task (so that we can be sure to apply it only to the TEI files).
Received request from SB to remove/update announcements section and insert past talks on talks page.
Trying to move a bit more quickly; three more to do, should be done tomorrow.
Added the hook for running a function after the HolLayer has been created, as tested with the Landscapes maps, and also abstracted most of the caption strings to properties in the hol namespace near the top of the file; that's not the ideal way to do this, but it's better than having them sit in the code until I come up with something better.
Polished off a couple of simple tickets from the repo and commented on a few more; tried a build of the data, but many errors, so:
- Made milestone[@type='unknown'] allowed but warned against, rather than invalid, so I could get over a bunch of half-finished documents.
- Fixed a bunch of encoding errors such as missing private URI prefixes.
- Wrote some Schematron to catch table encoding errors; up to now, we've relied on the VNU validator to catch unbalanced cells/rows at HTML validation time, but that's inefficient, so I've done something which should at least catch obvious errors right in the XML.
- Fixed some broken table code.
- Rebuilt and uploaded the site.
The designer had embedded the OpenLayers map directly in the home page using an iframe, which of course borked all the document links. I've now fixed that, although I'm not 100% happy with how it's working; I still need to find a way to maximize the use of space in the standalone version. Meanwhile, lots of troubleshooting for the two remote workers with Oxygen. Not being able to train people directly is problematic.
Got the list of students and poems from AC and built the teaching package. I optimize it a bit every time I do it. Will test the download in the lab tomorrow.
I've found a few good resources for map tiles.
The CRD takes aerial photos every two years and makes them available as WMS layers. Add 2015 to an OL4 map:
//CRD layerCRD = new ol.layer.Tile({ //extent: extent, source: new ol.source.TileWMS({ url: 'https://mapservices.crd.bc.ca/arcgis/services/Ortho2015/MapServer/WMSserver', //crossOrigin: 'anonymous', params: { 'LAYERS': '5', 'FORMAT': 'image/png', 'SRS': 'EPSG:3857' }, serverType: 'mapserver' }) });
The BC Government also provides aerial/satelite tiles
layerBCWMS = new ol.layer.Tile({ //extent: extent, source: new ol.source.TileWMS({ url: 'https://openmaps.gov.bc.ca/lzt/ows/', crossOrigin: 'anonymous', params: { 'LAYERS': '140', 'FORMAT': 'image/png', 'SRS': 'EPSG:3857' }, serverType: 'mapserver' }) });
and map tiles, too
layerBC = new ol.layer.Tile({ source: new ol.source.XYZ({ url: 'http://maps.gov.bc.ca/arcgis/rest/services/province/web_mercator_cache/MapServer/tile/{z}/{y}/{x}', numZoomLevels: 18, maxZoom: 17 }) });
Licensing is (as of October 2017) quite liberal.
With KB, interviewed VB for the former's workstudy position; green sheet not yet available.
Made some tweaks to the schema yesterday and Monday, and rebuilt the site. Hangouts meeting with GL and DH, reviewing progress, planning and making a few decisions on outstanding markup questions.
The build process is now creating complete place fragments to serve as AJAX popups for the map. There were some considerable difficulties in making this work, and I'm still for some reason unable the get my AJAX promise stuff to work on the map, while it works elsewhere. Still, good progress.
Met with AC, went over the last SSHRC app responses, and discussed a new strategy.
Received request from BAK to edit link on site to reflect current calendar.
Changes made; email sent to BAK advising site updated.
Posting time spent yesterday on:
- More conversion and cleanup of the placeography
- Long meeting with DH
- Work on aligning the placeography with the personography
Received request from BAK to update site with new student blog entry.
Task completed; confirmation email sent to BAK.
Received site update request from BAK.
Changes:
- removed old announcements and replaced with 2 new ones
- uploaded 2 new pdfs (workshop poster and Lansdowne poster)
- added 2 new announcements
- received new blog info from MF; added it to students page
- synchronized site
- sent confirmation email to BAK, MF advising task completed
Finished merging the 20th parliament data into the personography. This was very tedious and time-consuming, but it's done now. Hopefully nothing else was missed in the original spreadsheet.
Online meeting (notes on GitHub) clarified a few points and made some decisions; afterwards I followed up to handle one ticket, the replacement of typographic ligatures from XML files and default handling for them in the creation of XML in future.
Did one pass through the MakingEME site per AB, changing future tense to past now the conference is done. Also used a Google Sheets plugin to download a Twitter archive from the conference hashtag -- it's not yet clear what we'll do with that yet -- and I have a number of questions outstanding about changes to the more complex documents such as the Policies page.
BAK approved latest changes to site; now uploaded to live site.
Received requests from BAK for site updates re courses information for 2016-17.
Completed the updates and waiting for confirmation from BAK if anymore changes are needed before uploading to live site.
Converted semi-manually from Google Doc to HTML; changes will be made to the Google Doc with change-tracking so that I can easily port them over.
Lots of work restructuring the SGML-to-TEI, after testing with a couple more files; found some errors in existing SGML files too. Some fruitful discussion of verse and line encoding with the group.
Reconciled schemas from workshop and current IML-to-TEI output; we're now gradually converging on decisions about details, which I'm building into the ODD file. I've also reworked the original template and restructured the Oxygen package a bit, although the build file needs to be re-done.
Followed SA's excellent documentation to create a test version of the new event, but was a bit thrown when it didn't work as expected (I was allowed to vote repeatedly). After some reading of the code, I figured out from a comment that the version on the test account is most likely configured this way on purpose (I can't check it because it's in SA's account), so I took the existing production code and put a copy of that in my account pointing to the dev db to confirm that it would work as expected, which it did. I've now created the prod vote, which will go live on Wednesday morning.
Blended a lot of content from the old ODD file I created for the workshop last year into the new schema, and tweaked the XSLT to generate TEI from IML so that the new born-TEI encoding work is in line with the old stuff from IML. We're beginning to answer some questions about how our encoding practices should go.
All the existing records have now been published, and the new theme has been deployed, with the help of HR. I've also fixed the home page to include the Google map which TS created as part of the theme design.
I think that means all our work other than final checking/proofing etc. has been done, so it's ready for the launch; I think the About page could use some tweaks, but that's all.
It hasn't yet been migrated to the Library. The server is no longer groesbeek; it's robalo.comp.uvic.ca, and is based on Apache rather than nginx. The db is now outside the vm; it's running on gen1p.mysql.uvic.ca, not on robalo itself. I have access to the db (atom_ccap) on mysql.uvic.ca (log into gen1p) using a special hcmc user.
ST made some changes to the XML, but used an HTML editor, so it was a bit broken. Fixed it, and added support in the XSLT2 for a couple of TEI tags, generating HTML tags in the desc for a feature. This will be important going forward.
Meanwhile, met with JJ and GN to outline a grant application for making a proper job of this mapping functionality. Will write the core of it tomorrow.
Nothing like being on vacation when you actually need to make progress with something. In the last couple of days I've added all the document display functionality, added more sophisticated handling for centring features and sets of features, updated a lot of the documentation, refactored much of the code to move var declaration to the top of functions (not complete yet), fixed a bug in the search, made the colour sets user-settable, and made numerous tweaks to display and rendering functionality. There's one remaining bug in webkit with the white outline for the navigation panel selected item, but apart from that, I don't think there's much. Now I need to start working on the mobile display, I think.
It was time for a substantial de-crufting, so I used JSHint to find and fix a lot of issues, as well as refactoring to remove dependence on the hacky _this var, by increasing dependence on bind(). Results are a lot cleaner.
The navigation search functionality is mapped out and the interface components are in place, but I'm having trouble binding the onkeydown event on the search text input to the prototype function for doing the search. When I did this for MoEML, I was working with an already-constructed instance of the object, and I could therefore assign the function call in an onkeydown attribute value; since in this approach I'm writing all the code as prototype functions and the object doesn't exist until runtime, it's much harder. The issue is passing a reference to the event (which I need in order to detect what keys are pressed) through to the prototype function; the latter never seems to get called at all. Tricky.
Worked with ST on XHTML5 documents and got her started learning CSS and applying it to the documents. Also wrote a new XSLT to generate enhanced GeoJSON as well as a standard-format (per MoEML and GN and me) categories.json file from the spreadsheet data. The GeoJSON is valid and working, and all the core data is in it, so now I can start writing some generic code to handle it.
MS came by with three final articles which have not yet been encoded, and some intro material. We added the intro stuff, and encoded one of the articles, but it proved to be missing a lot of key info and still needs more work. I'll do the other two next week. We aslo discussed grant applications relating to a conference, and mapped out some costs.
Moving into dealing with some of the QME texts has thrown up new oddities in encoding in the originals, so I'm now back working on earlier texts.
Started work on the ODD file; I have the elements and attributes/classes sorted, I think, and I'm beginning work on specifying the subset of TEI Simple @rendition values I think we should use, as well as beginning some text documentation.
Met with the team teaching Eng 362 and mapped out a plan. Created a section of the MoEML repo to store the documents and set up the package build; added people to the svn permissions for that folder; created an initial ODD file to build a basic schema. Lots to do here, with a deadline of mid-Feb.
Got New West spreadsheet from BS, and after fixing a bunch of non-UTF-8 chars, tried an import; the result was 123 of those mysterious empty records with "translations" which I deleted last week, and the cause was that it was a tab-sep instead of a comma-sep file. Deleted them all (laborious), resaved and reimported, and all is good.
Discussion with DH following the first round of beta testing by editors. We decided to simplifiy the HOCR output considerably, discarding all the offset data, and try to parse out page numbers. I've implemented the changes and run them against Ont_Que English and Alb_Sask Federal; if they're OK, then we can move ahead with this.
Editors will be asked to signal column breaks with <hr>/
tags in Kompozer, and we can easily recover these and turn them into <cb/>
s in TEI.
Received request from SB (EMRC) to update announcements and talks page.Completed task, sent email confirmation to SB.
Received request from SB to update EMRC site with report of
event news.
Task completed and sent confirmation to SB.
MK very kindly sent me their complete Access database, which I'm now wrestling into submission. On first analysis, it looks like there are 1584 records in there which have an explicit association with "CHINESE" (culture/group) which are not among the ones they uploaded to our database so far.
117 of these items appear to have associated images.
I need to write some code to generate spreadsheets for importing this data, and while I'm doing that, I could also generate a human-readable version that MK could browse through to check that we're not adding any fields we shouldn't be, and that we're not including any items that should not be public.
We don't know yet whether the images might be available, but if they are, it should be possible to get them in too.
I think it may take me a week or so to get this done.
How I got at the Access data: GN opened the file in Access on a VM on his machine, and opened it to VNC; I was able to view it and export each individual table as an XML file, along with a map of the db schema. The XML files are large, but quite easy to figure out. The export process (table by table) was tiresome and time-consuming, though.
Met with PAB. She will soon supply some templates which I'll integrate into an Oxygen project for her. Meanwhile, I need to set up an svn repo for the project.
With SA, wrote a validator in XSLT (just because), which parses a CSV file intended for import into the db and flags a whole pile of different types of errors we've become accustomed to seeing.
Tested, bugfixed and updated the materials for English 500. I'll be testing them in the actual room next week, when it's free. I discovered some testing I had integrated into the poem transformation for JJ to use when marking made the XSLT fail in the student context, where other versions of the XML files were not available.
Updated schedule with Ra's additional hours for (Med.Project)
Went to Carsa Bldg., and MacKinnon Bldg. today to discuss room
possibilities for Performance event.
Met with TM; viewed both buildings' studios etc. Reported to DS
my findings.
Next steps:
Will need to meet again with TM once info received from DS.
Have scheduled meeting with TM (Carsa)tomorrow re conference special event
room.
Advised DS, sent email confirmation to TM.
Went to view rooms in Cornett, MacKinnon and Carsa Building with an eye to the special Performance event needs.
Rooms in Cornett: questionable
Other options:
Have requested a meeting with BW (Rec.Ser.Admin) to discuss Performance
event studio space requirements. Awaiting reply, then if need be will arrange meeting with PJ (McKin. Bldg) contact.
Sent update to DS, will keep him posted.
Met LS today who is working on volume conversion project.
Updated computer schedule with additional hours for LS (volume conversion project.
Met with KF today and viewed 3 rooms (Sch.of M). Option: possibly dual-purpose one room (preferably B016). Sent DS email with details; DS and RS will meet tomorrow to details. Will advise JN later with outcomes from meeting.
Spent today searching for rooms, will pursue next week.
Meeting with KF on Monday re rooms for Music Listening space; will ask re lockable
storage possibilities
Received request from ECH (Moses Proj) for additional time for workstudy ED.
Updated schedule; sent email confirmation to ECH, ED.
Continuing to search for event spaces; waiting for additional info from DS and others
Today, spoke with DS re updates from DS.
- hold off organizing Exhibit space - more info incoming from DS
JN's next step:
- forward email to DS that I received from ST (AVT) re webcasting/video streaming
- meeting with ST (AVT) and DS TBA until DS obtains more info from ELO contacts
- send email to DS re FAMIS report suggestion (DS to follow up)
Contact School of Music:
- organizing meeting with KF to discuss rooms
JN contacted today:
- KF in School of Music; meeting set for Monday, Oct. 5, 11:30am to view room(s) for Music Listening purposes during conference. Need from DS specific dates for room use and having to share with other user at same time may need to be negotiated.
- LN in Visual Arts re possible studio booking; LN recommended DS send email with
request and all specs, timeline etc to Chair and cc Fac. Manager of VA in case
summer course is not needing the studio in which case it's a possibility for ELO
Performance event. (JN sent email to DS re this request; DS will get dates)
Audio / Video info:
Spoke with UVic Help Desk to get info re: Audio, video streaming info; contacts,
general overview info; had info emailed to me to have on hand when I meet again with DS next week (date TBA)
Spoke with Facilities Management (7616) re contacts for furniture rentals (recliners etc.)
- suggested FAMIS request be submitted for assistance from Interior Design support
(cost involved for consultation services). Will retain this info for next mtng
with DS (date TBA)
Visited School of Mus. (MacLaurin); obtained name of contact person to speak
with next week (date TBA) re rooms availability; sound systems etc; will contact SofM again
next week after DS/RS's meeting takes place.
Spoke with ST re video streaming etc.; will send me email with services details;
advised him I will contact him again later next week when I have more info from DS.
JL brought down ST, who is working on a map-based collection of stories from the Stolo. Discussed the project, booked time for her, set up an SVN repo and put some basics (svn instructions etc.) in there, and we're ready to go. I'll meet her next Monday to get her started with svn and XML.
Today's task: to meet contact person on campus to discuss furniture requirements & terms
-JN:met ES (Degrees catering) brief meeting
Next steps:
- after meeting with DS on Sept.24, 2:00pm, JN will email ES to set up 2nd meeting with her to discuss furniture availability, cost, delivery/setup/return etc.
Booked meeting with DS for Thursday, Sept. 24 2:00-3:00pm to discuss further specifics re JN's conference task list.
JN & SA (HCMC) met to discuss details of JN's tasks re ELO.
Next steps:
- meet with DS to get specifics re 3 special "events"
- setup meetings with campus individuals once I know DS's specifics
- blog progress daily; cc SA on correspondence
Booked hours for new workstudy (SM) on Myths on Maps project; sent email confirmation to SM, LB, GN.
Received request from SB to update their site with new fall announcements.
Updated site; synchronized site.
Sent confirmation email to SB advising task completed.
JP's still rather vague plans seem to involve an external web application coupled with a database that really ought to be AtoM, but how the two are going to talk to each other is not clear yet. Did some digging into AtoM and its documentation and components to figure out how this might possibly be done. The most likely approach, given that the OAI-PMH interface is frankly broken (and in any case would require an XML database to make decent use of it, an added complication), seems to me to be ElasticSearch, which is the Lucene-based search backend. This maintains indexes of documents which should be sufficiently rich for us to call it from an external source, although I'm not sure how yet. That would get us as far as document identifiers; from there, presumably we could redirect to AtoM itself, or perhaps retrieve OAI-PMH records and transform those on the fly with XSLT in PHP. Ugly but hackable.
Got a spreadsheet of revised material description info from Nanaimo last week; I've now processed this with XSLT into SQL statements and used it to update the db.
After much back and forth with the Cumberland folks, we decided to test running a more permissive version of my code which would accept an image as a match if it began with the C-number, but just had some additional stuff on the end. This resulted in 104 candidate matches, which was more promising, but after looking at them (I have the script building a web page with links), they concluded that what they had thought were irrelevant suffixes (S for small, for instance) were actually significant, and none of these additional matches was actually correct.
So the plan now is to wait for the new DVD with full-size unedited images, all with correct filenames, to arrive in the mail, and to run it against those images and see what emerges. At least my script is now fully developed.
Meanwhile, TS added a bunch of stuff to the Guidelines document, and I've added some bits myself, and created a new distribution of the instructions (nice to have that done by ant, even including the upload). Note to self: always script this kind of thing if you can; it'll end up saving you hours.
Have updated the site with all of BAK's changes as requested.
Sent confirmation email to BAK advising task completed.
Synchronized the site.
Completed the updates with one exception, waiting for reply from BAK, confirming course information detail.
Received from BAK request for website updating with 2015-16 course information and addition to announcements.
Currently updating site with requested changes.
Branched the git repo and cleaned out all the multi-schema stuff from master. Also made a lot of progress on creating a user-friendly approach to generating a new play file and a new schema, so that users can follow along in the Instructions file and just type an id and a title and get their play file. I have one outstanding question on the Oxygen board; if I can't do what I want to the way I want to, I have a workaround in mind.
Met with PAB. Added new material to Beck site; discussed Oxygen project setup for MyNDIR editors (templates, authoring CSS, rendering transformations, code templates).
Updated LARG page with new workshop information and photos.
Updated announcements on images page also; uploaded files to server.
Sent email to BAK advising task completed.
Received another list of changes from BAK.
Have made the changes to site; uploaded files to server.
Sent email BAK advising site updated.
Updating web-dev site with current changes.
Received new links information.
Working in web-dev site have completed the following:
Resized all new photos for workshop.
Created new page for 2015 event.
Announcements:
- removed lansdowne lecture; archived poster
- added new announcements with some links to index page
In progress:
- new link re course forthcoming
Waiting for remainder of information before uploading changes from web-dev to live site.
Resizing workshop photos to include in recent updates request to site.
Editing announcements section.
Received request from BAK to update announcements (part I); relocate announcements items; archive announcements; create new page for 2015
images; file and resize images; further changes TBA (part II)
Currently working on site updates.
Re-ran the first import operation, and then did some manual fixes to the db in phpMyAdmin. Things I've learned:
- Ignore warnings about going straight from Repository to Item; intervening levels are not required.
- Use the actual full name of the repo ("Nanaimo Museum") in the CSV file for the repository field.
- Any mass data operations require CLI access; in practice, it's going to be better to learn how the db works and make edits in SQL.
Much good work on AtoM data ingestion formats and test materials, as well as broader project planning.
Did some prep for tomorrow's meetings with SR from Artefactual.