teiJournal

Having got the actual mechanisms of the backup system working (generating files and saving them onto the server), I started integrating the AJAX scripting so that backup calls don't actually leave the page. Getting it working with Firefox was easy -- I initially had a pipeline that returned a single XHTML anchor tag, of which the text was the new modified date of the file -- but the predictable problems surfaced with IE, so I had to rethink. In the end, I have this:

• The "Generate backup" button makes a call to the appropriate flowscript function for the type of backup being made.
• The flowscript creates the backup, then reads its last modified date.
• The flowscript hands off (through a cocoon.redirectTo call) to an XQuery script, passing the last modified date.
• The XQuery script generates a single XML node, which is actually another anchor tag.
• The output from the XQuery is processed into text by an XSLT script.
• The text is received by the AJAX object, which uses it to replace the text in the existing anchor.

While the page is waiting, the target anchor tag has its text replaced with a series of dots, which shows a simple progress system. The effect is that the date/time shown on the link is updated on the page, every time a new backup is generated.

This is tested and working on FF, IE, Safari and Opera. The next stage is to create a sequence wrapper which can call the script many times, causing the items to update one at a time, while showing per-item progress somewhere at the top of the page. That shouldn't be too hard now.

A kind soul on the Cocoon list pointed out a missing variable in my XHTML code (the "realpath" component was missing from the path to the folder I was trying to create -- a dumb typo), and I've now got XHTML backups working fine, including saving copies of the CSS and JS into the backup folders, to create independent packages for each document.

The next stage is to make this all function through AJAX. This isn't too complicated, but it will require some thought; I need to return some useful content (the date/time of the file created), in a usable format, to replace the current info on the page, so I'll need to read that using Java once the file is created.

Rewrote my backup page so that it now shows a useful and more readable table of all the files that exist, along with buttons to generate or re-generate hard copies of any documents. Documents that exist are shows as date/times of their last modification. The generation pipelines no redirect, on success, to the backups page they were called from, with a hash component to make sure the page doesn't jump around too badly when there's a lot of content on it. What's there so far works great.

However, I've spent all day struggling with the problem of creating directories in which to store the individual XHTML files. They should each have their own directory so that copies of the JS and CSS can also be stored there; that's required in order to make them actually functional and portable. However, I can't get the flowscript to create a new directory. After many trials, I've finally posted a message to the Cocoon mailing list. We'll see if anyone can suggest what might be causing the problem.

Updated some of the site pages in the protected version of the journal to match the external copies maintained on the journal website by the editors. This duplication is annoying, but it should go away if and when the journal is opened up to the world. We're also having preliminary discussions about digitizing old print versions, including the possibility of marking up some of the content.

Wrote the pipelines and flowscript for saving PDF, TEI, NLM 3.0 and NLM 2.3 files to the file system, and created a basic list page which links to the files that are there, so you can create backups and view them.

Next, I want to create the XHTML output, which involves creating a separate folder for each document, and dumping not only the output but also copies of all the CSS, JS and so on into the folder. Then it might be possible to zip the contents of the folder into a zip file.

The backups control page should also be arranged as a table, showing file generation dates, to make it clearer exactly what's happening.

After discussion with JF, WP and LR, I've made a formal proposal for a TEI SIG to work on proposals for a schema and for some controlled vocabulary lists (for bibliographic item types, and journal contribution types).

Came across this interesting list of biblio item types today: http://vocab.org/biblio/schema. I need to compare this with my list, to provide the basis for discussions on the bibliographic item type list.

HC pointed out that sorting by Vol/Issue (by clicking on the TOC table column headers) ignored the editorial sequence of items within issues. That's now fixed. Whenever the TOC is accessed "unsorted", or sorted ascending or descending by Vol/Issue, the individual items are sorted by editorial sequence within the issue.

The sort order in the contents page was a bit of a hack, based on an ad-hoc @n attribute in the root element; it worked fine for a single issue, but multiple issues require more sophisticated sorting, so I've replaced that attribute with a more formal designation of each document's position within its issue:

<publicationStmt>
...
<idno type="itemNo" n="03"></idno>
...
</publicationStmt> 

The leading zero makes for easier sorting in cases where perhaps the XPath or XSLT engine doesn't handle strict typing well. The <idno> element is mapped to the <elocation-id> element in the NLM output, giving us better NLM metadata.

The default sort sequence is now more sophisticated: it sorts by volume and issue descending (40, then 39, etc.), then by item number ascending (01, 02, 03...), so that the most recent issues sort to the top, but content within each issue sorts in sequence.

The documents for 40.1 have now been promoted to the published collection, and the editors will announce the publication after they've done a final check.

Following the recent loss of Joe Sheehan and Jim Pusack, the editors sent an in memoriam piece to be included in volume 40.1. This is now marked up, and necessitated also the addition of a new contribution type, which is tagged as "obituary".

I'm in the process of building a system whereby editors can go to a specific page on the site, and from it, view all the possible output formats for every document in the db; they can also generate "hard" copies of each of these formats and store them on the server as a permanent record (in case the db or Cocoon become unavailable).

I made the following progress today:

• A sitemap pipeline is in place to use the Directory Generator to create a list of the documents already in the backup directory (which is backups/{styleGuide}).
• Another pipeline, called by backups.xml?styleGuide={styleGuide}, generates a TEI file which lists all the documents in the database by xml:id; it also has an XInclude instruction in it which is then processed, and that pulls in the directory listing from the above pipeline, creating an XML file which has a list of the contribution documents, along with a list of the details of all the "hard backup" copies that already exist.
• Another pipeline, backups.xhtml?styleGuide={styleGuide}, processes that through an XSL transformation, which is currently only in skeleton form but will eventually create a rich page with links to view and generate any or all of the hard backups available.

So I guess I'm about a third of the way through this; the next stage will be generating individual items on demand, and the final stage will be generation of multiple items (all the PDFs, say, or all the XHTML files), along with a decent GUI reflecting progress and completion.