teiJournal

Began working on the XSLT for XHTML output of the search. Got the search form and fields all rendering themselves, with the correct selected data, and got started with the table of documents. The form itself seems to be losing all its data when it's submitted -- have to look at that. :-)

Rewrote and extended the contents.xq code to add:

• Sorting by document type (previously missing from the sort parameters, for some reason)
• XML <list> elements containing items for each distinct value of year, author, title (truncated), vol/issue, and content type.
• Addition of @rend attribute for an item in the list which matches an existing search parameter.

This will enable easy creation of drop-down lists with correctly-selected values in the XSLT which processes the contents page.

The next stage is to plan how that XSLT will be written -- in particular, how it will work with the rest of the site. I'm assuming at this stage that we'll want a standard set of processing libraries for the non-article site content, and that this content will be drawn from the db as well (although perhaps from another area of the db). I'll need to look at the strings components we have in the db, to see if it's going to be sufficient to build all the surrounding content with those alone, or whether we'll need some user-editable XML files for menus, headers, footers etc.

Turned off Java binding in eXist's conf.xml file (in cocoon/WEB-INF), and the Java module continues to work fine, so we've nailed that.

Looking at the rendering of documents in my pilot install on Greg's machine, I noticed whitespace missing from mixed content. This is a problem we've encountered before, but it took a while to remember what needs doing. This part of conf.xml needs to be changed, from:

<indexer caseSensitive="yes" index-depth="5" preserve-whitespace-mixed-content="no" stemming="no" suppress-whitespace="both" tokenizer="org.exist.storage.analysis.SimpleTokenizer" track-term-freq="yes" validation="none">

to:

<indexer caseSensitive="yes" index-depth="5" preserve-whitespace-mixed-content="yes" stemming="no" suppress-whitespace="none" tokenizer="org.exist.storage.analysis.SimpleTokenizer" track-term-freq="yes" validation="none">

In other words, two changes to attribute values. Then you have to re-upload all the documents in the the db.

Built my Java module and got it working on the test Tomcat stack on Greg's machine. As usual, the eXist documentation was slightly wrong, adding overhead to the existing pain. For the record, this is what works when you're importing the Java module in XQuery:

import module namespace su="http://hcmc.uvic.ca/namespaces/xqsearchparser" at "java:ca.uvic.hcmc.xqsearchutils.SearchParserModule";

The next thing to try is turning off Java binding in the config file. If it still works with binding turned off, then we have no security issue, and I can then try putting my module into the existing Cocoon setup on Lettuce; that would let me test and get the TOC page working for teiJournal while we're still waiting for it to get its own Tomcat stack.

Marked up Doug's intro, and in the process added handling for the <foreign> tag, and documented use of <hi> and <foreign> on the Website.

The president and president-elect's intro to the journal was marked up and added to the db.

Finally got the driver and function wrapper classes to build against the eXist jars, so I can now flesh out the wrapper that turns my search parser classes into a module I can call from XQuery without turning on Java binding. This was harder than I wanted it to be, mainly due to my unfamiliarity with the eXist source.

I've spent a lot of the day struggling to find a setup which will allow me to write extension modules for eXist. This is probably a fairly simple issue, but I'm new to Java so I'm having problems. I tried both in Eclipse and in NetBeans, and got a bit closer with the latter. The issue is figuring out how to link in the eXist source, and the libraries on which it depends such as apache and anntlr files. So far I haven't found a setup which enables my IDE to successfully build my classes while finding all of the related classes in the various locations.

Finally, I went through and added all the JAR files in the eXist tree to the project. That seems to have helped a bit, but it's still failing to find some stuff (QName and SequenceType). Back to this tomorrow.

Started looking through the instructions here about writing XQuery modules for Java. Downloaded the latest stable eXist release to get the source, so I can link into it with Eclipse. Discovered that the instructions refer to org.exist.xpath, which isn't in the source; these functions seem to have moved to org.exist.xquery.

I started with the decision that my own packages ought to be properly hierarchical and named, so I created a new package called ca.uvic.hcmc.xqsearchutils, and moved the current search classes into it. Then I recompiled the jar file, and checked that I can still invoke the classes in the old way, based on Java binding. This means I have to use this line:

declare namespace su="java:ca.uvic.hcmc.xqsearchutils.SearchParser";

instead of the simple class name to invoke the class. That works fine on our test system.

Next, I tried linking in the eXist source to my Eclipse project. There are lots of errors and warnings when I do this, so linking to an external folder might not be the way to go. I'm going to try putting the source inside my existing Java library.

This documentation covers the changes you have to make to the default download of Cocoon + eXist built as a WAR file to get everything working for our regular projects (of which teiJournal is a good example). This process was tested using a clean download of Cocoon+eXist today.

1. First, rename the WAR file to "cocoon". If you don't do that, you'll be dealing with ungainly long directory structures for ever.
2. Next, deploy the WAR file (i.e. Stop Tomcat if it's running, put the WAR file in its webapps directory, and restart it.
3. Check that Cocoon has deployed OK by going into the Tomcat manager application to see it, and also going to :8080/cocoon/.
4. Stop Cocoon from within the Tomcat manager so you can add some custom Java libraries.
5. For teiJournal, add the following libraries to cocoon/WEB-INF/lib:
   • TitleSortComparator.jar (sorts titles ignoring leading articles, etc.)
   • xqSearchUtils.jar (contains the search-string parsing functionality used for teiJournal's search).
   Both of these libraries were written with Eclipse, and will be found in your Eclipse workspace folders.
6. For support of XSLT 2.0, you need to add and configure the Saxon 8 libraries. Get the Saxon-B download from here. Then extract all the .jar libraries into cocoon/WEB-INF/lib.
7. Now we need to configure Cocoon so that Saxon can be called. First, open cocoon/WEB-INF/cocoon.xconf, and find the bit that refers to Saxon XSLT, which is commented out by default. Uncomment the code and change it according to the instructions in the file, so that it enables Saxon 8:

   	<component logger="core.xslt"
                role="org.apache.excalibur.xml.xslt.XSLTProcessor/saxon"
                class="org.apache.cocoon.components.xslt.TraxProcessor">
         <parameter name="use-store" value="true"/>
         <parameter name="transformer-factory" value="net.sf.saxon.TransformerFactoryImpl"/>
       </component>

8. Now we need to edit cocoon/sitemap.xmap to enable the Saxon transformer. In the <map:transformers> section, add this below the other XSLT transformers:

   <map:transformer name="saxon" pool-grow="2" pool-max="32" pool-min="8"
   		     src="org.apache.cocoon.transformation.TraxTransformer">
   	    <use-request-parameters>false</use-request-parameters>
   	    <use-browser-capabilities-db>false</use-browser-capabilities-db>
   	    <xslt-processor-role>saxon</xslt-processor-role>
       </map:transformer>

9. Add the following in the <map:serializers> section, to enable a couple more useful output formats:

   	<!-- Customization: compatibility setting for IE6 -->
       <map:serializer logger="sitemap.serializer.xhtml" mime-type="text/html" name="xhtml11_compat" pool-grow="2" pool-max="64" pool-min="2" src="org.apache.cocoon.serialization.XMLSerializer">
         <doctype-public>-//W3C//DTD XHTML 1.1//EN</doctype-public>
         <doctype-system>http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd</doctype-system>
         <encoding>UTF-8</encoding>
       </map:serializer>    
       <!-- Customization: set text output to UTF-8 -->
       <map:serializer logger="sitemap.serializer.text" mime-type="text/plain" name="text" src="org.apache.cocoon.serialization.TextSerializer">
         <encoding>UTF-8</encoding>
       </map:serializer>

10. Now, if you're going to use Java Binding in eXist XQuery modules, you'll need to enable it. Open cocoon/WEB-INF/conf.xml, and change <xquery enable-java-binding="no"> to <xquery enable-java-binding="yes">. IF YOU DO THIS, BE AWARE OF THE SECURITY ISSUES -- see below.
11. Restart Tomcat (restarting just Cocoon doesn't seem to be enough).
12. Check that Cocoon is running (:8080/cocoon/).
13. Check that eXist is running (:8080/cocoon/samples/blocks/exist/).
14. Start the eXist client for the first time (using the Webstart Client Launch link on the menu of the page above).
15. There will be no admin password at first; just log in as admin with no password, then change the admin password. You'll immediately get an error; don't worry, just close the client down and restart it, then log in with the new password.
16. Now you can add data and a project folder as necessary, and test the results.
17. Now, IF YOU'RE USING JAVA BINDING, you'll need to configure some security using XACML. We don't know how to do that yet :-)