Maintenance

Created a download archive of the project, along with the XML files, and posted those for MJ; also trawled through any docs I could find, and sent those, along with three initial points to look at:

• One namespace is different:
  
  OLD:
  declare namespace f="http://exist-db.org/xquery/local-functions";
  
  NEW:
  declare namespace f="http://exist-db.org/xquery/f-functions";
• One function name has changed:
  
  OLD:
  request:request-parameter
  
  NEW:
  request:get-parameter
• The need to expand external entities from the DTD in the XML files before uploading; they're trivial entities that map to perfectly good Unicode chars. The files should be free of external entities and encoded in UTF-8.

MJ, working on the Coldesp dollar to relieve me, is going to be looking at the old Graves and Abstracts sites and migrating them to a modern Cocoon/eXist, each in a separate one, for deployment on Lettuce. Today I tried for the first time to build eXist/Cocoon again since the stable 1.4 forked, and found a pile more problems that I had to solve -- mainly jars.xml issues, which I was able to fix by creating my own patch XSLT file that's run after the one packaged in eXist. After a couple of hours I got it working, and posted a working Cocoon/eXist on lancenrd for MJ to download. It's down to 59MB.

The eXist on Mustard isn't responding to anything except XQuery; no client seems to be available, or JNLP. I guess we'll have to recreate the db in the new setup.

As a test, I set up a local tomcat with two webapps directories.
In server.xml, at the very end, is a stanza like this:

<Host name="localhost"  appBase="webapps"
   unpackWARs="true" autoDeploy="true"
   xmlValidation="false" xmlNamespaceAware="false">
</Host>

If you add another one, with a different name (e.g. name="something.else.com") you can deploy webapps from all over the place. This could be useful when you want to set up a complete Cocoon in a homedir, and not on the server's local disk, as we're doing with mariage v4. I successfully added a webapps directory that was in a folder in my SAN homedir that had been mounted over samba.

Hopefully this means that we can do the same on the web servers. The only issue that might come up is those pesky apachsrv permissions. Not sure how best tot deal with those...

Removed the exist-samples directory, and confirmed that it's not needed; both Mariage and the eXist client work, and so do the admin interface features of the web interface. I've added that deletion into our script. I've also revised the script so that it saves a backup of the core eXist folder from SVN, and allows you to choose to use that if you'd prefer it, rather than a completely fresh checkout. That'll save time when the build process is being run many times, and might get aborted because of problems further down in the script.

Then went on to remove all the Cocoon samples which aren't needed, but replacing the file structure, the sitemaps, and the eXist block structure. That gives us the usual eXist admin pages, which are sometimes handy to have. The result is a total size of 100MB before a project is deployed in it. That's pretty darned good, I think. There may be other places I can trim, but the size of the project code is going to be a more significant factor than this in many cases, so I think this is a good basis to start with. Tested the new build script, and it works fine. Now I'm going to look at providing a test case for the XQuery Generator bug.

While there is still an annoying bug in the eXist trunk (no <exist:match> tags when running a query through the XQuery Generator in Cocoon, while we do get them when running the same query through the admin client), the ongoing disaster that is now the main eXist 1.1.1 install on the production Tomcat is now so flaky that I absolutely must move forward with a deployment of Mariage to a new stack. I noticed that there's a limited amount of space on the tomcat-dev file space, so I now have to trim down Cocoon as far as I can. I've started by trying to build a really lean Cocoon, by working through my local.blocks.properties and disabling as many blocks as I think I can get away with.

This is rather a trial-and-error thing, because although the file does provide some documentation on what blocks are dependent on others, the comments say explicitly that it's partial and you can't rely on it. My first attempt failed to build. Judging by the error messages, there may be a requirement for the forms block, so I've now gone back and enabled the ajax, forms and template blocks (which are interdependent). That didn't work, so I went back again and decided I needed to enable javaflow and ojb. Tried again.

That got me a complete, working Cocoon, with a working eXist, to which I restored the Mariage project -- but the site failed to come up because of a missing text generator! I figure this might be in the chaperon block, so I tried enabling that and...

SUCCESS! A working stack, into which I was able to restore the Mariage site and confirm all is working well. This stack comes in at only 495MB, compared with the 1.2GB we had before, so it's a huge step forward. It might still be possible to remove a lot of cruft and get the size of the thing down a bit, but 356MB of that is the Mariage site itself, which is heavy on hi-res images. So the Cocoon/eXist block comes in at about 140MB, which is not a great deal. Now it becomes practical to deploy leaner projects. We're go for tomorrow on the dev Tomcat, I think...

Moved the last few machines off the 128 subnet, and did the inevitable tidying up, including troubleshooting chou's inexplicable behaviour (not bringing up apache httpd on reboot - thanks, Michael J).
DNS was a bit sluggish as well, so we didn't really know whether things had turned out for about 2 hours.
All is well now. Chou and rutabaga have returned, and I've removed the node for king-henry completely. If ETCL brings it back online I'll re-instate the node.

Half way through a rebuild of a Windows box to add to the pool.
Cruft cleanup, user removal, updates, AV install etc. has been done.
Physical set up, network adjustments and final tweaks still outstanding.

Ran the process again, and nothing new was checked out from SVN, so it was building exactly the same tree as before (which worked). This version has the broken link. Next, I tried running ./build.sh clean on the eXist tree before running the process again. This one also failed, so build clean doesn't help. Finally, I tried another full delete of the entire eXist folder and ran it all again. That worked. So the only system that actually worked was to delete the eXist tree before starting. That's what I'll do from now on.

This morning I did a checkout (without deleting the existing tree) and a build; the JNLP link failed, and required that the cocoon app directory be inserted into it to get a working link. However, immediately afterwards I deleted the source tree and did a fresh checkout, and it worked correctly. It's clear that the file controller-config.xml is nothing to do with the problem, because that file is identical in the broken and working versions. It seems that deleting the source tree solves the problem. Now we need to see whether a simple build clean will solve it. More experiments continuing.

Did a fresh checkout and build of eXist this morning, after deleting my source tree, and it worked fine: the resulting web app was functional in every detail (so I saved it, just in case). Did another checkout at the end of the day, and the JNLP link on the eXist page didn't work -- but inserting the name of the web application's directory into it gives you a working link. It seems that a change to the source in the repository triggered this problem, and the suspect file is /exist_from_svn/eXist/webapp/WEB-INF/controller-config.xml, which had been changed during the day. So now we have a suspect. I've stashed a copy of it; if and when a checkout DOES work correctly, we can compare the two, and report the exact bug (if they're different in a way that suggests they are the cause).