Maintenance

ETCL user had a problem connecting through a physical port (which appears as "Down" in the nets web admin tool). Submitted Homer for it.

I installed Karmic on my iMac with relatively few problems, but audio was a bit of a hassle.
Turns out, I needed to edit the /etc/modprobe.d/alsa-base.conf file to include the following line:
options snd-hda-intel model=mbp3

I originally tried to add the option model=mbp3 to the existing options line, but it didn't take. When I added another options line with the model, it worked. (note: mbp3 means MacBookPro version 3)

Other choices that I tried were model=imac24 and model=auto. They didn't work, but I didn't try to add them on their own line.

Another option that I came across was 3stack-6ch, which apparently gives superior audio. No idea if it works.

I also have two mice: one Kensington Expert Mouse and one Apple Mighty Mouse. I want the Expert Mouse to be my main mouse. It must also be left-handed. After lots of searching I found that I should be able to create an FDI file that sorts it all out, but I swear it doesn't work. Here's what *did* work:

running "xinput list --short" got this back:
"Virtual core pointer" id=0 [XPointer]
"Virtual core keyboard" id=1 [XKeyboard]
"Kensington Kensington Expert Mouse" id=2 [XExtensionPointer]
"Built-in iSight" id=3 [XExtensionKeyboard]
"Microsoft Natural® Ergonomic Keyboard 4000" id=4 [XExtensionKeyboard]
"Sleep Button" id=5 [XExtensionKeyboard]
"Power Button" id=6 [XExtensionKeyboard]
"Video Bus" id=7 [XExtensionKeyboard]
"Microsoft Natural® Ergonomic Keyboard 4000" id=8 [XExtensionKeyboard]
"Power Button" id=9 [XExtensionKeyboard]
"Macintosh mouse button emulation" id=10 [XExtensionPointer]
"Mitsumi Electric Apple Optical USB Mouse" id=11 [XExtensionPointer]

I then ran this:
xinput set-button-map "Kensington Kensington Expert Mouse" 3 2 1 5 4 6 7 8
which *only* adjusts the buttons on the Expert Mouse, leaving the Apple Mouse alone.
Broken down, the 3 2 1 part swaps buttons 3 and 1 (bottom-left and bottom-right), while the 5 4 business reverses the scroll movement. I left the 6 7 8 bit alone, but as far as I can tell, there is no button that identifies itself as 6 or 7. Button 2 is the (physical)top-left button, and Button 8 is the (physical)top-right button. The current functionality of button 2 is to operate as the middle-click, which is fine with me.

Once happy with the arrangement I went to "Preferences->Startup Applications and added the xinput set-button-map bit (above) as a command (name it what you like) to run on Gnome startup. I'm not sure if this behaviour is available at the GDM...

NOTE: the string identifying my Expert Mouse contains 6 spaces, but this textarea doesn't want to display it.

Set up Griffin Powermate:
1) sudo aptitude install gizmod

2) Create input group and set up udev rule

sudo groupadd -f input
sudo gpasswd -a <username> input
sudo gedit /etc/udev/rules.d/99-input.rules
paste:
KERNEL=="event*", NAME="input/%k", MODE="660", GROUP="input"
KERNEL=="js*", NAME="input/%k", MODE="664", GROUP="input"

3)Autostart Gizomd for session

sudo gedit /etc/xdg/autostart/gizmo.desktop

paste:
[Desktop Entry]
Encoding=UTF-8
Name=Gizmo Daemon
Comment=Input mapping daemon
Icon=keyboard
Exec=gizmod
Terminal=false
Type=Application
Categories=
OnlyShowIn=GNOME;XFCE;

4)restart machine

Got the final XSLT issue that we know about cleared up, and tested the site. Everything seems to be good, although we can't yet test the PDF generation. Greg and I looked at the previous XEP setup and he downloaded the latest update to our licensed XEP; it now has a servlet component, which we may be able to call, or we may go with the same system we're currently using. Either way, I should be able to test the functionality locally over the next few days, while we wait for the new server to be deployed. I don't really want to try to deploy the ACH into apache-dev on Lettuce right now, because it's already pushed a bit hard by memory limitations, so we'll probably deploy it directly to the new server.

MJ provided me with a preview of the new version of the ACH site, which combines the old web.uvic.ca front-end with the eXist-based abstract/program pages, in a single Cocoon project. This is working well, with only one or two little glitches to solve. Took a look through the code, and sent back a couple of suggestions for solving an outstanding issue with abstract 203.

I have enough hardware to replace all of the noisy old TAPoR machines, including a groovy and cheap Acer Revo nettop (which looks suitable for use in our context - and really cheap). Three of the machines are built, and I have some RAM and GFx cards coming to spruce them up for dual-monitor use. This *should* allow us to make the most of what we have for a couple of years in B045. I haven't tracked my time on this as it's kind of pedestrian stuff - not really blog worthy - but very time consuming.

I've also been working with CS on a login/auth scheme that will allow us to configure each machine to use LDAP to allow Netlink holders access to the box. The PAM stack config has been done, and we're now looking at the skel/profile config so that new users will have a homedir etc. when they log in for the first time.

MJ provided a download of the Graves webapp, which I grabbed and tested locally. It worked fine. Pushed it up to the server, and had a couple of failures; I then discovered the tree was full of shadow files beginning with dot+underscore, which I think came from MJ's Mac. Deleted all of those, and then uploaded to the main Tomcat. To make it work, we then had to get the Data folder chowned to apachsrv by sysadmin -- reminder to self and Greg, to follow up on this, because when we met with RE, he was going to look into the possibility of giving us access to a script we could run to do this.

One remaining problem is the character encoding in the submission form, which is suffering from the issue with server.xml listed here. We're working on that. Once that's fixed, there's a possibility it will break existing forms in e.g. Scancan, but that's probably going to be easy to fix; and in any case, Scancan is another candidate for a port to eXist 1.4.

Had a meeting with MJ to look at the last few remaining problems. It looks as though the character encoding issue in the search form may just be a question of starting up Tomcat with UTF-8. Other display issues look like they may be related to extra whitespace in the XML, caused either by the expansion of entities, or by errors or laxness in the original markup. In any case, they may be fixable by judicious use of <xsl:strip-space> combined with <xsl:preserve-space>. Hoping to get this port finished by the end of the week.

Greg and I had a long and fruitful meeting with RE this morning about upcoming rebuilds and reconfiguration of the TAPoR boxes. This is the basic plan:

• In the short term, RE will collapse the two DB servers into one, freeing up one of the more powerful machines. Greg is currently going through dbs on the servers to determine what obsolete ones can be dumped, to simplify the process. If there's time, they'll all roll onto one server, then the other will be rebuilt with a 64-bit OS, and they'll be moved to that one, freeing the first; if time is a problem, then they'll just complete the first migration.
• The freed-up db server will be rebuilt as a 64-bit Tomcat box, and will house (initially) two Tomcats, "legacy" and "active". Both will end up with more memory than the current running Tomcats.
• All current Tomcat projects will be migrated from Lettuce to the new server, with the current OLD Cocoon (from Tomcat Prod) going into Legacy, and all the other projects going into Active.
• When MJ has finished his porting work, the Graves and Abstracts projects will be installed on the new Tomcat server in Legacy, and links and virtual domains re-pointed. Then there are a couple of other services running on Mustard which need to be moved (eg Mongrel/Ruby). Those can be copied to Lettuce. Once that's done, Mustard will be ready for a rebuild.
• Mustard will be rebuilt with the latest Apache, PHP, etc., and brought up as a "cluster of one"; existing PHP/web projects will be copied to it from Lettuce, and tested. A side benfit of this is that all apps running on our cluster will have the advantage of the wildcard SSL cert., currently only available under hcmc.uvic.ca
• The cluster of one will go live, and Lettuce will then be brought down for a rebuild, as a clone of Mustard. Then it will be added to the cluster, so we have true failover for our web projects.

Other issues discussed

A problem has recently arisen with the NFS machine (arugula) where backups are taking nearly 12 hours, likely due to insufficient RAM. RE feels that there might enough spare RAM available to him to add up to 2GB to arugula - this will hopefully alleviate the problem. While down for the RAM we might also install RHEL5 and the new TSM client. If so, the downtime is likely to be a full day.

Currently, we have homedirs in the same filesystem as application dirs (like www). RE suggested that we look at splitting homedirs from app dirs so that our *mission critical* filesystems are as small as possible. We would also deploy the archive fs previously discussed as soon as enough space is freed up to make this possible. This would create at least 3 permanent filesystems: home, app and archive.

We also made a request for a script that will allow us to sudo run it and change permissions on certain dirs to make them writable by other users - very handy to have this functionality when deploying apps and data for other users.

/home1t/ is currently just about full. We also have requests coming in to expand the filesystem. We discussed with RE the best way of managing this unfortunate combination of events, and we arrived at a potential solution. Advice has been offered to concerned parties and we will wait for a response before proceeding.

MDH posting some edited highlights of MJ's work porting the Graves project from eXist 0.9 to 1.4, as a guide to future porting of old projects, and then the subsequent port of the ACH abstracts project. This will be enhanced and updated over time.

• Moved the Graves application code into a Cocoon subfolder called site. This is our current preferred method of doing things; it allows us to retain ownership of the files containing the application logic, while the WEB-INF directory can be owned by the user under which Tomcat is running, so it can work with the database.
• Moved XQuery files from the project root into an /xq/ subfolder. This is common sense, and is our current practice in all our projects.
• Added the session name space to XQuery files: declare namespace session="http://exist-db.org/xquery/session";.
• Changed request:request-parameter to request:get-parameter everywhere.
• Changed util:eval to util:eval-with-context. [Note: I'm not clear on why this was needed, or what the differences are; we should revisit this. I use util:eval in many projects.
• In addition to the changes from util:eval() to util:eval-with-context(), in some cases I was able to simply remove the erroneous second parameter in the function call:
  let $index_query := f:build-index-query("/db/graves/index", $year, $month),
  OLD $index := util:eval($index_query, "")
  NEW $index := util:eval($index_query)
• Forced some variables to be cast as xs:ints, as eXist is more strict about type checking. This was required throughout the original XQuery.
  OLD $prev_day := if(($month eq 3) and ($year eq 1935)) then '22' else '01',
  NEW $prev_day := if(($month cast as xs:int eq 3) and ($year cast as xs:int eq 1935)) then '22' else '01',
• Updated calls to request:encode-url, which now requires anyURI type parameters.
  OLD: $doc_uri_request := request:encode-url("xrequest.xq")
  NEW $doc_uri_request := session:encode-url(request:request-uri())
• Properly escaped ampersands in URL like places.
  OLD <prev href="{$doc_uri_request}?action=search&collection=...
  NEW <prev href="{$doc_uri_request}?action=search&collection=...
• Updated calls to request:set-session-attribute() to session:set-attribute().
  OLD request:set-session-attribute("query_stored", "true")
  NEW session:set-attribute("query_stored", "true")
• Changes either to the definition or the functionality of substring mean that "bad" code that used to function OK now needs to be fixed; an initial start-position parameter of 0 should have been 1, and although the code used to work (presumably where an invalid value was encountered, eXist substituted 1), now a zero is used, so one fewer characters is returned. Check any start-position params in substring to make sure they're 1-based.
• Image width and height attributes have been fixed (960 vs 906px).
• Added @summary to all tables, with an empty value if it's a layout table. The attribute is required for validation.
• Fixed retrieval of multiple instances of reference items when only one was required (and since they have unique ids, the result was invalidity due to duplicate ids)
  OLD <xsl:apply-templates select="//rs" mode="annotation" />
  NEW <xsl:for-each-group select="//rs" group-by="@key">
<xsl:apply-templates select="." mode="annotation" />
</xsl:for-each-group>
• Where older processors expecting an atomic value, when passed a sequence, would use the first element of the sequence, Saxon seems to be more picky. You have to make sure you're passing an atomic value rather than a sequence if that's what the processor expects:
  OLD <xsl:sort select="author/name/@reg" />
  NEW <xsl:value-of select="author[1]/name[1]/@reg" />
• All <xsl:element> and <xsl:attribute> constructors were replaced by simple inline XHTML code, except where they're necessary. This makes for simplicity and readability.
• The Cocoon XInclude transformer appears to be rather unintelligent when it comes to the output of xmlns attributes; it leaves all sorts of unnecessary ones in the output code. MJ came up with a much better solution: a template in the XSLT (2.0) that matches <xi:include>, and does this:

  <xsl:copy-of select="fn:doc('../includes/header.inc')" />

  The details are still being worked out; one possible issue is the requirement to know the relative location of the included document during the XSLT process, and another might be when the included document is in the eXist db, not on the file system.
• For the ACH, in which the project includes merging a static website (hosted originally on web.uvic.ca) with the Cocoon webapp, there was also a need to change the directory structure slightly, so that all documents are served from URLs in the project root (cocoon/site). Previously some were served out of subfolders. This standardization is a good idea generally, and makes it much easier to map the inclusion of images and so on. All my recent projects already work this way.

These are errors relating to the XML of the Graves project itself, not likely to be encountered in other projects:

• <div type="Abstract">

  should be

  <div type="abstract">

  .
• @TEIform attributes are unnecessary and have been deleted; these were, we think, a byproduct of the DTD expansion process that also expanded the entities (which we wanted).

And errors specific to the ACH:

• Missing "http://" at the beginning of a URL in abstract 154.
• Failure of FO for abstract 203 to load (still under investigation).
• Layout issue with the panelist list in abstract 147 (still under investigation).