Maintenance

Yesterday my encrypted home dir ran out of space and ecryptfs got into a problem where it could not write changes (including deletions I was making to reclaim space), and it became unmountable. All my data was safe on another mounted volume, so today I rebuilt the machine with a fresh install of Ubuntu and reconfigured everything. All seems to be working well again; there will be a few things I haven't remembered to install, but much more that I had installed and would never have needed again. I've reconfigured docker to store stuff on the spinning disk (it was a docker container that cause the problem), and made some other similar tweaks to keep the home dir size under control. Managed to do other work as all this was going on, too, using the laptop and the desktop.

Later note: It proved simpler and cleaner to symlink /var/lib/docker to a location with lots of space on my spinning disk. Normally when I test this docker container, I want it to start from nothing (i.e. preserve no data from previous runs) so there's no advantage to mounting external storage.

After many iterations of build, run, watch jobs fail, figure out missing packages, tweak scripts, and round again, I have five jobs building successfully, and another four remain to be tested. I was running my docker instance with local data like this:

docker run --name teijenkins -p 8080:8080 -p 50000:50000 -v /var/jenkins_home tei-jenkins

but that was problematic because the image was exploding in size as it checked stuff out and built jobs, and I ran out of space on my SSD. I copied the contents of the /var/jenkins_home directory to my spinning disk and I'll be trying this instead:

docker run --name teijenkins -p 8080:8080 -p 50000:50000 -v /home/mholmes/WorkData/tei/jenkins_home:/var/jenkins_home jenkins

but I have to free up enough space to run the image in the first place. Working on that now.

Working again on a docker version of teijenkins, I've decided to base it off the official jenkins:stable image, which in turn inherits from openjdk:jdk-8, which comes ultimately from debian jessie. It's going OK: I have all the required stuff for TEI builds installing correctly, rnv building, and Jenkins running, but I'm failing to pre-install the Jenkins plugins we need right now.

Once you have an instance running, this is how to attach to it and get a shell (assuming the running instance default app is not a shell that you can use, as is the case with a Jenkins server):

sudo docker exec -i -t teijenkins /bin/bash

where "teijenkins" is the name of the running instance, set by using e.g.:

docker run --name teijenkins -p 8080:8080 -p 50000:50000 -v /var/jenkins_home tei-jenkins

The attempt to build a Jenkins server using a pure shell-script approach, which worked for Ubuntu 14.04, is now problematic for 16.04 and in particular for current versions of Jenkins, so I'm taking a different tack and trying to create a Docker image. Early steps are going well; it remains to be seen if I can get the whole thing to work, but in the meantime the learning is generally useful.

Reminding myself of the 3 steps to ensure UTF gets from form via php to mysql database and back via php to report:

1) In each php page put the following meta tag in the head element
<meta charset="utf-8"/>

2) Immediately after creating the object that connects to the DB tell it to use UTF8
mysqli
$mysqliConn = new mysqli($DBHost, $DBUser, $DBPass, $DBDatabase);
$mysqliConn->query('SET NAMES utf8');
$mysqliConn->query('SET CHARACTER SET utf8');
PDO
$conn = new PDO("mysql:host=$servername;dbname=$dbname;charset=utf8", $username, $password);
$conn->exec("set names utf8");

3) In the database (and each table) make sure the character encoding / collation is something like utf8_unicode_ci

As we move towards deploying standalone eXist/Jetty applications for our projects, we're figuring out how best to configure them. One issue is that we're probably going to want to point the subdomain (graves.uvic.ca or whatever) at the /apps/graves/ subfolder, but we're still going to need access to some of the default eXist applications such as eXide and the dashboard. This can be accomplished by adding the following line to the webapp/WEB-INF/controller-config.xml file:

<root pattern="/apps/graves/apps" path="xmldb:exist:///db/apps/"/>

Add it as the first entry before similar entries. The effect of this is to leave all the existing graves app functionality handled by the apps/graves/controller.xql, but hand anything accessed on /apps/graves/apps to the appropriate app controller. My testing with eXist 3 RC1 confirms that this works; it should mean that on going live, the dashboard (for instance) would be accessible on graves.uvic.ca/apps/dashboard (and we can access it over TLS for better security when logging in).

Due to a bug in the svn client in Jenkins, svn externals cannot be handled, so I had to set the Jenkins jobs to "ignore externals" (the default), and then check for the presence of the external file(s) at the beginning of the build process; if they're missing, ant will do an svn export using exec to put it where we need it. Curses. But working now.

I'm blogging this because the Red Bean book is confusing, various different syntaxes have worked in different svn versions, and it took me a few minutes to figure out what would work.

I wanted to centralize some XQuery modules that will be shared between different projects in Endings (and elsewhere potentially). This is how I ended up doing it (example from Mariage):

[In the Mariage repo, "static" folder:]
[Create a local folder in the repo for all externals:]
svn mkdir externals
cd externals

[Specify a subfolder name which will be external, along with the remote path to the source repo:]
svn propset svn:externals "xquery https://revision.hcmc.uvic.ca/svn/hcmc/xquery" .

[svn up and svn commit]

A number of jobs were not configured to discard old builds, so I had to manually prune them by working on the filesystem. There were a lot of plugin updates to do as well, and a surprising number of plugins are installed; it seems to me that a lot of basic functionality is now handled through plugins rather than bundled into the main app, so the number of plugins has proliferated.

Our Android tablet is sluggish and not really useful, so when I saw a post about the release of this useful tool, I used it to install Cyanogenmod 12.1 on the tablet. Works a treat, and is much faster. Keep this tool handy.