Maintenance

I've always been frustrated by only being able to have one copy of Oxygen running. I like to work on different projects in different workspaces, so when something large is building, I can just switch to another project for a bit. I discovered this:

https://www.oxygenxml.com/forum/topic14636.html

which solves the problem. Edit the oxygen.sh file in the Oxygen folder and add this:

-Dcom.oxygenxml.MultipleInstances=true\

in the set -- \ section. Then edit the .Desktop file that starts the Oxygen editor and comment out the existing line, which looks like this:

Exec="/home/mholmes/Oxygen XML Editor 23/oxygen23.1" %U

and add this one instead:

Exec="/home/mholmes/Oxygen XML Editor 23/oxygen.sh"

You have to omit the %U. Then you can start Oxygen as many times as you like. However, the last one to close will dictate the remembered settings.

It looks like the apt-mirror script project is dead. The repo directory structure has changed since 2018 and when I added 20.04 to the mirror it did not download a bunch of stuff.

So, I'm testing debmirror. Which also has problems, but is still being developed. The version of Debian I'm using on guava uses an old version of debmirror, so I grabbed the script off of the git repo and installed it locally.

Along with a wrapper script it seems to be much easier to use than apt-mirror was. Fingers crossed it works well when I start to run it as a cron job.

With the new mirror I've had an initial success with a 20.04 PXE build of one of our new lab NUCs.

For the last couple of weeks I've been noticing some very weird behaviour in the two Jenkins build jobs where the svn repo has GitHub externals. The projects simply build all the time, whether or not there's been any change to any of the repos (svn or git).

We can't have that, of course; it's only happening to two repos right now, but it as more mini sub-projects get shared between repos, we'd end up with Jenkins simply queuing builds endlessly.

I'm still researching this, but I suspect it might have something to do with branch information; when (say) the DVPP repo is subscribed to the master branch of the staticSearch git repo, the master branch may not change, but the dev branch might; that means you end up with weird svn info like this (running svn info . in the external directory inside the Keats project):

URL: https://github.com/projectEndings/staticSearch.git/branches/master
Relative URL: ^/branches/master
Repository Root: https://github.com/projectEndings/staticSearch.git
Repository UUID: 04f9d699-67a6-817a-6227-ce57b7db7bff
Revision: 588
Node Kind: directory
Schedule: normal
Last Changed Author: martindholmes
Last Changed Rev: 570
Last Changed Date: 2020-01-13 11:41:56 -0800 (Mon, 13 Jan 2020)

You'll see that there are two revision numbers; the last-changed is 570, but the current revision is 588. Presumably that's because the master branch last changed in 570 but the dev branch changed more recently (as you'd expect). I have a suspicion that the svn polling mechanism in Jenkins interprets this to mean that there are new changes, and does a build. But it may be some other bug.

Anyway, I'm thinking that it might be a better alternative to pull static copies of the remote repo contents during the build process. The only disadvantage of this is that you can't live-edit the remote repo's code in the context of one of the projects using it, then commit from there back into the real repo. But that really just means that you have to have solid testing components in the original repo rather than relying on the consumer projects to do testing during active development.

JT and I have been working on our staticSearch code in a GitHub repo. We're also both using it in projects housed in other repos, including Keats and DVPP in the UVic Subversion repos. In the process of managing that, I've found a number of useful things that I think are worth recording.

You can access GitHub git repos as if they were svn repos, using standard svn. That's useful for people who may be OK with svn already, but wary of learning git.

But because you can do that, you can also add a GitHub repo as an external in a Subversion repo. I set up a folder in the Keats repo called "staticSearch" which is defined as an external with the URL:

https://github.com/projectEndings/staticSearch/trunk

This URL tracks the default branch of the project on GitHub. The default branch in this case is the master branch. But I can switch the branch that's being tracked to another one, for example:

https://github.com/projectEndings/staticSearch/branches/dev

which is what I'm doing today. Once you've set up the external, any svn update will bring down the latest changes to the staticSearch project into the local copy of the Keats repo.

So far so good. But you can also edit stuff in that folder, and svn-commit it back to the Github repo, where it shows up as a commit in git. That means it's easy to work on the staticSearch code in the context of another project (where you have all the real data to work with) and commit your changes back to the repo; then when you've finished your dev branch work, you can merge it into master, then reset your project external to track the stable master branch instead, and then go away and work on the staticSearch dev branch in the context of some other repo.

This is really useful, and so far I'm seeing no unwanted side-effects.

The Cisco AnyConnect client is an encumbrance, especially when trying to use the VPN in a script. Fortunately, there's a workaround - openconnect. Here's how to use it:

1. Install a package: sudo apt install network-manager-openconnect-gnome
2. CLI:
   
   • Create a new interface that a regular user can use: sudo ip tuntap add vpn0 mode tun user greg
   • Connect: openconnect --authgroup=Default --quiet --interface=vpn0 --script='sudo -E /usr/share/vpnc-scripts/vpnc-script' https://vpn.uvic.ca --user=$NETLINKID
3. GUI:
   
   • open Gnome Control Centre -> Network
   • Add a VPN by clicking the + and choosing 'Cisco AnyConnect Compatible VPN (openconnect)
   • Fill in the gateway. Configuration complete.

NOTE: read the docs for more info on why things are done this way

NOTE: if you need to debug a connection, add '--dump-http-traffic' to your command

NOTE: to remove a network device do 'ip link delete $DEVICE' - e.g. 'sudo ip link delete vpn0'

NOTE: this method has been tested successfully from off-campus.

Command explained
=================

• authgroup: Default or VPN Multi Factor (you can use a YubiKey, but I don't think it has any effect in our case)
• interface: the interface created above. Note that this allows network-facing code to run without root. The vpnc script needs root, but that isn't the same, see here
• script: there is a script that gets run once the handshake is done. It's located at /usr/share/vpnc-scripts on Ubuntu.

New desktop migration process:

• Nuke Windows from new desktop.
• Install Ubuntu and all the software you can remember needing.
• Create a single encrypted partition on the spinning disk. Easy to do now in the Disks app.
• Pull that disk out, and migrate data from existing desktop encrypted drive to it.
• Put the drive back in the new machine.
• Create all required symlinks so that everything sensitive is on the encrypted drive, but the system drive is clean.
• Start work, and fix everything that's broken, install missing things, etc. as you go.

Migrated first JSRDocs_dev then JSRDocs forward to the new AdaptiveDB codebase, and tested. All seems good. That's the last of the important database projects that I'm aware of that will need porting. Backup of the db taken today; backups stored both locally and in the folder of the live site.

So far I've updated both VPN dbs and all four active Landscapes dbs to the latest codebase, meaning that they will run on both PHP 5.6 and PHP 7. For VPN and the Powell St db, I've put the following stuff in .htaccess files to ensure that we're actually running on the new server and taking advantage of PHP 7, because the speed payoff is significant:

setEnv UVPHP_VERSION 70

RewriteEngine On
RewriteBase /
RewriteCond     %{HTTP_HOST} ^hcmc\.uvic\.ca$ [NC]
RewriteRule     (.*)  https://webserver2.hcmc.uvic.ca%{REQUEST_URI}

Once the new server takes over from the old, we'll have to remove the redirects. I have one more database to port, the jsrDocs one.

Now that the required ports are opened up and working for both servers, I decided to see how straightforward it would be to get Let's Encrypt certs set up for the subdomains. The answer turns out to be that it's trivial:

  sudo add-apt-repository ppa:certbot/certbot
  sudo apt install python-certbot-apache
  sudo certbot --apache -d teijenkins.hcmc.uvic.ca
  sudo certbot renew --dry-run

The certbot did an automated challenge by temporarily tweaking the apache config and testing the tweak; then it downloaded and installed the certs, and even updated the virtual hosts file to point at them. The last line tests the renewal process, which has to happen every three months, and for which certbot has installed a cron job. If that fails, you get an email.

Spent a lot of the day on minor tweaks to the two Jenkins servers. The https problem on jenkins.hcmc.uvic.ca was a VLAN ACL issue, now fixed by NETS, so it's working OK now. Both machines now have 8GB of RAM, and the disk of the TEI Jenkins has been expanded by 50GB, so that should last for a little while. I haven't yet set up the Apache config on teiJenkins because I wanted to use the opportunity to document it properly, but it's now trying to build jobs (largely failing for various reasons relating to config).