Maintenance

These are my ongoing notes on how to get Jenkins up and running on a headless Ubuntu 18.04.

• Initial install: make sure you configure LVM to provide a big enough volume for what Jenkins needs in the root filesystem. By default, the Ubuntu installer will create a volume of only 4GB; if that happens, extend it as needed from the remaining free space.
• OPTIONAL: Get rid of cloud-init; it's installed by default but I see no value in it:

  
      wait until the VM boots
      login
      echo 'datasource_list: [ None ]' | sudo -s tee /etc/cloud/cloud.cfg.d/90_dpkg.cfg
      sudo apt-get purge cloud-init
      sudo rm -rf /etc/cloud/; sudo rm -rf /var/lib/cloud/
      reboot
  

• Reconfigure the timezone to America/Vancouver, since it may be UTC by default:

  sudo dpkg-reconfigure tzdata

• You must

  sudo apt-add-repository universe

  before installing Jenkins, because it requires a package called daemon which is in that repo. Other stuff needs it too.

• sudo apt-get install ntp

• Install subversion. Although Jenkins has its Java implementation, some of our jobs use the command line app to get info.
• You must install Java 8 (openjdk-8-jdk), which is in the universe repo.
• Install jing. This will probably install a newer Java and make it the default, so

  sudo update-alternatives --config java

  , and set it back to 8.
• Install ant, ant-contrib, and libxml2-utils.
• Install linkchecker.
• Jenkins setup requires that you configure a relative path to the files. If you're runing a vm and proxying the port to your own machine, beware: don't choose the selected port on which you're viewing Jenkins. Instead, choose the port that you know Jenkins is running on IN THE VM. If you screw this up, you can fix it by editing jenkins.model.JenkinsLocationConfiguration.xml in /var/lib/jenkins.
• Install sendmail so Jenkins can send email.
• When setting up Jenkins, accept the suggested package of general-use plugins.
• In Jenkins, install log parser plugin and the ThinBackup plugin.
• In the Jenkins configuration, set the svn version to 1.7 (matches our revision server).
• In the Jenkins global security settings, choose "Project-based Matrix Authorization Strategy" and allow anonymous users to read the appropriate stuff.
• Once Jenkins set up and running, edit /etc/default/jenkins to add an appropriate value for the CSP policy. For example, for an utterly permissive policy, do this:

  JAVA_ARGS="$JAVA_ARGS -Dhudson.model.DirectoryBrowserSupport.CSP=\"\""

  to set the CSP allowing HTML to work properly. Consult docs on CSP to decide exactly what you need.
• As the jenkins user, create a directory /var/lib/jenkins/backups, then use that in the configuration of the ThinBackup plugin. I use this for the full backup schedule: H 12 * * 6 and this for the differential: H 12 * * 1-5
• For Mariage only: install imagemagick and libvips-tools.
• We usually use a configuration where the job workspaces are inside the job folder itself. To make this work, edit jenkins/config.xml, comment out the workspaceDir line, and replace it:

  <!--<workspaceDir>${JENKINS_HOME}/workspace/${ITEM_FULL_NAME}</workspaceDir>-->
    <workspaceDir>${ITEM_ROOTDIR}/workspace</workspaceDir>
  
  

• For Moses project only:

  sudo apt install python3 python3-pip python3-lxml

  sudo -H pip3 install dicttoxml nltk numpy plotly jupyter

  Then install nltk punkt data:

  sudo python3

  import nltk

  nltk.download('punkt')

• Make sure that the folder nltk_data ended up in /usr/share. If it didn't, copy it there. It seems to end up in the wrong place often.

After rebuilding my desktop, VirtualBox worked fine except that my Windows VM could not connect to the shared folder I had configured for it. After finding this page, I was able to run this command:

VBoxManage setextradata "Windows 10" VBoxInternal2/SharedFoldersEnableSymlinksCreate/vmShare 1

where "Windows 10" is the name of the VM, and "vmShare" is the name of the share, then restart the VM, and it worked. Obscure thing, so I document it here. The issue is that the path to the shared folder depends on a symlink on my system, since the shared folder lives on an encrypted data drive, not the system drive.

Decided to upgrade my desktop, and encountered a known common scenario whereby both gdm and lightdm seem to get borked and no GUI can be started. Eventually did a reinstall. Note to self: once email and browsers are properly set up and stable, move those home folders onto the spinning disk so that they don't have to get recreated when rebuilding.

In a previous post I went through in detail how to set up the virtual host for Apache and to configure the /etc/hosts file, but I didn't give much detail on Apache itself. I've just gone through the whole process again, and I'm now documenting that as an addendum:

Configure required apache modules:

sudo a2enmod proxy
sudo a2enmod proxy_http
sudo a2enmod ssl

Generate a certificate for the server to use:

sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/ssl/private/spud.key -out /etc/ssl/certs/spud.crt

Restart Apache:

sudo systemctl restart apache2

Of course the name and location of the cert must match the name and location specified in the virtual host file.

I'm now able to build the trunk of the eXist repo to get a working (but not yet tested) eXist 4. I can start testing our apps with it now.

A bit late coming to this, and it took an hour or so to get everything back up and running, but fonts are way nicer and the Japanese input works better.

Put the new HD into the machine and tried to do a direct dd copy from the old HD in a USB cradle, but the operation failed at the same point twice (after about 15GB). In the end, I created a new encrypted volume on the new disk, formatted it as ext4 and set an scp job going to bring everything down from Rutabaga. The results will be pretty messy, I know, but there's no real alternative. I'll have to do a bunch of chmods to handle .svn folders and such; it may be simpler to check things out again. If it's possible to get my VMs off the old drive, that's the next thing to try because those aren't backed up. Only the Windows one gets used, though, and that's easy to rebuild if necessary.

In prep for RA work, installed QGIS locally and worked through the initial parts of the tutorial. Will keep going with this. So far it's pretty straightforward.

Now that Open Layers fully supports vector layers, we're looking at the practicality of running a vector tile server for our projects. Starting from this docker example, I created a script which I can run on a standard Debian Stretch install to create a working tile server:

#!/bin/bash

#This is to be run on a standard Debian Stretch install.

#Install core stuff
apt-get update && DEBIAN_FRONTEND=noninteractive apt-get -y install apt-transport-https curl unzip build-essential python libcairo2-dev libprotobuf-dev xvfb 

#Temporarily use a specific source for the exact nodejs version we need.
echo "deb https://deb.nodesource.com/node_4.x jessie main" >> /etc/apt/sources.list.d/nodejs.list
echo "deb-src https://deb.nodesource.com/node_4.x jessie main" >> /etc/apt/sources.list.d/nodejs.list

#Install it.
apt-get -qq update && DEBIAN_FRONTEND=noninteractive apt-get -y --allow-unauthenticated install nodejs 

#Now remove the source.
rm /etc/apt/sources.list.d/nodejs.list
apt-get clean

#Added these in order to get the npm install to run properly. 
#The problem was building canvas.
apt-get -y install libjpeg62-turbo-dev libpango1.0-dev libgif-dev g++

#Create directory for tileserver application.
mkdir -p /usr/src/app

#Get the Klokantech code for the server.
cd /usr/src/app 
curl -L https://github.com/klokantech/tileserver-gl/archive/master.zip -o master.zip
unzip master.zip 
cp -r tileserver-gl-master/* ./
rm -rf tileserver-gl-master

#Install the node stuff
npm install --production

#Set environment variable
echo NODE_ENV=\"production\" >> /etc/environment

#Create the folder for the mbtiles files (you'll need to supply these later).
mkdir /data

#In case other servers are installed and running, stop them.
systemctl stop apache2 mysql
systemctl disable apache2 mysql

echo "Now put your mbtiles files into the /data folder, and run /usr/src/app/run.sh."
#Start the tileserver on port 80.
#run.sh

This could form the basis for a VM-based tileserver for our projects, including the Confederation Debates; running a server for all of Canada is quite practical due to the efficiency of the vector format.

Just recording this because it took a little while to figure out:

We discovered this morning that the B and T conference site program would not print past the first page; it was as if the rest of the content was invisible to the print renderer (in all browsers). The culprit turned out to be this, in style.css:

.internal {
  position: relative;
  overflow: auto;   <<------------------------this line
  width: 100vw;
  height: 100vh;
  -webkit-overflow-scrolling: touch;
}

It hides anything beyond the viewport window from the print renderer. I was able to override by adding:

@media only print {
  .internal {
     overflow: visible;
  }
} 

I let PS know about it, since it's his stylesheet.