Permalink 05:29:53 pm, by mholmes, 291 words, 13 views   English (CA)
Categories: Servers, R & D, Activity log, Activity log, Documentation; Mins. worked: 180

Jenkins on 18.04

These are my ongoing notes on how to get Jenkins up and running on a headless Ubuntu 18.04.

  • Install subversion. Although Jenkins has its Java implementation, some of our jobs use the command line app to get info.
  • You must install Java 8. The problem is that it's not available in the repo, so I had to add a webupd8team ppa and then install the Oracle version. :-(
  • You must sudo apt-add-repository universe before installing Jenkins, because it requires a package called daemon which is in that repo.
  • Jenkins setup requires that you configure a relative path to the files. If you're runing a vm and proxying the port to your own machine, beware: don't choose the selected port on which you're viewing Jenkins. Instead, choose the port that you know Jenkins is running on IN THE VM. If you screw this up, you can fix it by editing jenkins.model.JenkinsLocationConfiguration.xml in /var/lib/jenkins.
  • Install sendmail so Jenkins can send email.
  • When setting up Jenkins, accept the suggested package of general-use plugins.
  • In Jenkins, install log parser plugin.
  • Once Jenkins is running, go to Manage Jenkins / Script Console, and run this:
    System.setProperty("hudson.model.DirectoryBrowserSupport.CSP", "")
    to set the CSP allowing HTML to work properly.
  • If you create jobs by creating a folder inside jenkins/jobs and adding a config.xml file, don't forget to also create a workspace folder there, otherwise Jenkins seems to want to put the workspace elsewhere.
  • For Moses project only:
    sudo apt install python3 python3-pip python3-lxml
    sudo -H pip3 install dicttoxml nltk numpy plotly jupyter
    Then install nltk punkt data:
    sudo python3
    import nltk
    ...but there must be more because the build still fails because of missing punkt.


Permalink 09:43:26 am, by mholmes, 102 words, 8 views   English (CA)
Categories: R & D, Activity log, Documentation; Mins. worked: 30

VirtualBox weirdity: shared folders using symlinks

After rebuilding my desktop, VirtualBox worked fine except that my Windows VM could not connect to the shared folder I had configured for it. After finding this page, I was able to run this command:

VBoxManage setextradata "Windows 10" VBoxInternal2/SharedFoldersEnableSymlinksCreate/vmShare 1

where "Windows 10" is the name of the VM, and "vmShare" is the name of the share, then restart the VM, and it worked. Obscure thing, so I document it here. The issue is that the path to the shared folder depends on a symlink on my system, since the shared folder lives on an encrypted data drive, not the system drive.


Permalink 05:05:41 pm, by mholmes, 61 words, 15 views   English (CA)
Categories: R & D, Activity log; Mins. worked: 120

Borked upgrade to Ubuntu 18.04

Decided to upgrade my desktop, and encountered a known common scenario whereby both gdm and lightdm seem to get borked and no GUI can be started. Eventually did a reinstall. Note to self: once email and browsers are properly set up and stable, move those home folders onto the spinning disk so that they don't have to get recreated when rebuilding.


Permalink 10:56:48 am, by sarneil, 129 words, 82 views   English (CA)
Categories: Announcements; Mins. worked: 90

BOM in htaccess file

I was creating an htaccess file to restrict access to a folder on nfs.hcmc.uvic.ca. I used BBedit to create the file. It's configurations said it was not include a Byte Order Mark in the file (i.e. encoding was set to UTF-8, and not UTF-8 with BOM). When I uploaded the file to the server and accessed it in the browser I got an internal server error. Sysadmin checked logs and told me there was a BOM (\xef\xbb\xbf) at the start of the file. That did not appear in BBEdit (even with Show Invisibles enabled). I opened the file in Nano and no BOM was visible in that editor either. I deleted the file and retyped it from scratch in Nano, and that worked.


Permalink 01:45:12 pm, by Greg, 219 words, 42 views   English (CA)
Categories: Announcements; Mins. worked: 0

Changes in local Debian package build system

In the past I have generated local Debian packages on my workstation and uploaded them to the apt server.
Recently I moved them to the apt server itself so packages can be updated remotely if need be.
The build script is basically a fancy wrapper around a pretty standard debian build method. It downloads upstream code (like for oxygen) and re-packages it, as well as building meta packages and a config package that is site-specific.
I am currently building the following packages:

  1. hcmc-desktop - a meta package that installs packages we use like java, fonts, ssh, subversion, etc.

  2. hcmc-conf - a configuration package that includes a collection of scripts/files that set up a machine for use in our lab. This includes everything from setting JAVA_HOME and mimelists (for oXygen etc.) to auto-configuring hostname, default skel and firefox profiles.

  3. hcmc-auth-sssd - this installs and configures SSSD, which we use for LDAP authentication on our machines. It's a separate package from hcmc-conf for practical reasons.

  4. hcmc-oxygen - this is a repackaging of the oXygen XML editor for deployment in our labs. Doing this way allows easy upgrades.

  5. hcmc-style - basically just look-and-feel stuff such as wallpaper, theme, icons, and colour-schemes that I like.

The script finishes by invoking reprepro to ingest the package and offer it via apt install.

Permalink 12:55:43 pm, by Greg, 163 words, 60 views   English (CA)
Categories: Servers, Documentation; Mins. worked: 0

TFTP service documentation

The tftp boot daemon on the apt server has its configuration file in the file /etc/default/tftpd-hpa.
The config file points to the filesystem location of the deliverables and a few options.

The deliverables we're using in the netboot setup are:
grub (a directory)
initrd.gz (the primordial OS that gets booted)
linux (the kernel)

The grub directory contains:
grub.cfg (the config file that provides the menuentries you see on-screen among other things)
grubnetx64.efi.signed (the boot app that actually boots the machine - AKA bootx64.efi)
theme (a directory containing pngs and the theme.txt file that arranges the pngs on the screen)
unicode.pf2 font (I think this is fall back font that can be removed, but I haven't tested the theory).

If you need to start/stop/restart the tftp daemon:
sudo systemctl start tftpd-hpa
sudo systemctl stop tftpd-hpa
sudo systemctl restart tftpd-hpa

To watch it at work, do:
sudo tail -f /var/log/syslog | grep tftpd


Permalink 03:22:46 pm, by mholmes, 122 words, 46 views   English (CA)
Categories: R & D, Activity log, Documentation; Mins. worked: 90

Setting up Apache for testing Jetty/eXist

In a previous post I went through in detail how to set up the virtual host for Apache and to configure the /etc/hosts file, but I didn't give much detail on Apache itself. I've just gone through the whole process again, and I'm now documenting that as an addendum:

Configure required apache modules:

sudo a2enmod proxy
sudo a2enmod proxy_http
sudo a2enmod ssl

Generate a certificate for the server to use:

sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/ssl/private/spud.key -out /etc/ssl/certs/spud.crt

Restart Apache:

sudo systemctl restart apache2

Of course the name and location of the cert must match the name and location specified in the virtual host file.


Permalink 04:08:57 pm, by mholmes, 29 words, 32 views   English (CA)
Categories: Servers, R & D, Activity log, Documentation; Mins. worked: 60

eXist build script tweaked and now working

I'm now able to build the trunk of the eXist repo to get a working (but not yet tested) eXist 4. I can start testing our apps with it now.


Permalink 01:25:27 pm, by Greg, 33 words, 40 views   English (CA)
Categories: Servers, Documentation; Mins. worked: 0

Reprepro notes

While fiddling with adding a new distro to reprepro I had cause to remove one...

  1. Remove reference to it from the $repodir/conf/distributions file

  2. sudo reprepro -Vb /path/to/name-of-your-repo --delete clearvanished
Permalink 01:21:13 pm, by Greg, 207 words, 40 views   English (CA)
Categories: Servers, Documentation; Mins. worked: 0

Setting up GPG v2 key on cli

SHA1 keys are no longer recommended so I went through the process of generating a new set of keys for use on the apt server. Here's how I did it (followed this).

  1. apt install gpgv2 package. Xenial installs v1 by default. Not sure if v2 is required, strictly-speaking.

  2. Generating keys requires a quantity of entropy which can be hard to generate on a CLI system. I apt installed the pkgs rng-tools and haveged, then ran 'sudo rngd -r /dev/urandom -W 4096' which generates enough entropy for a build. You can check the available entropy by running 'cat /proc/sys/kernel/random/entropy_avail'.

  3. Create key with 'sudo gpg2 --full-gen-key'. Answer questions. No need to add a comment. Do set an unlock password, though.

  4. Result is a barf of info including a line like 'gpg: key 3GD4831G marked as ultimately trusted'. You'll ref 3GD4831G in reprepro on the SignWith line.

  5. Export an armoured public key with 'sudo gpg2 --armor --output my_public_key.asc --export 3GD4831G'. Note that the command is gpg2 and that the key has an 'asc' suffix. I have reason to believe that armoured keys that do not use either a gpg or asc suffix will eventually be ignored on import.

:: Next Page >>


This blog is the location for all work involving software and hardware maintenance, updates, installs, etc., both routine and urgent, in the server room, the labs and the R&D rooms.


XML Feeds