Now that the required ports are opened up and working for both servers, I decided to see how straightforward it would be to get Let's Encrypt certs set up for the subdomains. The answer turns out to be that it's trivial:

  sudo add-apt-repository ppa:certbot/certbot
  sudo apt install python-certbot-apache
  sudo certbot --apache -d teijenkins.hcmc.uvic.ca
  sudo certbot renew --dry-run

The certbot did an automated challenge by temporarily tweaking the apache config and testing the tweak; then it downloaded and installed the certs, and even updated the virtual hosts file to point at them. The last line tests the renewal process, which has to happen every three months, and for which certbot has installed a cron job. If that fails, you get an email.