Maintenance

My machine at home has one 4K monitor on the right and a 1920x1200 monitor on the left, which is in portrait mode. I'm using Wayland.

Getting the monitor settings to persist through reboots and screen locking/sleeping is problematic. Even worse, when the screen blanks (locking) and is subsequently unlocked I lose my rotation and the resolution drops to 1680x1050.

Annoyingly, the max resolution of 1920x1200 is not available at this point (no idea why) so, although I can rotate it, the resolution is rubbish. At this point I can only reboot the machine to get things back to a usable state.

The problem has been partially resolved by setting kernel args in grub (video=DP-1:1200x1920@60). Although this makes my settings persistent through power cycles it has no impact on the screen blanking issue.

I came upon a suggestion to add my monitors.xml to the gdm settings, which sounds reasonable:
sudo cp ~/.config/monitors.xml /var/lib/gdm3/.config/
sudo chown gdm:gdm /var/lib/gdm3/.config/monitors.xml

Apparently the monitor layout will persist across restarts and screen locks.
We'll see.

Note to self, very handy thing:

avahi-browse -vatr

To get Thunderbird working with UVic's Exchange calendar you need to have a version of Thunderbird that incorporates the fix for sending passwords in the clear - this means at least v43. There is a version 44 beta out right now that works quite well, so I went through the process of documenting how I got things going. This also works with v45, which was just made available on the PPA.

Install

1. Add the PPA and install/upgrade in the usual apt-get way.
2. download Lightning plugin - note Thunderbird version requirements for each version
3. download the Ericsson exchangecalendar plugin - currently v3.6-beta1

Email account settings
Account settings->Server settings
Server name: mail.uvic.ca, port 993
Connection security: SSL/TLS
Authentication method: Normal password
Outgoing server (smtp): mail.uvic.ca, port 465
Outgoing server connection security: SSL/TLS
Outgoing server authentication method: Normal password

Configure calendar

1. Click the 'Switch to the calendar tab' button (top-right) to view your local calendar (called 'Home').
2. Add new calendar by right-clicking below your 'Home' calendar, and choose 'New Calendar'. Select 'On the network' for location.
3. Select 'Microsoft Exchange 2007/2010/2013' for format, then name the calendar.
4. Check the 'Hosted Exchange' option, not the 'Office 365' option.
5. Check 'Use Exchange's autodiscovery function. Fill in Username (Netlink ID) and Domain (UVIC). Click 'Perform autodiscovery'. You should be challenged for a password.
6. If you got this far you'll be asked to choose an EWS server. I only got one to choose from: https://mail.uvic.ca/ews/exchange.asmx

We have a shared calendar for vacations and so forth. To share a calendar, log in to the account that owns the calendar (through OWS) and right-click the calendar you want to share, and choose 'Share calendar'. In the dialogue that appears, add the email address of the person you want to share with (note that this can only be a UVic email address). It will send an email to this person that has an XML attachment. They can create a new calendar based on the information included in this attachment.

Add shared calendar

1. The email that the share recipient gets has an XML file attached (called 'shared_metadata.xml'). It contains two things we need: SmtpAddress, which is the email address of the person that shared the calendar; and FolderId, which is a very long string (hex?). These are XML elements, and we only need the text values, nothing else.
2. Add a new calendar as above, but in the 'Primary email address' field, use the email address of the person that shared the calendar - not yours.
3. Use your Netlink ID in the 'Username' field.
4. In the 'Share folder Id' field, add the hex value of the FolderId element (from the shared_metadata.xml file).

That should be it. I've had no trouble with this setup, other than being unable to see nested folders - but this problem has been there for ages. For example, I have a folder for the Humanist list that I've placed inside a 'Lists' parent folder. Thunderbird shows me the Lists folder, but not the Humanist folder. This is true for all folders more than one layer deep.

I wanted to create a new 'sub-repo' in the main hcmc repo for the SVG graphics I've been re-producing ('brand' grpahics and so forth), but I forgot how to add a sub-dir to an existing repo without checking out the parent - and I definitely didn't want to do that.
So here's how:
mkdir graphics
svn import graphics https://revision.hcmc.uvic.ca/svn/hcmc/graphics
Now I can populate the graphics dir and svn commit.

I've been meaning to try this for quite a while. The idea is that you can boot a machine over the network in order to install a new OS (or load a livecd, thin client, or whatever). It requires participation from the NETS folks, but they're happy to help out. Here are my notes:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This process requires that a DHCP template get created for our VLAN.
The network folks need to know the IP of the tftp server and the filename of the executable (in the format $tftproot/path/to/$filename.efi.signed)

On the server end (following this: https://wiki.ubuntu.com/UEFI/PXE-netboot-install),
1) sudo apt-get install tftpd-hpa inetutils-inetd
2) sudo nano /etc/default/tftpd-hpa
append this to the file:
RUN_DAEMON="yes"
OPTIONS="-l -s /var/lib/tftpboot"
3) sudo nano /etc/inetd.conf
append this to the file:
tftp dgram udp wait root /usr/sbin/in.tftpd /usr/sbin/in.tftpd -s /var/lib/tftpboot
4) sudo service tftpd-hpa restart

Once the DHCP template is set up and the tftp server configured you can deploy your netboot stuff

At a bare minimum you'll need to
1) Create a $tftproot/grub directory
2) Download the signed grub efi executable to $tftproot/grub:
wget http://archive.ubuntu.com/ubuntu/dists/trusty/main/uefi/grub2-amd64/current/grubnetx64.efi.signed
and put it here: $tftproot/grub/grubnetx64.efi.signed -
Note that, although it can technically go anywhere, it must end up where the DHCP template expects it to be.
It also MUST use the '.signed' file extension. Using plain '.efi' will stop this from working (I think it's a UEFI thing)
** the md5sum of the trusty version and the wily version are identical, so we should abe able to use them interchangeably
3) Download netboot.tar.gz (the following is trusty with a 4.x kernel):
wget http://archive.ubuntu.com/ubuntu/dists/trusty-updates/main/installer-amd64/current/images/netboot/netboot.tar.gz
Extract package: tar -zxvf netboot.tar.gz
3) Create a grub.cfg file at $tftproot/grub/grub.cfg to get things going. It at least needs to define one target. Here's an example:
################################
### grub.cfg using a preseed ###
menuentry "Install Ubuntu" {
set gfxpayload=keep
linux /ubuntu-installer/amd64/linux gfxpayload=800x600x16,800x600 -- auto=true url=http://YOUR_PRESEED_SERVER/preseed.cfg quiet
initrd /ubuntu-installer/amd64/initrd.gz
}
################################

Also note that the client machine will need PXE boot enabled (and possibly secure boot disabled)

I've run through this four times successfully, so I'm calling these instructions final.

Required software

• MySQL Community Server (v5.6): http://dev.mysql.com/downloads/mysql/
• MySQL Workbench (v6.2): http://dev.mysql.com/downloads/workbench/
• MS SQL Server 2008 R2 (Express Edition): http://www.microsoft.com/en-ca/download/details.aspx?id=30438
• MS SQL Server Management Studio: available from the download window on above URL.

Note: the Microsoft downloads are named SQLEXPR_x64_ENU.exe and SQLManagementStudio_x64_ENU.exe

Once everything is installed and configured, make sure that MS SQL Server and MySQL server are running (type services.msc at a command prompt).

Import CGWP data using SQL Server Management Studio.

1. Connect to Server popup should provide sufficient default settings. Click 'Connect'.
2. Object Explorer displays a list of objects, including 'Databases'. Right-click 'Databases' and choose 'Restore Database...'.
3. To restore from the CGWP dump, Choose 'From device' as the 'Source for restore' and click the browse button to find the dump file. Select the binary dump file (you need to choose 'All Files(*)' in 'Files of type' before you can see the file). It will show up in the 'Select the backup to restore' list if the file is usable. Check the 'Restore' box beside the backup set you want to restore.
4. In the 'Destination for restore' field, copy the name of the backup set from the 'Select the backup sets to restore' panel. A database with this name will be created as part of the restoration process. Click 'OK'. Note that you will get an error if you try to restore a backup set to a database with a non-matching name.
5. You should get a message telling you that the restore was successful. You can quit the Management Studio now.

Migrate existing MS SQL database to MySQL using MySQL Workbench

NOTE: everything about the MySQL Workbench under Windows seems a bit sluggish. Be patient after clicking buttons.

1. We need a connection to a running instance of MySQL. On the MySQL Workbench home page, click on the ⊕ beside 'MySQL Connections' at the top of the page to launch the 'Setup New Connection' wizard.
   • Provide a 'Connection Name'. Click 'Configure Server Management...' button (bottom-left) to launch the 'Configure Local Management' wizard.
   • On the first page of the 'Configure Local Management' wizard, click 'Next'. You'll be prompted for the MySQL root password. Provide it (select 'Save password in vault' or you'll have problems later). Click 'OK'.
   • As long as a connection could be made, this page ('Testing the Database Connection') is merely informative. Click 'Next'.
   • If there is only one instance of MySQL running, click 'Next' on the 'Set Windows configuration parameters for this machine' page.
   • As long as a connection could be made, this page ('Testing Host Machine Settings') is merely informative. Click 'Next'.
   • Review settings or click 'Continue'.
   • Back in the 'Setup New Connection' wizard, click 'OK'.
2. Choose the 'Database Migration' shortcut on the right.
3. If you have already set up a system DSN for the MS SQL DB, click on 'Start Migration'. If you haven't, create one now.
   • Click the 'Open ODBC Administrator' button.
   • Choose 'System DSN' tab and click the 'Add...' button.
   • Select a driver: SQL Server Native Client 10.0 (although both will work it appears that this is the MS-preferred driver)
   • Configure the ODBC connection (see 'DSN setup' instructions below)
4. Click the 'Start Migration' button.
5. On the 'Source Selection' page, choose 'Microsoft SQL Server' as the Database System, and 'ODBC Data Source' as the Connection Method. You should see the DSN you created in the 'DSN' filed. Remove 'sa' from the 'Username' field if present. Click 'Next'.
6. On the 'Target Selection' page, choose the 'Stored Connection' you made in step 1. Click 'Next'. The 'Fetch Schema List' page should complete on its own without configuration or interaction. Click 'Next'.
7. On the 'Schema Selection' page, choose the database you want to migrate (it will have the name of the database you restored in MS SQL Server Management Studio). You can leave the 'Schema Name Mapping Method' alone. Click 'Next'.
8. The 'Reverse Engineer Source' page should complete on its own without configuration or interaction. Click 'Next'.
9. On the 'Source Objects' page, you will be prompted to select the objects you want to migrate. In this case we have: 'Migrate Table objects'; 'Migrate View objects'; and 'Migrate Routine objects'. I have been unsuccessful in migrating anything other than the tables, and, having looked through the views and stored procedures (aka Routine objects) I think it's safe to ignore them.
10. The 'Migration' page should complete on its own without configuration or interaction. Click 'Next'.
11. On the 'Manual Editing' page you can review any objects before the actual migration takes place. I consistently get 55 warnings and no errors. The warnings I've seen are collation issues (e.g. 'Collation SQL_Latin1_General_CP1_CI_AS migrated to utf8_general_ci') and NULL value problems (e.g. 'Default value N'NA' is not supported.' and 'Default value N'SUR' is not supported.'). As far as I can tell, they can safely be ignored.
12. On the 'Target Creation Options' the only thing selected by default is 'Create schema in target RDBMS'. This is sufficient for our needs. Click 'Next'.
13. If this is the first time through, the 'Create Schemas' page will run on its own. If it finds an existing MySQL schema you will need to allow MySQL Workbench to drop it.
14. The 'Create Target Results' page allows you to edit the individual SQL solutions produced by MySQL Workbench to do the actual migration. Leave this alone. Click 'Next'.
15. On the 'Data Transfer Setup' page 'Online copy of table data to target RDBMS' should be checked. You can also opt to create a batch file that will 'copy the data at another time'. Click 'Next'.
16. Copying the data will take a few minutes, and a progress bar will be displayed while the copy is running. Anything else is likely the result of a config error.

DSN setup for SQL Server Native Client 10.0 Driver

• Name: CGWP
• Description: ODBC connection to CGWP backup
• Server: (local)\sqlexpress (or $MACHINENAME\sqlexpress)
• Click 'Next>' button
• How should SQL Server verify the authenticity of the login ID? With Windows NT authentication using the network login ID = checked
• Connect to SQL Server to obtain default settings for the additional configuration options = checked
• Click 'Next>' button
• Select the 'Change the default database to:' option to include the database in the DSN - it's helpful. The name of the database will show up in the dropdown list as long as a connection can be made. Other than this, leave the other settings alone.
• Click 'Next>' button
• You can leave the settings on this panel alone.
• Click 'Finish' button to review/test the config. Note that a successful config/test connection does not always mean that you won't have problems later in the migration process!

MySQL Workbench can also dump the data from the new MySQL database. Just open up the MySQL connection that you created at the beginning and select 'Data Export' from the navigator on the left. There's a lot of data, so be sure to select complete insert statements in the advanced export options.

eXist 2.2 serializers now use the XQuery 3.0 rec. which breaks up the serialize instructions into individual declarations. So, according to the spec, you serialize as XHTML 5 like this:

declare namespace output = "http://www.w3.org/2010/xslt-xquery-serialization";
declare option output:method "xhtml";
declare option output:html-version "5.0";
declare option output:media-type "text/html";

Except, it looks like eXist may have a bug in there, because the above produces html without a doctype. To get around this use xhtml5 instead of just xhtml in the output:method declaration.

Also, we're using "text/html" for media-type instead of "application/xhtml+xml" because some stuff just doesn't work using xhtml+xml (Google Maps, specifically).

NOTE: Remember to change your page to use xquery 3.0

After numerous tries with various approaches, I finally managed to end up with a 40 Gig virtual machine to host Windows.
The procedure that worked for me was to
- clone the existing hard drive using the function in the GUI of Virtual Box (and not the command line). VB created a new folder and put the new virtual machine file into that folder.
- go into the command line and make the virtual file 40 gigs in size
sudo VBoxManage modifyhd newVirtualMachine.vdi --resize 40960
- start up virtual box
- select the new virtual machine
- click on settings, click on disk, right click, choose "Remove"
- still on settings, right
- in Windows, start Windows Computer Management
- go to Disk/Storage management
- right click on existing small partition, select "extend"
- click OK in dialog box

When I tried using the command line to clone the virtual drive, I got the file, but the GUI in Virtual Box didn't see it.
I tried a couple of other partition managers before stumbling on the Windows Computer Management and either I couldn't get them to install properly or I couldn't get them to extend the smaller partition to a bigger one.

The most helpful website I found was
http://tips.kaali.co.uk/2012/03/16/expand-or-increase-the-size-of-virtual-box-vdi-dis/ though I did not create the hard drive and then clone the existing drive to the new drive as that page says, rather I cloned the existing drive and then resized it to 40G.

Finally, when I started windows in the new virtual machine, I got a "this copy of windows may not be authentic" warning. I followed the link on the warning and that "authenticated" my copy.

Looking at the Jenkins interface this morning, I realized that I'd forgotten to give the anonymous user read access to jobs, so I've done that. I also tracked down the issue with the stylesheets to (I think) a permissions problem with the oXygen installation; it was installed via sudo, so all the files ended up owned by root, whereas the job is running under a different user. I've chowned everything to hcmc:hcmc, and we'll see if that solves it, but it might have to be chowned to whatever user Jenkins runs under. Running a stylesheet build now to see if it works.

EDIT: But it doesn't. The same error shows up even when java is world r-x. Posted on the oXygen forum for a possible solution, and also heard from SR that he remembers this but doesn't remember how he fixed it.