Academic

Background research on MoEML SSHRC app is helping with lots of other stuff.

Research/writing around collation and versioning. There are some recent clear statements of the problems and approaches, but nothing really new as far as I can see.

For possible use in a MOOC being created in France. I'm breaking down existing content into 5-minute segments with very explicit instructions and screenshots, making it much more friendly for autonomous learning; the results will be very useful for our own students getting started with XSLT. The French collaborators will do the filming; we'll have access to the materials.

Very finicky process re style and submission steps. In the process, I found two bugs in the jTEI rendering to ODT, one of which I've filed as a bug on GH and the other I'm discussing with RVDB. Fingers crossed for the article.

I'm putting one of my presentations from DH this year in as a submission to DSH, so I'm revising it, writing and abstract, etc.

Some decisions and correspondence with jTEI authors; some time spent debugging build problems with the new git setups on Jenkins servers.

Met with CP re using Archive-It to ingest Mariage and other projects; determination is that simple AJAX will probably work, but everything else will require explicit (if hidden) indexing that the archive tool can follow. We also talked about the possibility of actively generating WARC files as part of our build process; that seems a reasonable idea to me.

Team meeting in the afternoon made final revisions to the application and budget.

Editors meeting 6am; follow-up work (assigning reviews, minutes etc.) after getting to work.

Final proofing and corrections for a few articles just posted.

I found a tutorial for constructing a tile server from scratch, so today at lunch I set one up.
The instructions are nearly perfect - as a result I was able to basically copy-paste the commands. However, there were a few things that I had to sort out on my own. So, here are my notes, with no attempt to be particularly coherent:

OSM build notes
Following instructions from: https://switch2osm.org/serving-tiles/manually-building-a-tile-server-14-04/
Other sites to refer to:
OSM data downloads: http://download.geofabrik.de/ (a bit slow. For the whole planet look here: http://planet.openstreetmap.org/)
OSM Bright style: https://github.com/mapbox/osm-bright
Edit OSM styles: http://wiki.openstreetmap.org/wiki/Mapnik#Mapnik_Styles
Mapnik GitHub: https://github.com/mapnik/mapnik

I notice it uses Mapnik 2, and the current version is Mapnik 3. Not sure why, but it doesn't suit current purposes to experiment with changing this.

At the bit where you create a user in postgres it says:
createuser username # answer yes for superuser (although this isn't strictly necessary)
I didn't get the opportunity to make this choice, but it didn't seem to cause any trouble.

When compiling mapnik I got the following notices:
dependencies not met for plugin 'occi', not building...
dependencies not met for plugin 'rasterlite', not building...
According to this https://github.com/mapnik/mapnik/issues/950 these notices can be ignored.

The unzip part of OSM Bright did not produce the expected ne_10m_populated_places_simple directory, but unzipped the contents in to the praent directly. I created a folder for them and continued.

I downloaded the BC dataset (350MB), which took an age, and stored it in /usr/local/share/maps/planet/north-america/canada/british-columbia. For reference, metro regions, like Victoria and Vancouver, can be downloaded from here: https://mapzen.com/data/metro-extracts

The process can only be launched by a pg user, so in this case, I changed to the osmdbadmin user and ran:
osm2pgsql --slim -d gis -C 16000 --number-processes 3 /usr/local/share/maps/planet/north-america/canada/british-columbia/british-columbia-latest.osm.pbf
which returned the following error: Out of memory for dense node cache, reduce --cache size
and died. The VM had 4GB RAM. I gave it 16GB and it worked.
NOTE: using something other than the whole planet causes grief with the osmbright.xml settings - layers have a very specific idea of what zone is being rendered. Specifically, I got an error about parsing the srs attribute in the osmbright.xml file for the land-low layer. The line in question looks like this:

<Layer name="land-low" status="on" srs="+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0.0 +k=1.0 +units=m +nadgrids=@null +wktext +no_defs +over">

After a few false starts I have tiles rendering.
I had to increase the disk size in the VM because disk use had topped 100GB (I cranked it to 500GB using 'VBoxManage modifyhd "/path/to/HDD.vdi" --resize 500000'). I also cranked up the swap partition size to 50GB by starting the VM with an install disk and running gparted - I had to move the swap to the end of the disk in order to extend the existing ext4 partition.

It appears that when OSMBright.xml gets compiled using make.py it doesn't pay attention to the fact that you've added a 'type' declaration. I got errors about a required attribute ('type') not being available. I solved it by manually adding it to the OSMBright.xml file. It would be nice if I could figure out why I had to do that, though.

Also, the max zoom seems to be hard-coded to 18 in mod_tile. It looks like you can change it in the mod_tile source file 'renderd.py', located in the root directory of the source code pulled from github. It looks like the stock setting is 18, and I found ref's that suggest that it cannot be set higher than 20. I haven't tried yet.
NOTE: it *may* be adjustable through the renderd.conf setup. See here.

=========================================================================================
Resources for next steps
A 'standalone' tile generator. Part of Mapnik
TileMill, which has apparently been superseded by
MapBox Studio
Alternate styles for generating tiles. This one look cool, but would need work to incorporate it. I think TileMill might be useful here.