Maintenance

The update to MariaDB 8 has made a number of things more difficult when it comes to configuring databases. Today we worked through a number of issues with JA, and this is what we learned:

1. HCMC can grant permissions to users on specific dbs, but it only seems to work at the command line, not using the adminer form interface. Use CLI commands in the adminer interface.
2. When you grant permissions to a user using GRANT SELECT.... ON `db`.* to `user` (where user has already been created), you should NOT specify any server (no @something following the user name). If you specify nothing, you get the wildcard %, which is what you want.
3. Once you have a wildcard % in the permissions, do NOT specify any further permissions which are more specific for that user; if you do, the wildcard will be rendered useless, although you'll still see it in there. So:

- DO NOT specify a host in the command

- Thereby get a wildcard

- Don't undermine the wildcard by adding further more specific permissions.

Homebrew’s website:
https://brew.sh

Install with an account with admin privileges if possible. If that is not possible at all then you can go down this rabbit hole (if not be prepared for permissions issues):
https://superuser.com/questions/619498/can-i-install-homebrew-without-sudo-privileges

based on this (untar anywhere):
https://docs.brew.sh/Installation#untar-anywhere

Normal installation procedure:

Install command line tools if you don’t have them:
xcode-select --install

Launch the terminal, then copy this command and press enter, homebrew will install (this command is found on homebrew’s homepage)
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Now install svn by typing this command into the terminal:
brew install subversion

Common installation issues can be found here:
https://docs.brew.sh/Common-Issues

On macOS it usually involves reinstalling the command line tools after a major OS upgrade, so:
xcode-select --install

Then:
brew upgrade

Uninstalling homebrew will fix a FUBAR install:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/uninstall.sh)"

I've always been frustrated by only being able to have one copy of Oxygen running. I like to work on different projects in different workspaces, so when something large is building, I can just switch to another project for a bit. I discovered this:

https://www.oxygenxml.com/forum/topic14636.html

which solves the problem. Edit the oxygen.sh file in the Oxygen folder and add this:

-Dcom.oxygenxml.MultipleInstances=true\

in the set -- \ section. Then edit the .Desktop file that starts the Oxygen editor and comment out the existing line, which looks like this:

Exec="/home/mholmes/Oxygen XML Editor 23/oxygen23.1" %U

and add this one instead:

Exec="/home/mholmes/Oxygen XML Editor 23/oxygen.sh"

You have to omit the %U. Then you can start Oxygen as many times as you like. However, the last one to close will dictate the remembered settings.

If you want to use an emoji (like ❎) but don't want it to be green you can use the variation selector (0xFE0E)to change it to the text representation.
Use it in CSS like this:
.myClassName::after { content: \274E\FE0E; }
which will look like this ❎︎

Some emojis don't have a text representation.
To check what the text version looks like (if one exists) for any given character, go here and enter a utf-16 string (like 274E) or the character itself (❎). This is a very crude tool. Don't expect much.

I'm just posting this because I've had to spend over an hour trying to figure this out. I had what I thought was a bog-standard Ubuntu 20.04 installation with ImageMagick installed from the repos. This gives you ImageMagick 7. On my desktop, it just wouldn't work properly; attempting to process images at the command line gave me missing delegate errors:

no decode delegate for this image format `JPEG' @ error/constitute.c/ReadImage/562

I remembered having hit this before once, and knew to go and change some settings in policy.xml, but that just didn't work. Eventually I had to do this:

• Uninstall and purge imagemagick
• Find all imagemagick-related folders and files and delete them (there are lots, including symlinks to the binaries all over the place)
• reinstall imagemagick (still didn't work)
• sudo apt install graphicsmagick-imagemagick-compat

The last one seems to be the magic bullet.

I was able to get the W3C checklink tool working as a Docker container after modifying the Dockerfile found at https://github.com/stupchiy/docker-w3c-checklink/blob/master/Dockerfile to add some missing dependencies; this is my working file:

FROM ubuntu

MAINTAINER Sergiy Tupchiy <stupchiy@gmail.com>

# Install dependencies required for link-checker distribution installation
RUN apt-get update && apt-get install -y \
      cpanminus \
      make \
      build-essential \
      libssl-dev \
      curl \
      libnet-ssleay-perl \
  && rm -rf /var/lib/apt/lists/*

ENV CHECKLINK_VERSION 4_81

ARG CHECKLINK_URL=https://github.com/w3c/link-checker/archive/checklink-${CHECKLINK_VERSION}.tar.gz

RUN set -x \
  && curl -sSL ${CHECKLINK_URL} -o /tmp/link-checker.tar.gz \
  && mkdir -p /usr/src \
  && tar -xzf /tmp/link-checker.tar.gz -C /usr/src \
  && rm /tmp/link-checker.tar.gz \
  && cd /usr/src/link-checker-checklink-${CHECKLINK_VERSION} \
  && cpanm --installdeps . \
  && cpanm IO::Socket::SSL \
  && cpanm LWP::Protocol::https \
  && perl Makefile.PL \
  && make \
  && make test \
  && make install \
  && rm -rf /usr/src/link-checker-checklink-${CHECKLINK_VERSION}

ENTRYPOINT ["/usr/local/bin/checklink"]
CMD ["-h"]

I'm now able to run it using:

docker run -it --rm link-checker checklink -H --recursive https://mapoflondon.uvic.ca > report.html

The plan is to build a complete local copy of e.g. MoEML and then start a Python simple server there, and point the link checker at it.

PostgreSQL (psql) has several xml output methods.

For example, to dump a table to an XML file:

# /usr/pgsql-9.6/bin/psql -v ON_ERROR_STOP=1 -d name_of_db
name_of_db=# COPY (SELECT table_to_xml('name_of_schema.name_of_table', true, false, '')) to '/path/to/backup.xml'

There are two problems with using XML outputs:
1) The XML is pretty basic
2) Because of the way PostgreSQL works the whole transaction is done at once so memory allocation on large tables will kill the process (or the server) and you end up with messages like this:

server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.

The advice from sysadmin is don't use XML output for big things.

These are blocks of code used in the TEIP5-Test-dev and oxygen-tei-bleeding jobs respectively to create required symlinks; this requirement is now provided by a plugin.

#Move to the Stylesheets-dev job folder.
cd ../../Stylesheets-dev

#Get the last build number into a variable.
BUILDNUM=`cat builds/permalinks | grep lastSuccessfulBuild | awk -F"lastSuccessfulBuild " '{print $2}'`

#Remove any existing alias.
rm -f lastSuccessful 2> /dev/null

#Create up-to-date symlink.
ln -s "builds/$BUILDNUM" lastSuccessful

#Move to the TEIP5-dev job folder.
cd ../../TEIP5-dev

#Get the last build number into a variable.
BUILDNUM=`cat builds/permalinks | grep lastSuccessfulBuild | awk -F"lastSuccessfulBuild " '{print $2}'`

#Remove any existing alias.
rm -f lastSuccessful 2> /dev/null

#Create up-to-date symlink.
ln -s "builds/$BUILDNUM" lastSuccessful

I've finally diagnosed and fixed the build problem with the TEI Jenkins server. It was caused by a change in the way Jenkins handles the archiving of artifacts; it stopped making symlinks on the file system to point to the latest successful build products, and instead uses a text file that has to be parsed. We use those symlinks to build other jobs using the products of previous jobs, so I've added a build step that parses the text file and recreates the symlink, and that seems to be working. 120 minutes altogether, over a couple of weeks.

I think I'm getting closer to figuring out what's happening with my broken Jenkins, and it's interesting. Having reconstructed the Stylesheets-dev job from scratch, I've discovered that instead of the old system of creating filesystem aliases pointing at the last successful build, jenkins now simply maintains a text file called builds/permalinks which looks like this:

lastFailedBuild 2
lastStableBuild 3
lastSuccessfulBuild 3
lastUnstableBuild -1
lastUnsuccessfulBuild 2

and then the Jenkins web interface uses that information to make the artifacts available for download. However, there's now no filesystem path to retrieve them, so when the downstream job tries to run Ant with -DXSL=/var/lib/jenkins/jobs/Stylesheets-dev/lastSuccessful/archive/dist/xml/tei/stylesheet, it fails, because there's no such path.

I'm not sure whether this is due to a change in Jenkins itself, or a change due to a plugin update, or something else. It happened some time after June 19. I know this because if I look at the stable Oxygen plugin build, I see that the "lastSuccessful" filesystem link is pointing at build 19, but in fact both build 20 and build 21 were successful. That means that anything depending on those filesystem links may not yet know that they're broken (pointing at old builds); only when the old builds have eventually been deleted would the links point to nothing, and a dependent build then fail.

My Jenkins version is 2.189, whereas the other two TEI Jenkinses are both 2.176.2. So it's possible that my Jenkins is providing a useful warning for something that in due course will happen to the others as well.

It's also possible that it's some kind of sporadic bug that affects some Jenkins installs and not others. There is a report which looks similar from some years ago:

https://issues.jenkins-ci.org/browse/JENKINS-43540

I'm still investigating... 120 minutes...