HCMC Blogs

Last day of flood week.

Before, alpha sorting distinguished umlauted characters from non, so would sort like this:
Baa
Bad
Bag
Bäa
Bäd
Bäg
Whereas HT wants
Baa
Bäa
Bad
Bäd
Bag
Bäg
I created a person_name_normal field and put into it the person_last_name field, and then went through and replaced all instances of ä with a, ö with o and ü with u, so the normalized field no longer has the umlauted characters.

I now have to revise the code for output so that it includes the person_name_normal field and sorts on that field rather than on person_last_name.
I also have to revise the code that populates the database so that it properly includes the normalized version of the person_last_name automatically.

We wanted a folder called OralHistoryDocs to contain two subfolders "originals" and "redacteds". We wanted the originals folder to be not accessible other than to members of a specified user gropu (i.e. didn't want people able to browse to that folder in the web interface).

Here’s the deal with “hidden” folders in svn
To make a folder hidden, you have to add something like this to the svnusers file (user landscapes/oralHistoryDocs/originals as the example):
[landscapes:/oralHistoryDocs/originals]
* =
@landscapesradisheds = rw
The first line specifies which repo and path to affect
The second line denies read and write access to all users
The third line allows read and write access to members of the group landscapesradisheds

Results In the web-based interface:
- if I browse to landscapes/oralHistoryDocs I do not see the originals folder even if I'm logged in as a member of the landscapesradisheds group
- if I browse to landscapes/oralHistoryDocs/originals logged in as a member, I see the contents of that folder. I haven't tested this, but I'm assuming that if I'm not a member of the group, the authentication will fail and I won't get in.

Results on the command line:
The hidden folder seems
- to be invisible to the svn crawler
- has a record in whatever db structure svn uses
- that it's contents can be addressed

If I create an oralHistoryDocs folder locally and then svn checkout the oralHistoryDocs folder from the server, I do not get the originals subfolder, nor any of the files in it. If I then try to create an originals folder on the local copy and svn add it to the repo, I get at “originals folder already exists, so can’t be added” error from svn. So, there seems to be no way to get a nested svn-hidden folder to behave at the command line like an svn-normal folder.
On the other hand, if I create an originals folder locally and then from within that folder do an svn checkout of the originals folder from the server, I get the contents of the originals folder from the server. I can update, add, delete commit as normal. So, for each hidden folder in the server copy of the repo, the user would have to create a separate svn repo for just that folder and execute svn commands as usual, which will update the nested folder on the server.

Structure on server:
landscapes
- oralHistoryDocs
- - originals
- - - a.txt
- - - b.txt
- - redacteds
- - - c.txt

Structure on local
landscapes
- oralHistoryDocs
- redacteds
- - c.txt
oralHistoryDocsOriginals
- a.txt
- b.txt

I suspect the commands are being run as some kind of system user (which is not a member of the landscapesradisheds group), but have done no research.

In the office for most of the day.

Unable to do anything 2020-02-05 due to flooding situation at home.

worked 3 hours.

Second day working from home.

For the last couple of weeks I've been noticing some very weird behaviour in the two Jenkins build jobs where the svn repo has GitHub externals. The projects simply build all the time, whether or not there's been any change to any of the repos (svn or git).

We can't have that, of course; it's only happening to two repos right now, but it as more mini sub-projects get shared between repos, we'd end up with Jenkins simply queuing builds endlessly.

I'm still researching this, but I suspect it might have something to do with branch information; when (say) the DVPP repo is subscribed to the master branch of the staticSearch git repo, the master branch may not change, but the dev branch might; that means you end up with weird svn info like this (running svn info . in the external directory inside the Keats project):

URL: https://github.com/projectEndings/staticSearch.git/branches/master
Relative URL: ^/branches/master
Repository Root: https://github.com/projectEndings/staticSearch.git
Repository UUID: 04f9d699-67a6-817a-6227-ce57b7db7bff
Revision: 588
Node Kind: directory
Schedule: normal
Last Changed Author: martindholmes
Last Changed Rev: 570
Last Changed Date: 2020-01-13 11:41:56 -0800 (Mon, 13 Jan 2020)

You'll see that there are two revision numbers; the last-changed is 570, but the current revision is 588. Presumably that's because the master branch last changed in 570 but the dev branch changed more recently (as you'd expect). I have a suspicion that the svn polling mechanism in Jenkins interprets this to mean that there are new changes, and does a build. But it may be some other bug.

Anyway, I'm thinking that it might be a better alternative to pull static copies of the remote repo contents during the build process. The only disadvantage of this is that you can't live-edit the remote repo's code in the context of one of the projects using it, then commit from there back into the real repo. But that really just means that you have to have solid testing components in the original repo rather than relying on the consumer projects to do testing during active development.

First day working from home while dealing with basement flood.

2020-02-03:

Monday:
1 hour admin making sure student timesheets go to people who can sign them, informing HCMC staff about the situation, and rescheduling a workstudy from GRS.
1 hour setting up a new GitHub repo for the facsimile viewer code, separate from the DVPP repo where it started life, and then configuring it as an external for DVPP. Tested and working.
0.5 hours debugging Moses PDF build problem for SK (Moses).
0.25 hours renaming facsimile files for CH (LEMDO).
0.5 hours creating HCMC facsimile viewer.
Total 3.25 hours. Will debit 3.75 from my G&T.

Tuesday:
0.25 hours updating virtual servers.
0.5 hours regenerating stats for DVPP, building new TEI files for Macmillan 1860, and running OCR on those files.
0.25 hours relocating hcmcFacsViewer code in svn repo, and adding MoEML content to it.
0.5 hours debugging and fixing a problem with the DVPP build.
1 hour debugging and fixing an HTML validity problem in ColDesp.
0.75 hours working on rendering of born-digital site pages in ColDesp.
1.5 hours JTEI Issue 12 Editors' meeting (with Janelle and Kathryn Tomasek).
0.5 hours accessing mango through work machine to get sql dumps; running TEI generation and OCR on DVPP Macmillans 1870 and 1880.
Total 5.25 hours. Will debit 1.75 from my G&T.

Wednesday:
No chance to do anything. Will debit 7 hours from my G&T hours.

Thursday:
In the office from 7.35.
1 hour preparing for English 481.
1.5 hours teaching English 481.
0.5 hours working on Wendat documentation and Oxygen setup.
1 hour catching up with email.
0.5 hours ColDesp meeting/discussion with GL and KSS.
1 hour working on DVPP.
Total 5.5 hours. Will debit 1.5 from my G&T.

Friday:
0.5 hours updating DVPP stats, generating TEI for Hobby Horse 1890, OCRing Hobby Horse 1890, and sending email about rotated images which got lost yesterday.
0.5 hours reworking and testing the use of svn export instead of an svn:external setting for the Keats project, to solve the circular build problem on Jenkins.
0.5 hours porting that fix to DVPP, for use with facsViewer code and staticSearch code. Hopefully our circular builds are now solved.
1 hour adding support for thumbnail images to the facsViewer codebase; this is required for the Despatches static site.
1.5 hours starting work on implementing Despatches static image brower: mostly fixing existing build problems.
Total 4 hours. Will debit 3 from my G&T.