HCMC Blogs

Got the final corrections to BB's article done, and some tweaks to others; more stuff came in, and my TODOs have now been checked and re-organized. 60 minutes.

Wrote the proper abstract for my own DVPP paper and added it to the submission. Tweaked the collective DVPP abstract and submitted it to ConfTool; full version to be done tomorrow. 120 minutes.

Finished the next article on my plate ahead of tomorrow's meeting. 45 minutes.

A number of changes arising out of discussions yesterday and today:

Documentation and rendering for choice/sic/corr implemented and tested.

Link to XML on poem page rendering (needs prettying up).

Discussion on how to encode elision initiated on TEI-L.

Fixes to rendering to handle previously-unexpected scenarios in which lgs appear within notes and/or epigraphs, but should not be processed or counted as lines in the main poem.

360 minutes.

I now have a fully-operational system that uses ORCID to log a user in (so no passwords are stored in the DB) and restricts them to sections of the site that their role demands. For example, if a user logs in and the role they have is 'editor' they will not be able to see the 'publish' page or the contributor manager. There are some niceties to add (proper messages when ACL redirects user, etc.), but the functionality is all there.

I've also written a groovy user management system (groovy because it uses an editable table). Contributors can be added or edited, but for the time being, they cannot be deleted using this manager. Instead of deletion, the lowest level is 'de-activated'. They can still log in but all they will see is the regular information page about their CGWP account and any messages we provide.

When a contributor logs in and runs a search, they will now see a link in the quick-view that will open an editor page for that record.

Following up on this post, I now have the ORCID login/log out working reliably, but there's a caveat (below).

The javascript solution is to set mode to no-cors, accept json, and include credentials.

fetch('https://sandbox.orcid.org/userStatus.json?logUserOut=true',{
mode: 'no-cors', headers: {accept: 'application/json'}, credentials: 'include'})
.then(function(response){ orcidSessionStatus('logout');})
.catch(function (err) { console.log(err); });

I found that removing any of those configs will cause it to fail. If you watch the console you won't actually see any json returned (although there IS some). The mode: no-cors setting makes this an opaque request so the lack of response is expected. I follow-up with a function (orcidSessionStatus) that checks the logged-in status.

There is a caveat: in testing this solution I was unable to make it work in a regular browser session - that is, a regular window using my configured browser. It *only* worked in an unused/fresh browser. Specifically, if I launched Chromium with a temporary profile the ORCID log out worked. If I used my regular browser it did not.

I spent a lot of time testing stuff like removing extensions, clearing cookies, changing settings, etc. Nothing worked. I then reset my browser and re-installed my extensions, and that did it.

Conclusion: something in my browser setup broke my ability to make ORCID force-logout work. I have no idea what it was.

I'm at the point where I need to start thinking about how the nightly update/refresh is going to work.

Each night the DB needs to update the data tables with any vetted rows from the edit schema and refresh the materialized views.
A good SO post  suggests a few things:
1) cron - */30 * * * * psql -d your_database -c "REFRESH MATERIALIZED VIEW CONCURRENTLY my_mv"
2) trigger
3) listen/notify
Of those, only cron is practical as we will do maintenance on a schedule, not based on events.
Downside to cron is that it doesn't know if there have been problems.
As this will be a 2-step process (update data schema / refresh materialized views) our choice with cron is:
1) Ignore failures. If the failure is on the updates any refresh might be wasted.
But, a refresh doesn't take long (90 seconds-ish) so wasting cycles isn't a big deal.
If the failure is on the refresh how do we know? Would we need to do an audit to find discrepancies?
2) Write some code that runs before the view refresh that determines if errors occured during update and takes action (e.g. email and bail)
I also found pg_cron, which might be more elegant. Downside is that it needs to be installed

Note: if cron/psql is the answer, these psql flags will be useful:
-d database name
-c command
-b echo errors to standard or error output
-f filename (i.e. an sql file that contains the commands)
-L log file name
-q quiet

Both GL and I have hit the same problem on our workstations: the JNLP for the Despatches eXist instance will no longer run due to Java errors. We are still able to upload from Rumpole, but should some update bork that one too, we'll be stuck; we may have to create a VM with an old Ubuntu to get stuff into the old app. Upyerdb doesn't work either, because that eXist is too old for it. The static version can't come soon enough.

I've been working on generating JSON files in support of the search, mostly with the aim of supporting the search filters, as we do in other projects like Keats. The work is going well, but what I'm not really clear about yet is where to draw the line between convenience and file size; for example, I can include details of the periodical title and folder path with every poem retrieved under any category, or I could include only an id and have a small JSON file for periodical information which I use to look that info up when needed. Still thinking about all of this.

579 new page-images (in three different sizes) from the following volume have been added to the Colonial Despatches collection: RG7 G8C Vol 17 (British Columbia 1870 Despatches from London). These will now be linked into the transcription documents.