CGWP

I started getting warnings in FF: Some cookies are misusing the recommended “sameSite“ attribute

I tried various methods, which mitigated some of the problem, but I continued to get admonished about my cookies.
It looks like people on older versions of PHP are getting the warning and resorting to a hack. We're on the verge of upgrading so I didn't want to have any of that.

It looks like an .htaccess one-liner is solving the issue for me:
Header always edit Set-Cookie (.*) "$1; SameSite=Strict; Secure"

So it turns out that regular army types don't have attestation records/papers. Presumably because they didn't 'enlist'.
This means that users will require special instructions for people like Lt. Colonel Thomas Fraser Homer-Dixon

I have a functional editor, and I'm ready to test it. For the time being I'll do my own testing to make sure the egregious errors are fixed.
Next task is to start integrating the new editor design from PS.

There are nearly 200 distinct strings representing some kind of award/citation/honour in the DB right now. I pruned the list of official awards (like the Military Cross) and am left with 146 honours of some sort that I'd like to deal with.
I'm thinking that we could move them to a researcher's note, or create an array field in the data.person table called 'honours'. Either way, I'd also like to create an info page that explains what each one is and why it was generally awarded - they all sound impressive, but some were just for participating and others were basically rich dudes giving chums a bauble. In any case, there are some interesting stories in there.

Here's the list (spelling not corrected):
1911 King George V Coronation Medal
1914/15 Star
1914 Star
1937 Coronation medal
1939 - 1945 Star
Albert Medal
British War Medal
Bronze Medal for Military Valour (Italy)
Canadian Efficiency Decoration
Canadian Forces Decoration (CD)
Canadian Forces Decoration (CD) w/ 1st and 2nd Clasp
Canadian Volunteer Service Medal
Cavalier of the Order of the Crown of Italy
Chevalier de l'Ordre de Leopold (Belgium)
Chevalier de l'Ordre de Leopold II (Belgium)
Chevalier de l'Ordre Royal du Lion (Belgium)
Chevalier of the Legion of Honour
Colonial Auxiliary Forces Long Service Medal
Colonial Auxillary Forces Officer Decoration
Commander Military Order of Avis (Portugal)
Commandeur, Ordre de la Couronne (Belgium)
Companion of the Order of St Michael and St George
Companion of the Order of the Bath
Coronation Medal
Coronation Medal (1937)
Coronation Medal (1953)
Croix de Chevalier (France)
Croix de Guerre avec Etoile (France)
Croix de Guerre avec Palm (France)
Croix de Guerre avec Palms (Belgium)
Croix de Guerre (Belgium)
Croix de Guerre (Czecho-Slovakia)
Croix de Guerre (France)
Croix de Guerre with Bronze Star (Belgium)
Croix de Virtue Militara - 1st Class (Romania)
Cross Civique 1st Class (Belgium)
Cross of Military Merit 3rd Class (Bavaria)
Cross of St. George 3rd Class (Russia)
Cross of St. George, 4th Class (Russia)
Crown of Romania
Czechoslovak War Cross
Czechoslovak War Cross 1914-1918
Defence Medal
Defence Medal (WW2)
Duke of Jassy (Romania)
Efficiency Decoration
European Service Medal
France and Germany Star
Gold casualty stripe
Gold Stripe
Good Conduct Badge
Good Service Medal
Grand Cross of Order of Romania
Honourable Mention to Secretary of State for War
India Medal
King George Silver Jubilee Medal 1910-1935
Kings' South African Medal
Knight Commander of the Order of St. Michael and St. George
Knight Commander of the Order of the Bath
Knight Grand Cross of the Order of Carlos III of Spain
Knight Grand Cross of the Order of Christ of Portugal
Knight Grand Cross of the Order of Merit of Bulgaria
Knight Grand Cross of the Order of Pedro I of Brazil
Knight Grand Cross of the Order of the Rising Sun of Japan
Knight Grand Cross of the Order of the Rose of Brazil
Legion d'Honneur - Croix de Chevalier
Legion d'Honneur - Croix d'Officier
Legion of Merit (Degree of Commander)
Long Service in the Colonial Auxiliary Forces
Long Service Medal
Medaille de Reconnaissance (France)
Médaille des Épidémies
Medaille des Epidemies en Argent
Medaille des Epidemies en Bronze
Medaille des Epidemies en Vermeil
Medaille d'Honneur Avec Glaives
Medaille d'Honneur en Argent (France)
Medaille Militaire (France)
Medal of St. George, 3rd Class (Russia)
Medal of St. George, 4th Class (Russia)
Member of the Order of Canada
Member of the Royal Victorian Order
Memorial Cross
Memorial Plaque
Memorial Plaque & Scroll
Mercantile Marine Medal
Meritorious Medal
Meritorious Service Medal
Natal Medal
None
North West Canada Medal
Officier de la légion d'honneur & Croix de Guerre avec palme
Order of Canada
Order of Danilo, 2nd Class (Montenegro)
Order of Icarus
Order of Merit - Poland
Order of Regina Maria (Romania)
Order of St. Anne (Russia)
Order of St. Anne, Second Class with Swords (Russia)
Order of St. Anne, Third Class with Swords (Russia)
Order of St. George (Russia)
Order of St. Stanislas
Order of St. Stanislas, 1st Class (Russia)
Order of St. Stanislas, 2nd Class (Russia)
Order of St. Stanislas, 3rd Class with Swords (Russia)
Order of St. Stanislas (Poland)
Order of St. Vladimir (Russia)
Order of the British Empire
Order of the British Empire - Civil Division
Order of the Crown (Belgium)
Order of the Rising Sun, 4th Class (Japan)
Order of the Sacred Treasure, 4th Class (Japan)
Order of the Star of Romania
Order of the White Eagle with Swords, 5th Class (Russia)
Order of the White Eagle with Swords (Russia)
Ordre de Leopold (Belgium)
Ordre de l'Etoile Noire (Poland)
Polish War Cross
Queen's South Africa Medal
Royal Canadian Mounted Police Long Service Medal
Royal Engineers Transportation Branch No 105
Royal Humane Society Bronze Medal
Royal Humane Society’s Certificate
Royal Naval Reserve Officers Decoration
Royal Victorian Medal
Russian Medal of St. George
Saint-John of Jerusalem
Saviour of Romania
Service Medal
Service Medal of Canada
Silver Jubilee
Silver Medal for Military Valeur (Italy)
Territorial Efficiency Medal
Territorial Force Efficiency Medal
The Golden Aristion Andrias (Order of Gallantry)
The Médaille d'Honneur
Tibet Medal 1903-04
Victory Medal
Victory & War Medals
Vimy Pilgrimage
Volunteer Decoration
War Medal 1939 - 1945
War Medal (WW2)
Wound Stripe
Yser Cross (Belgium)
Yser Medal (Belgium)

I've spent a fair bit of time trying to rationalize a compact structure for multiple enlistments and I've come to the conclusion that any compactness that I may achieve comes at a processing cost that isn't worth it.
So, I've created a new 'enlistments' table that will have a one-to-many relationship with the person table - that is, every person can be associated with many enlistments.
Nearly 12,000 records have multiple dates-of-birth - either from mistakes or multiple enlistments. Either way I'll need to set something up that makes it easy to sort that out (some simpler version of the geocoder app that displays the two+ DOBs and lets the editor decide which one is right, or if it's a multiple-enlistment issue.

I'll also need to sort out the edits schema to figure out the best way to store edits (reproduce structure or use something like json).

I now have a fully-operational system that uses ORCID to log a user in (so no passwords are stored in the DB) and restricts them to sections of the site that their role demands. For example, if a user logs in and the role they have is 'editor' they will not be able to see the 'publish' page or the contributor manager. There are some niceties to add (proper messages when ACL redirects user, etc.), but the functionality is all there.

I've also written a groovy user management system (groovy because it uses an editable table). Contributors can be added or edited, but for the time being, they cannot be deleted using this manager. Instead of deletion, the lowest level is 'de-activated'. They can still log in but all they will see is the regular information page about their CGWP account and any messages we provide.

When a contributor logs in and runs a search, they will now see a link in the quick-view that will open an editor page for that record.

Following up on this post, I now have the ORCID login/log out working reliably, but there's a caveat (below).

The javascript solution is to set mode to no-cors, accept json, and include credentials.

fetch('https://sandbox.orcid.org/userStatus.json?logUserOut=true',{
mode: 'no-cors', headers: {accept: 'application/json'}, credentials: 'include'})
.then(function(response){ orcidSessionStatus('logout');})
.catch(function (err) { console.log(err); });

I found that removing any of those configs will cause it to fail. If you watch the console you won't actually see any json returned (although there IS some). The mode: no-cors setting makes this an opaque request so the lack of response is expected. I follow-up with a function (orcidSessionStatus) that checks the logged-in status.

There is a caveat: in testing this solution I was unable to make it work in a regular browser session - that is, a regular window using my configured browser. It *only* worked in an unused/fresh browser. Specifically, if I launched Chromium with a temporary profile the ORCID log out worked. If I used my regular browser it did not.

I spent a lot of time testing stuff like removing extensions, clearing cookies, changing settings, etc. Nothing worked. I then reset my browser and re-installed my extensions, and that did it.

Conclusion: something in my browser setup broke my ability to make ORCID force-logout work. I have no idea what it was.

I'm at the point where I need to start thinking about how the nightly update/refresh is going to work.

Each night the DB needs to update the data tables with any vetted rows from the edit schema and refresh the materialized views.
A good SO post  suggests a few things:
1) cron - */30 * * * * psql -d your_database -c "REFRESH MATERIALIZED VIEW CONCURRENTLY my_mv"
2) trigger
3) listen/notify
Of those, only cron is practical as we will do maintenance on a schedule, not based on events.
Downside to cron is that it doesn't know if there have been problems.
As this will be a 2-step process (update data schema / refresh materialized views) our choice with cron is:
1) Ignore failures. If the failure is on the updates any refresh might be wasted.
But, a refresh doesn't take long (90 seconds-ish) so wasting cycles isn't a big deal.
If the failure is on the refresh how do we know? Would we need to do an audit to find discrepancies?
2) Write some code that runs before the view refresh that determines if errors occured during update and takes action (e.g. email and bail)
I also found pg_cron, which might be more elegant. Downside is that it needs to be installed

Note: if cron/psql is the answer, these psql flags will be useful:
-d database name
-c command
-b echo errors to standard or error output
-f filename (i.e. an sql file that contains the commands)
-L log file name
-q quiet

I've spent a while working on a user-management strategy. Here's what I've come up with.

CGWP will not in any way manage or store passwords for accounts. We will retain only publically-accessible information and rely on ORCID to provide authentication. Users will require an ORCID account to become a contributor.

CGWP is a registered app at ORCID, and is allowed to use the API. Using ORCID examples I have a system that works like this:

1. users click a log-in link, which shuffles them off to ORCID to sign in.
2. Once signed in, ORCID returns them to a CGWP page that checks that they are a CGWP user and gets a responsibility parameter for their account. This is then used to allow access to, for e.g. the editor.

I am currently working on the last issue wherein ORCID's logout mechanism isn't working for me. I have a test here. Next is to finish off the editor.

While the massively-parallel approach appeared to be working on my desktop when run overnight, some processes did seem to have crashed near the beginning; on GN's machine the whole thing died within minutes. I've now rewritten it so that only four (configurable param) processes are run in parallel at the same time, to hopefully reduce the load on the machine. We're now testing that.