Permalink 01:48:30 pm, by sarneil, 56 words, 23 views   English (CA)
Categories: Activity log; Mins. worked: 180

prepended public. to table names

Modified about three dozen files to explicitly set the postgres schema by prepending "public." mentions of table names in SELECT etc. queries. Addresses a possible security concern. Modest testing suggests no problems have been introduced, but if something pops up later, search for "public." will find the instances. No other modifications were made to the files.


Permalink 03:58:54 pm, by sarneil, 74 words, 427 views   English (CA)
Categories: Activity log; Mins. worked: 240

tidy up prod, dev, backup and postgres client

Before making the recent changes to the VIHistory site, I realized my dev, svnrepo backup and production instances were out of sync so I spent a couple of hours sorting all that out.

Also, due to recent security problems on some apache servers, the pgadmin is no longer accessible, so I installed an app pg3admin and configured it to ssh to the postgres server so I could do basic admin through that interface.

Permalink 03:53:51 pm, by sarneil, 85 words, 427 views   English (CA)
Categories: Activity log; Mins. worked: 120

Teamsters, coachmen sorted out

There are three records in the occupation table that kind of overlapped: 1 occupation_description = Coachman occupation_code = 54095 2 occupation_description = Coachman occupation_code = 98620 3 occupation_description = Teamster, Hackdriver, Drayman occupation_code = 98621 I changed #2 so that the occupation description is "Teamster" and removed "Teamster, " from #3. Now up to PD to ensure that the affected records in fact have the appropriate values. (I suspect only the relatively small number of records in #1 and #3 are candidates, as this whole task was initiated by records of #2 displaying "Coachman" instead of "Teamster".
Permalink 03:49:41 pm, by sarneil, 214 words, 456 views   English (CA)
Categories: Activity log; Mins. worked: 180

Year-sensitive subdistricts in popup window

In the searchcensus page, PD wanted the list of subdistricts that appeared in the popup list to be sensitive to which census year check boxes were selected. I modified the showDataListYear function in the script/showdatalist.js file to check the status of those checkboxes and then added an argument to the call to datalist.php to send that information to the popup page.

In datalist.php, I modified the code so that if there is a value in the yearsChecked argument, a newly written block of code is executed which ensures that only those subdistrict records which belong to the selected years appear in the popup window.

It is possible as it stands for the user to select e.g. 1901 checkbox, then pick a subdistrict from 1901, then deselect the 1901 checkbox and check the e.g. 1891 checkbox, at which point the configuration which may result in a query that returns no values (namely all records from 1891 that have a subdistrict name specified in 1901). The only exception to this is if there happens to be an 1891 subdistrict name which is identical to the 1901 subdistrict name. In that case, you'd get the records from 1891 that have the subdistrict name from 1891 that happens to be the same as the selected 1901 subdistrict name, but that's probably expected behaviour.


Permalink 11:22:38 am, by sarneil, 339 words, 438 views   English (CA)
Categories: Activity log; Mins. worked: 600

display year for each district in datalist viewer

Many of the districts/subdistricts appear in only certain censuses. (e.g. no Alberni in 1881, then one in 1891, 1901, 1911). Turns out each location is a record in the locations table that knows nothing about other location records (i.e. Alberni1891 has no relationship with Alberni1901).
In the search gui, there is a popup dialog that used to list all the distinct values of the subdistrict text (e.g. "Alberni"). Problem with that is that the user doesn't know whether a given district ("Alberni") exists in a given year (1881 vs 1891). I tried coming up with a smart way of manipulating the year checkboxes based on what was happening with the subdistrict values, but that was adding more UI problems then it was solving. So, I added the year to the subdistrict text for each value in the drop down (e.g. Alberni 1891, Alberni 1901) so at least the user can see which years that district exists in.
Next problem is that I can't use e.g. "Alberni 1891" as the value to search for in the SQL query, because of course that string doesn't appear in the subdistrict field of the locations table. So, I had to strip the year off of the value in the text box the user sees. The user must still check the checkbox for the year(s) they want to include in the search. Not ideal, but the best compromise I could come up with.
I modified the default settings to that all years are checked by default. So if the user selected "Alberni 1891" and hit submit, they'd actually get values for Alberni for 1891, 1901 and 1911. The hits show the date, so it would be pretty obvious they're getting results they don't want, and they'd then have to go back to the search form and uncheck the years they don't want. That seems better than the previous behaviour, in which the first year checkbox only was checked by default, so in our example, the user would get back 0 hits (as the first census year there was no Alberni district).

Permalink 11:10:11 am, by sarneil, 31 words, 441 views   English (CA)
Categories: Activity log; Mins. worked: 360

added Alberni data to census 1911

Finished processing the files from PD, then had to debug conflicting HISCO code numbers and a couple of other anomalies in the data. That data now part of the census_1911 table.


Permalink 09:02:06 am, by sarneil, 164 words, 577 views   English (CA)
Categories: Activity log; Mins. worked: 90

export database to spreadsheet failed

PD and ES of History wanted to get a dump of the 1901 Census. They tried using the Export feature on the site, but all they got was a white, empty window when they executed it. Looks like the number of records they were trying to export (about 46,000) exceed some RAM or processing time limit in php, so the job was never returned, resulting in the white window. I did a query to export all the males, then all the females. Those two exports worked. I then combined the two into one big spreadsheet file.

I then went into phppgadmin and discovered there were 5 records unaccounted for. Those had a value of 9 in the sex field. I then went back to the website GUI and searched for each of those five records and then incorporated them into the spreadsheet.

So, I did get the client what they asked for by a workaround, but didn't precisely identify the cause of the problem, let alone a solution.


Permalink 11:16:54 am, by sarneil, 39 words, 624 views   English (CA)
Categories: Activity log; Mins. worked: 60

minor updates to buildings stuff

Made final tweaks to content and organization of building stuff. Property section now has a building construction documents page with intro/background blurb and link to the search building construction documents data set. Made number of content changes too.


Permalink 01:35:50 pm, by sarneil, 76 words, 755 views   English (CA)
Categories: Activity log; Mins. worked: 120

further revision to building construction documents

At PD's request, I rejigged the properties chunk of the site. renamed the folder from ta to properties, took the one existing page (taxassessment.php) and split it into properties.php (landing page with links to tax assessment and building documents), taxassessment.php (intro and linke to 3 tax assessment tables), buildings.php (blurb on background and link to search page).
Also made minor tweaks to index.php page in root and the streetname_changes.php file.

Permalink 12:13:08 pm, by sarneil, 369 words, 1118 views   English (CA)
Categories: Activity log; Mins. worked: 60

rename folder caused svn problems

Seems like renaming a folder using svn rename is a bad idea. Probably better to create a new folder manually copy the files over then manually add all the new stuff to the svn repository. commit all that and assuming it is working, then delete the unwanted folder from the repo. I.e. manually do the copy and delete that rename supposedly does.

I wanted to rename a folder from "taxes" to "properties". The folder contained one file. I did this:
svn rename taxes properties
no problem. Then I committed the repository and got this:
svn: Commit failed (details follow):
svn: Item '/site_root/content/taxes' is out of date

2) So I tried updating the site and got back a taxes folder.
Then I cd'd to the containing folder and tried this:
svn delete taxes
worked. Tried to commit and got:
svn: Commit failed (details follow):
svn: Aborting commit: '/Users/sarneil/Documents/Projects/history/ViHistory/instances_of_site/svnrepo/trunk/site_root/content/ta' remains in conflict
then tried:
svn delete --force taxes
worked. Tried to commit and got:
svn: Commit failed (details follow):
svn: Aborting commit: '/Users/sarneil/Documents/Projects/history/ViHistory/instances_of_site/svnrepo/trunk/site_root/content/ta' remains in conflict

3) did this:
svn status
and got
D C taxes
> local delete, incoming edit upon update

4) Did a search on stackoverflow, and based on that, then tried
touch taxes
svn revert taxes
worked. I decided to try getting rid of the file inside first:
svn rm ta/taxassessment.php
worked. Then committed the change and now have things in sync with an empty taxes folder.

5) Then tried to delete the pesky folder
svn delete taxes
worked. Tried to commit that and got
svn: Commit failed (details follow):
svn: Item '/vihistory/trunk/site_root/content/ta' is out of date

6) Last resort on stackoverflow was "In a nutshell, the trick is to go to the .svn directory (in the directory that contains the offending file), and delete the "all-wcprops" file."
Did that, followed by
svn update
svn commit
and the taxes folder is now gone in the repository and in my copy.

No idea which of those steps (other than 6) are required.

:: Next Page >>


viHistory is a web site that is a teaching, learning and research tool. It's principally about the history of Vancouver Island in British Columbia, but it is also a vehicle for exploring the larger field of Canadian history during the late 19th and early part of the 20th century. It allows census, directory and tax assessment roll data from the late 19th and early 20th centuries to be searched in many ways. It also incorporates IMaP to display historical maps. The project director is Dr. Patrick A. Dunae.


XML Feeds