Times Colonist Transcript Database

filter by topic code, in particular second and subsequent code of records which contain more than one topic code

process data for 1896 and 1897 and upload into database - done Dec. 2007 / Jan 2007

I need to:
- renumber the remaining topics
- in the database do a number of search and replaces so that the values in the "topic" field for each record are correct
- put the text for each topic into the appropriate table.
Then ready to go back to code for query and reporting on this field.

Claire Carlin has taken over as contact person for Humanities site. Need to meet with her and set out plan for how to proceed. She is reviewing existing site and my prototype for new site.

Still to do:
improve text search (case-insensitivity for highlighting)
filter by topic code
process results to xhtml or tab delimited text for export
there are also anomalies in the data introduced by the proofers that will have to be removed, and John has a couple more years of data for the general index coming.

Get data from John. Get conventions on topic codes for John and normalize data with those. Process data into files suitable for upload to sql db. As of end of March, waiting for 3 of 4 datasets and information from John Lutz on subject codes etc. which I'll need before I can proceed any further. Written to him to get that information.

Create GUI to allow:

text search OR records with subject code OR all records

restrict date range for any of above queries

return results in xhtml table.

By mid-April, had basic and advanced text search, date range and filter by paper name or collection done.
Still to do: improve text search (case-insensitivity for highlighting) filter by topic code and process results to xhtml or tab delimited text for export.

Get db account from sys admins Create table for each dataset (or maybe one big table for all the datasets). Task completed last week of February.

Add field to TC main index to include name of paper.

Look at structure of data files for other datasets and write code to transform them to same data structure.

By end, should have one file for each dataset, ready to be imported into an sql table - one table for each dataset.

Got data from John early April and finalized procedures then.