Category: Activity log

14/11/18

Permalink 05:00:50 pm, by mholmes, 41 words, 2 views   English (CA)
Categories: Activity log; Mins. worked: 60

Duplicate rows eliminated, and rows sorted

I've got a working process that detects and removes actual duplicate rows from the table. But there's still the question of why we have multiple rows for the same starting property. I'm finding it impossible to figure that out so far.

13/11/18

Permalink 03:54:38 pm, by mholmes, 40 words, 3 views   English (CA)
Categories: Activity log; Mins. worked: 240

Finished the fourth plan; more problems

Implementing the revisions from the latest discussions, I'm encountering situations in which the same property id has two or three rows. Only a few, but we can't figure out why they're happening. More to do, still in discussions with JSR.

09/11/18

Permalink 03:58:15 pm, by mholmes, 40 words, 5 views   English (CA)
Categories: Activity log; Mins. worked: 180

Half-way through fourth plan

The fourth plan necessitates rewriting all the code to retrieve T1 and T2, so I'm a good way into that, but more questions arose while doing that, which are still being worked out, so we're not done this week, unfortunately.

06/11/18

Permalink 04:02:05 pm, by mholmes, 183 words, 7 views   English (CA)
Categories: Activity log; Mins. worked: 60

Fourth plan finalized

This is it, to be implemented ASAP:

1. ensure that custodian sales are found NOT subject to the requirement to be non-nominal; then add a constructed variable: NOMINAL_CUSTODIAN_SALE=1.

2. After finding a T2, search for any qualifying transactions on descendant properties of the T1 which occur before 1942-03-04, and output a constructed variable with a count of these: T1_T2_INTERVENING=5. 

3. After finding T1 and a subsequent custodian sale T2, search for NON-NOMINAL sales to JCs on any descendant property of the original property between the T1 date and the custodian sale date. In such a case, abandon construction of the original row and instead construct a new row in which that non-nominal sale is the T1 (retaining the already-found custodian sale as T2).

4. After finding T1 and a subsequent custodian sale T2, and assuming action #3 does not cause the row to be rejected, search for NOMINAL sales to JCs on any descendant property of the original property between the T1 date and the custodian sale date. Output a constructed variable with a count of these (NOMINAL_LATE_SALES_TO_JC=x).


05/11/18

Permalink 04:43:19 pm, by mholmes, 340 words, 9 views   English (CA)
Categories: Activity log; Mins. worked: 90

Fourth plan revision and meeting

Discussions over the weekend led to a meeting today in which we thrashed out some of the issues, and we're half-way to a new plan; the immediate requirement was to investigate the cases where there is a sale to a JC buyer between the start date of the custodian activity and the actual sale of a property by the custodian. For the record, the quick-and-dirty XQuery to answer this question is below, along with the answers. It's clear these cases are not edge cases we can ignore; there's a definite pattern here.

let $baseDir := 'file:///home/mholmes/WorkData/history/stanger-ross/landscapes_of_injustice/svnrepo/trunk/xml/',
$startDate := xs:date('1942-03-04'),
$vanDbFile := concat($baseDir, 'landscapes_live_current_lotBased.xml'),
$mapDbFile := concat($baseDir, 'landscapes_mapridgelive_current_lotBased.xml'),
$dbs := (doc($vanDbFile), doc($mapDbFile)),

$custSales := distinct-values(($dbs//title[hasCustodianSeller != '0'][@isNominal='false'][not(matches(effectiveDate, '00'))]/@id))

for $t in $custSales
let $title := $dbs/descendant::title[@id = $t][1],
$tDate := xs:date($title/effectiveDate),
$lotIds := $title/lotsForTitle/lot/@id,
$targets := $dbs/descendant::title[not(matches(effectiveDate, '00'))][lotsForTitle/lot/@id = $lotIds][someOwnersJapanese != '0'][xs:date(effectiveDate) lt $tDate and xs:date(effectiveDate) gt $startDate]

return if (count($targets) gt 0) then concat('Title ', $t, ' has JC purchases in the key window: ', string-join(distinct-values($targets/@id),  ','), '
') else ()
 Title 1034 has JC purchases in the key window: 1111,1159,1160
 Title 1701 has JC purchases in the key window: 1805,1807
 Title 2926 has JC purchases in the key window: 2963
 Title 5323 has JC purchases in the key window: 5379,5423
 Title 5460 has JC purchases in the key window: 5502
 Title 3799 has JC purchases in the key window: 1172
 Title 4113 has JC purchases in the key window: 4192
 Title 4640 has JC purchases in the key window: 4708
 Title 5068 has JC purchases in the key window: 5122,5123
 Title 6453 has JC purchases in the key window: 6456
 Title 6605 has JC purchases in the key window: 6607
 Title 6649 has JC purchases in the key window: 6661
 Title 29141 has JC purchases in the key window: 29144
 Title 29329 has JC purchases in the key window: 29278

02/11/18

Permalink 03:38:16 pm, by mholmes, 34 words, 7 views   English (CA)
Categories: Activity log; Mins. worked: 180

New revision plan, many changes

Per plan worked out in email and on the phone with JSR, itemized as plan_3.txt, made a range of changes to the process of generating the spreadsheet. Some changes still being figured out.

01/11/18

Permalink 05:26:50 pm, by mholmes, 38 words, 8 views   English (CA)
Categories: Activity log; Mins. worked: 240

Revision plan for property stats spreadsheet and implementation

In emails with JSR, worked out the plan below, and implemented all the code parts; I'm now halfway through the data dictionary, which is proving helpful in clarifying our procedures for me as well as a potential reader.

22/10/18

Permalink 04:29:12 pm, by mholmes, 48 words, 8 views   English (CA)
Categories: Activity log; Mins. worked: 60

Next phase on property stats

Made a number of updates to generation of XML, GeoJSON and spreadsheet information to make for clearer output, and to build 2018 values into the whole process so any new products are consistent. Maps are now easier to read, and all include 2018 dollars. Lots more to be done, though.

12/10/18

Permalink 04:42:57 pm, by mholmes, 79 words, 11 views   English (CA)
Categories: Activity log; Mins. worked: 360

Spreadsheet, web interface and story maps now done

Today was long and complicated but it seems to be working. I'm now producing JSON files for each row in the spreadsheet table, and I have an HTML representation of the spreadsheet which provides links to the maps from those story files. That puts us in a position to examine lots of real cases and see whether there are any flaws in our retrieval and processing, as well as to get a sense of the actual stories we're uncovering.

11/10/18

Permalink 05:10:15 pm, by mholmes, 53 words, 12 views   English (CA)
Categories: Activity log; Mins. worked: 360

Spreadsheet now working

As far as I can tell, the spreadsheet is now doing what it's supposed to, after consultation with JSR and reworking. We will need only one spreadsheet rather than two. I'm now working on generating GeoJSON file for each row, so we can look at what we're generating and do human sanity checks.

:: Next Page >>

Landscapes of Injustice

Reports

XML Feeds