Category: Activity log

16/01/19

Permalink 03:50:17 pm, by mholmes, 1972 words, 6 views   English (CA)
Categories: Activity log; Mins. worked: 180

Deep dive into one Maple Ridge property

As instructed by JSR, did a deep dive into the property. Results:


Summary: the lot spreadsheet appears to be constructed (in this test case, at least) according to the rules that we set up. The full process for checking this appears in a short version and a long version below, but first, my take:

MY TAKE ON THIS:

There are two issues here. First, the Maple Ridge properties start to fragment very quickly indeed, giving rise to a huge number of descendant properties and therefore titles; the chances of getting a simple, clean sequence which aligns with the narrative you're chasing are probably pretty slim.

Second, we don't have a rule which says that when we are retrieving T3, it should NOT be a custodian sale IF the T2 is a custodian sale. I think it would make sense to have that rule, to avoid the situation we see in this case where T2, T3 and T4 are all custodian sales. Alternatively, we could adopt a rule which says that when there are multiple custodian sales to choose from at the T2 stage, we always take the latest rather than the earliest. Nevertheless, these measures may not solve your problem; do we need a similar rule that says that if T3 is a VLA sale, then T4 is not allowed to be one? What if T3 is not a VLA sale, but T4 turns out to be one?

Another option would be to adopt a rule which says that at every stage, the subsequent property in the sequence must be a descendant property not of the original ur-property but of the property or properties which were on the preceding retrieved title -- so T3 must be on descendant properties of the properties on T2, not on T1, and T4 must cover descendant properties of T3. This is more likely to result it what looks like a coherent sequence, but it is also much more likely to fail to retrieve a successful sequence at all, resulting in a much diminished number of spreadsheet rows, and therefore less statistically convincing results.

---------------------------------------------------------------------

DEEP DIVE INTO PROPERTY 21178

Base property: T:12 S:20 Q:NE 1/4 PL:3189 L:1 (p_21178)

---------------------------------------------------------
QUERY on overlaps.xml:

declare default element namespace "http://hcmc.uvic.ca/ns";

let $baseProp := '21178',
$overlaps := //overlap[@prop1=$baseProp or @prop2=$baseProp][xs:float(@percentProp1InProp2) ge 50],
$overlapIds := for $o in $overlaps return ($o/@prop1, $o/@prop2),
$distinctOverlapIds := distinct-values($overlapIds)
return string-join($distinctOverlapIds, ', ')
---------------------------------------------------------

Overlapping properties: 21373, 21298, 21297, 21622, 21623, 21616, 21617, 21618, 21621, 21619, 21620, 21574, 21575, 21416, 21682, 21681, 21374, 21614, 21376, 21375, 21615, 21330, 21329, 21255, 21252, 21257, 21256, 21254, 21253

Ids of titles which cover all of these properties:

---------------------------------------------------------
XQuery on landscapes_mapridgelive_current_titleBased.xml:

let $titles := //title[props/prop[@refid=('21373', '21178', '21298', '21297', '21622', '21623', '21616', '21617', '21618', '21621', '21619', '21620', '21574', '21575', '21416', '21682', '21681', '21374', '21614', '21376', '21375', '21615', '21330', '21329', '21255', '21252', '21257', '21256', '21254', '21253')]],

$titleDesc := for $t in $titles order by $t/titleInfo/ttl_date return concat($t/@id, ': ', $t/titleInfo/ttl_title_code, ' (', $t/titleInfo/ttl_date, ')')

return string-join((for $t in $titles order by $t/titleInfo/ttl_date return $t/@id), ', ')
---------------------------------------------------------

26684, 26959, 27763, 26688, 26675, 26723, 26732, 26733, 26725, 26730, 26788, 26796, 26791, 26792, 26799, 26807, 26811, 26809, 26812, 26814, 26816, 26762, 26763, 26883, 26927, 26854, 26884, 26885, 26929, 26974, 26984, 26857, 26849, 26987, 26871, 27564, 26945, 26956, 26944, 26955, 26958, 26994, 27581, 27582, 27929, 27014, 27015, 27016, 27571, 27017, 27020, 27654, 26949, 26970, 27930, 27649, 27936, 27021, 27667, 27668, 27712, 27714, 27673, 27729, 27728, 27730, 27731, 27734, 27733, 27677, 27940, 27727, 27759, 27679, 27766, 27710, 27769, 27724, 27708, 27761

Details of these titles:

XQuery on landscapes_mapridgelive_current_lotBased.xml:
---------------------------------------------------------
let $titleIds := ('26684', '26959', '27763', '26688', '26675', '26723', '26732', '26733', '26725', '26730', '26788', '26796', '26791', '26792', '26799', '26807', '26811', '26809', '26812', '26814', '26816', '26762', '26763', '26883', '26927', '26854', '26884', '26885', '26929', '26974', '26984', '26857', '26849', '26987', '26871', '27564', '26945', '26956', '26944', '26955', '26958', '26994', '27581', '27582', '27929', '27014', '27015', '27016', '27571', '27017', '27020', '27654', '26949', '26970', '27930', '27649', '27936', '27021', '27667', '27668', '27712', '27714', '27673', '27729', '27728', '27730', '27731', '27734', '27733', '27677', '27940', '27727', '27759', '27679', '27766', '27710', '27769', '27724', '27708', '27761'),

$titles := for $tId in $titleIds return //titles/title[@id = $tId][not(preceding::titles/title[@id=$tId])],

$titleInfos := for $t in $titles order by $t/effectiveDate return concat($t/@id, ': ', $t/code, '	',
$t/effectiveDate, ' ',
' nominal: ', if ($t/@isNominal = 'true') then 'yes ' else 'no  ',
' custSale: ', if ($t/hasCustodianSeller = '1') then 'yes ' else 'no  ',
' vlaBuy: ', if ($t/hasVlaOwner = '1') then 'yes ' else 'no  ',
' vlaSell: ', if ($t/hasVlaSeller = '1') then 'yes ' else 'no  '
)

return string-join($titleInfos, '
')
---------------------------------------------------------

26688: 32738E    1920-05-10  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
26675: 74159E    1927-10-16  nominal: no   custSale: no   vlaBuy: no vlaSell: no
26684: 106669E    1934-03-03  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
26723: 106822E    1934-03-21  nominal: no   custSale: no   vlaBuy: no vlaSell: no
26732: 107183E    1934-04-24  nominal: no   custSale: no   vlaBuy: no vlaSell: no
26733: 107254E    1934-05-02  nominal: no   custSale: no   vlaBuy: no vlaSell: no
26725: 107963E    1934-07-18  nominal: no   custSale: no   vlaBuy: no vlaSell: no
26730: 109668E    1934-12-06  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
26788: 113071E    1935-10-29  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
26796: 116972E    1936-09-18  nominal: no   custSale: no   vlaBuy: no vlaSell: no
26791: 132983E    1939-09-15  nominal: no   custSale: no   vlaBuy: no vlaSell: no
26792: 149801E    1942-04-02  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
26799: 166279E    1943-12-28  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
26807: 169926E    1944-04-17  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
26811: 169945E    1944-04-17  nominal: no   custSale: yes  vlaBuy: yes vlaSell: no
26809: 169933E    1944-04-18  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
26812: 169952E    1944-04-18  nominal: no   custSale: yes  vlaBuy: yes vlaSell: no
26814: 170808E    1944-04-25  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
26816: 170828E    1944-04-25  nominal: no   custSale: yes  vlaBuy: yes vlaSell: no
26762: 171885E    1944-05-18  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
26763: 171903E    1944-05-18  nominal: no   custSale: yes  vlaBuy: yes vlaSell: no
26883: 175591E    1944-09-02  nominal: no   custSale: no   vlaBuy: no vlaSell: no
26927: 281892E    1951-04-16  nominal: no   custSale: no   vlaBuy: no vlaSell: no
26854: 291940E    1952-01-11  nominal: no   custSale: no   vlaBuy: no vlaSell: yes
26884: 295379E    1952-04-16  nominal: no   custSale: no   vlaBuy: no vlaSell: yes
26885: 295380E    1952-04-16  nominal: no   custSale: no   vlaBuy: no vlaSell: no
26929: 313897E    1953-08-05  nominal: no   custSale: no   vlaBuy: yes vlaSell: no
26974: 322796E    1954-02-23  nominal: no   custSale: no   vlaBuy: no vlaSell: yes
26984: 359159E    1956-01-26  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
26857: 412345E    1958-07-28  nominal: yes  custSale: no   vlaBuy: yes vlaSell: no
26849: 423191E    1958-12-05  nominal: yes  custSale: no   vlaBuy: no vlaSell: yes
26987: 448583E    1959-12-14  nominal: no   custSale: no   vlaBuy: no vlaSell: no
26871: 450855E    1960-01-07  nominal: yes  custSale: no   vlaBuy: no vlaSell: yes
27564: 452250E    1960-01-18  nominal: no   custSale: no   vlaBuy: no vlaSell: no
26945: 467021E    1960-09-26  nominal: no   custSale: no   vlaBuy: no vlaSell: no
26956: 469702E    1960-11-17  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
26944: 468183E    1960-11-30  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
26955: 468184E    1960-11-30  nominal: no   custSale: no   vlaBuy: no vlaSell: no
26958: 484848E    1961-09-12  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
26994: 507210E    1962-11-20  nominal: no   custSale: no   vlaBuy: no vlaSell: no
26959: 533338E    1964-04-09  nominal: no   custSale: no   vlaBuy: no vlaSell: no
27581: 538458E    1964-07-13  nominal: no   custSale: no   vlaBuy: no vlaSell: no
27582: 538459E    1964-07-13  nominal: no   custSale: no   vlaBuy: no vlaSell: no
27929: 608026E    1967-08-14  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
27014: 627673E    1968-03-13  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
27015: 627674E    1968-03-13  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
27016: 627675E    1968-03-13  nominal: no   custSale: no   vlaBuy: no vlaSell: no
27571: 634784E    1968-06-18  nominal: no   custSale: no   vlaBuy: no vlaSell: no
27017: 634747E    1968-07-09  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
27020: 640865E    1968-09-25  nominal: no   custSale: no   vlaBuy: no vlaSell: no
27654: 656339E    1969-03-10  nominal: no   custSale: no   vlaBuy: no vlaSell: no
26949: 658596E    1969-04-09  nominal: yes  custSale: no   vlaBuy: no vlaSell: yes
26970: 658597E    1969-04-09  nominal: no   custSale: no   vlaBuy: no vlaSell: no
27930: 660594E    1969-05-02  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
27649: 675235E    1969-10-30  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
27936: 683572E    1970-01-27  nominal: no   custSale: no   vlaBuy: no vlaSell: no
27021: 728793E    1971-09-21  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
27667: G72055E    1971-10-29  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
27668: G112249E    1972-02-01  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
27712: H10857E    1972-02-24  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
27714: H10856E    1972-02-24  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
27673: H14697E    1972-02-28  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
27729: H48509E    1972-05-30  nominal: no   custSale: no   vlaBuy: no vlaSell: no
27728: H42064E    1972-06-07  nominal: no   custSale: no   vlaBuy: no vlaSell: no
27730: H58317E    1972-06-20  nominal: no   custSale: no   vlaBuy: no vlaSell: no
27731: H60992E    1972-07-10  nominal: no   custSale: no   vlaBuy: no vlaSell: no
27734: H65306E    1972-07-24  nominal: no   custSale: no   vlaBuy: no vlaSell: no
27733: H90242E    1972-09-13  nominal: no   custSale: no   vlaBuy: no vlaSell: no
27677: H92378E    1972-09-15  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
27940: H96413E    1972-09-27  nominal: no   custSale: no   vlaBuy: no vlaSell: no
27727: H101791E    1972-10-18  nominal: no   custSale: no   vlaBuy: no vlaSell: no
27759: H124001E    1972-12-06  nominal: no   custSale: no   vlaBuy: no vlaSell: no
27679: J5100E           1973-01-24  nominal: no   custSale: no   vlaBuy: no   vlaSell: no
27766: J55243E    1973-06-06  nominal: no   custSale: no   vlaBuy: no vlaSell: no
27710: J71976E    1973-07-04  nominal: no   custSale: no   vlaBuy: no vlaSell: no
27763: J86568E    1973-07-06  nominal: no   custSale: no   vlaBuy: no vlaSell: no
27769: K20906E    1974-03-14  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
27724: K68691E    1974-08-02  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
27708: K85245E    1974-08-20  nominal: yes  custSale: no   vlaBuy: no vlaSell: no
27761: L91605E    1975-11-07  nominal: no   custSale: no   vlaBuy: no vlaSell: no


To select T1 for this property:

Get the latest non-nominal property FOR THE ORIGINAL BASE PROPERTY before the starting date (1942-03-04):

26675: 74159E    1927-10-16  nominal: no   custSale: no   vlaBuy: no vlaSell: no

To select T2 for this property and its descendants we first look for any custodian seller
titles in the period at all, nominal or not:

26811: 169945E    1944-04-17  nominal: no   custSale: yes  vlaBuy: yes vlaSell: no
26812: 169952E    1944-04-18  nominal: no   custSale: yes  vlaBuy: yes vlaSell: no
26816: 170828E    1944-04-25  nominal: no   custSale: yes  vlaBuy: yes vlaSell: no
26763: 171903E    1944-05-18  nominal: no   custSale: yes  vlaBuy: yes vlaSell: no

Since we find some, we take the first in the list as our T2:

26811: 169945E    1944-04-17  nominal: no   custSale: yes  vlaBuy: yes vlaSell: no

To get T3, we now get the next qualifying title (i.e. the next qualifying title which follows
the custodian sale. That also happens to be a custodian sale, this time to the VLA:

26812: 169952E    1944-04-18  nominal: no   custSale: yes  vlaBuy: yes vlaSell: no

To get T4, we do the same thing again: get the next qualifying title, which turns out
to be another custodian sale:

26816: 170828E    1944-04-25  nominal: no   custSale: yes  vlaBuy: yes vlaSell: no

To get T5, we get the last qualifying transaction on a descendant property which happens before
the end date (1972-03-04):

27936: 683572E    1970-01-27  nominal: no   custSale: no   vlaBuy: no vlaSell: no

This sequence makes sense according to the rules and the spreadsheet. 

14/01/19

Permalink 05:16:51 pm, by mholmes, 97 words, 8 views   English (CA)
Categories: Activity log; Mins. worked: 300

Meeting, plan 8, and some of the work for it

Met with JSR and we looked at the stats work. There appears to be an anomaly around the average 2018 price per metre for Maple Ridge properties in the 42-49 date-range, so we planned a diagnostic for that; I've now written that, and it seems like the anomaly must be due to some specific transactions from the period which don't happen to make it into the lot-based spreadsheet. More checking to do there. Meanwhile, I updated the data dictionary to include the VLA spreadsheet, and the new plan also includes a significant addition to the main lot-based spreadsheet.

19/12/18

Permalink 05:11:30 pm, by mholmes, 27 words, 12 views   English (CA)
Categories: Activity log; Mins. worked: 60

Example encoding for Zotero lib

Working with GL and GN, created an example TEI encoding for the Zotero data. Needs an ODD file w/schema and documentation, as well as Oxygen project.

17/12/18

Permalink 04:31:49 pm, by mholmes, 60 words, 12 views   English (CA)
Categories: Activity log; Mins. worked: 120

Plan #7 title story maps

I've written the first pass at story maps for every title in the dbs. I think it's working, but we'll see what happens when Mr. Jenkins gets his hands on it. If it does work, I'll add an interactive query function to the map page so that you can type in a title or lot number to get its map.

11/12/18

Permalink 04:05:04 pm, by mholmes, 50 words, 12 views   English (CA)
Categories: Activity log; Mins. worked: 180

New spreadsheet nearly ready

I have the new VLA spreadsheet building, but it took a bit of reworking of the original transformations of db data, so I want to do a thorough check of the results. Also, I'm half-way through enhancing the web versions of these spreadsheets, and I'd like to get that done.

10/12/18

Permalink 05:02:58 pm, by mholmes, 94 words, 10 views   English (CA)
Categories: Activity log; Mins. worked: 240

Plan #7 progress

I've written, but not yet tested, functions for retrieving a list of V1 titles; getting a V0 title from a V1; and getting a V2 from a V1. I've also tweaked the way the plan GeoJSON is being generated so that the year is encoded in the filename for simplicity; and I've fixed the problem where the listings page for plans was not being generated. At least, I'm pretty sure I have; the results are still working through Jenkins. I've also raised a couple of queries for more detailed clarification on the spreadsheet plan.

Permalink 11:28:35 am, by mholmes, 244 words, 11 views   English (CA)
Categories: Activity log; Mins. worked: 60

Plan # 7

The Maple Ridge data is proving problematic in the way that the VLA forms part of the process, so to analyse this, a new spreadsheet has been proposed (plan below). I'll begin work on this today.

*** Seventh plan modification: Adding a new spreadsheet for Maple Ridge VLA-sold properties***

 - For each SALE BY THE VLA TO ANY PURCHASER, nominal or not:

 - Let that sale be V1.

 - Based on V1, retrieve a preceding sale TO the VLA on that or any 
   overlapping property. 
     - We assume that properties were only subdivided, not combined, 
       so any sale to the VLA on an overlapping property constitutes 
       the sale of that property or its direct parent. This sale may also 
       be nominal.
       
 - Let that sale be V0.
 
 - Now retrieve the next non-nominal sale on any descendant 
   property of the property-set constituting the V1 sale.
   
 - Let that sale be V2.
 
Now output a single row as follows:

Row id (generated number)
V0 data
V1 data
V3 data

Data for each sale is identical to the data in the current lot-based
spreadsheet, except that for each sale we add the following 
additional columns:

 - VX_DECLARED_VALUE
 - VX_DECLARED_VALUE_PER_SQ_M
 - VX_DECLARED_VALUE_PER_SQ_M_2018
 - VX_MARKET_VALUE
 - VX_MARKET_VALUE_PER_SQ_M
 - VX_MARKET_VALUE_PER_SQ_M_2018
 
These columns are empty unless the title has a declared value 
and/or a market value.

04/12/18

Permalink 04:11:08 pm, by mholmes, 34 words, 11 views   English (CA)
Categories: Activity log; Mins. worked: 60

Finished data dictionary; created downloadable package

Pace Mr. Jenkins, I have the downloadable package being created to support the chapter JSR is writing, and I've finished the data dictionary. I think this bit of the project should now be done.

29/11/18

Permalink 09:35:24 am, by mholmes, 81 words, 9 views   English (CA)
Categories: Activity log; Mins. worked: 260

Plan # six

Yesterday and today: JSR asked for more changes/additions to the original spreadsheet, and also for a second spreadsheet, organized by title. I've implemented the changes to the first spreadsheet, which is now renamed, and created the second. I found some anomalies in the data in the process, and fixed a couple of bad dates, while reporting some more suspicious numbers from the db. Also implemented an HTML version of the new spreadsheet linking to title maps, which will be handy.

19/11/18

Permalink 04:49:03 pm, by mholmes, 77 words, 13 views   English (CA)
Categories: Activity log; Mins. worked: 240

Next property stats spreadsheet done; could be that we're finished

This spreadsheet, which I started at the weekend, contains brief analyses of the numbers of properties which end up being included (directly or indirectly) in the main spreadsheet, compared with the total number of properties in the database, for each study area location.

At the same time, I diagnosed and fixed a bug in the generation of GeoJSON for maps, which would place two copies of the same property inside a single title category on the map.

:: Next Page >>

Landscapes of Injustice

Reports

XML Feeds