Meeting with JS-R: planning for infrastructure
Posted by mholmes on 07 Apr 2014 in Activity log
Met with JS-R and SA to discuss the hardware and software infrastructure to support each of the research clusters. Subsequent discussion with GN. Points arising:
- We will create a VM for the Landscapes project, running MariaDB, PHP, and ultimately Tomcat 8 with eXist. It will mount around 4TB of disc space.
- The Adaptive DB code will be updated to handle binary doc uploads.
- We will arrange with tech-savvy nodes for them to replicate our data probably through rsync cron jobs.
- Sysadmin will administer the VM for the usual cost.
- We will consult with individual research clusters asap to get a basic idea of their needs and skill levels with regard to data collection and entry.
- We will put together a presentation for the meeting in May (20th-22nd or thereabouts) detailing our plans so far, and all the possible options, so they can all see their own data collection and storage in the context of what others have asked for or may need. This will allow us to revisit preliminary decisions.
- Basic data collection systems will be in place for testing by around the middle of June, and data collection should start by the beginning of July.
- The land title work headed by JS-R will use a simplified version of the existing Properties db for data collection, but querying will be done on a generated XML version which is optimized for the sorts of queries we want to do, and which will be based around the property rather than the title; this will have massive data duplication to speed queries, so that we can avoid lookups and join-type operations in the interests of speed.