Copyright held by
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Further details of licences are available from our
Licences page. For more
information, contact the project director,
Provider: University of Victoria
Database: The Map of Early Modern London
Content: text/plain; charset="utf-8"
TY - ELEC
A1 - Holmes, Martin
ED - Jenstad, Janelle
T1 - Information for MoEML Programmers
T2 - The Map of Early Modern London
PY - 2020
DA - 2020/06/26
CY - Victoria
PB - University of Victoria
LA - English
UR - https://mapoflondon.uvic.ca/for_programmers.htm
UR - https://mapoflondon.uvic.ca/xml/standalone/for_programmers.xml
ER -
RT Web Page
SR Electronic(1)
A1 Holmes, Martin
A6 Jenstad, Janelle
T1 Information for MoEML Programmers
T2 The Map of Early Modern London
WP 2020
FD 2020/06/26
RD 2020/06/26
PP Victoria
PB University of Victoria
LA English
OL English
LK https://mapoflondon.uvic.ca/for_programmers.htm
Programmer, 2018-present. Junior Programmer, 2015-2017. Research Assistant, 2014-2017. Joey Takeda was a graduate student at the University of British Columbia in the Department of English (Science and Technology research stream). He completed his BA honours in English (with a minor in Women’s Studies) at the University of Victoria in 2016. His primary research interests included diasporic and indigenous Canadian and American literature, critical theory, cultural studies, and the digital humanities.
Data Manager, 2015-2016. Research Assistant, 2013-2015. Tye completed his undergraduate honours degree in English at the University of Victoria in 2015.
Research Assistant, 2012–2013. Cameron Butt completed his undergraduate honours degree in English at the University of Victoria in 2013. He minored in French and has a keen interest in Shakespeare, film, media studies, popular culture, and the geohumanities.
Research Assistant, 2012-2014.
Director of Pedagogy and Outreach, 2015–present. Associate Project Director, 2015–present. Assistant Project Director, 2013-2014. MoEML Research Fellow, 2013. Kim McLean-Fiander comes to
Janelle Jenstad is Associate Professor of English at the University of Victoria, Director of
Programmer at the University of Victoria Humanities Computing and Media Centre (HCMC). Martin ported the MOL project from its original PHP incarnation to a pure eXist database implementation in the fall of 2011. Since then, he has been lead programmer on the project and has also been responsible for maintaining the project schemas. He was a co-applicant on MoEML’s 2012 SSHRC Insight Grant.
Our editorial and encoding practices are documented in detail in the Praxis section of our website.
This documentation provides basic information for programmers needing to work on the MoEML infrastructure and build process. It covers programming languages used, code organization, software requirements, the static build process, and tips and tricks for working more efficiently on the code.
The
As mentioned above, the organization of the software repository is somewhat confusing for two reasons: first, the repo size is large, and we don’t want to force regular encoders to download the whole thing. This means the data component of the repo needs to be self-sufficient in some ways, so programming code and resources are included there which would be better placed elsewhere in a perfect world. Secondly, the project has gone through many phases in which different components were used (Cocoon, eXist, even PHP) so there are remnants of stuff which really should be reorganized.
These are the important areas for programmers:
backup_schemas
contains copies of schemas
which the build process would normally download from the web,
to ensure we have the latest versions. When the download fails,
these are used in order to allow the build to proceed.
db
is confusingly named, because it
was once part of an eXist XML db folder structure. It
contains the following comonents:
agas
contains the image files used
in the Agas Map page.data
contains all the TEI XML and
related files (images, binary documents) which form
the intellectual content of the project. Because this
is where encoders work, this folder also includes the
project schemas (in rng
, although the schema
constraints are created in ODD and Schematron, and then
build into RNG). This folder also includes a
utilities
folder with some important
build components, in particular the diagnostics code and
the schema build code.redirects
currently contains only one
file, an XML file called redirects.xml
where
we specify how to handle ids (which are tied to URLs) which
need to be retired. We specify where each retired id should
be redirected to, so that pages do not simply disappear
from the site when new versions are released.site
contains site components and
resources that are edited by designers and programmers,
used in building the website.ise
contains some old versions of Internet Shakespeare
Edition plays which were part of an experiment to link between the
two projects. The long-term status of this experiment is undecided;
ignore this folder for the moment.jenkins
contains two components related to the build
process for the project which runs on our Jenkins CI server. config.xml
is the configuration file for that build; this should be updated whenever
a change is made to the build configuration on the server. The other file,
moeml_log_parse_rules.txt
, is a set of rules which is used
by Jenkins to determine whether a build has failed or succeeded. In the
course of a normal build process, words such as obsolete
is what you would expect: a place where we
stash data and code files which are no longer needed.presentations
contains the materials for presentations
made by project members that relate directly to static
is the folder which contains all of the
code used to build the current version of the site.
css
has all the various CSS files used in the current XHTML5
version of the site.exist
contains code related to the version of the site
which was hosted in the eXist XML database; this was used for version
6.3, but from version 6.4 onwards we have moved to a completely static
version which does not require a backend database, so this code will
eventually be moved to the obsolete
folder.externals
is a folder which is configured to bring
in some XQuery code from other repositories, used in the eXist version
of the site. This will eventually be removed.fonts
contains all the web fonts used in the current
version of the site, and in the PDF versions of the Mayoral Shows.fopConfig
contains a configuration file for the
FOP PDF processor
which is used for generating the PDF versions of Mayoral Shows.js
contains a variety of JavaScript libraries used
in the static website.ssExtras
contains two files which are used as part
of the staticSearch component of the site build (described at length
below). xsl
contains all the XSLT code used in our current
build processes, to create the website and the Mayoral Show PDFs. This
will be described in detail below.static
which are not part of the svn repository; examples
are site
and staticSearch
. These are created
during the build process.
utilities
contains a range of libraries and code modules
some of which are essential for everything (e.g. the Saxon XSLT processor) and
some of which are one-off transformations used to fix problems. Many of these
files are obsolete and a cleanup of this folder is long overdue.workshops
contains materials used for teaching workshops
for RAs on specific topics such as regular expressions and XPath.This is a list of software that is required for running the various build processes. Some of it is actually stored in the repository, and some must be installed on the machine doing the build.
The following software is stored in the SVN repository, so does not need to be installed locally:
To run the various MoEML build processes, you will need the following software to be installed on your machine. At present most of the build processes have to be run on *NIX systems because they depend on command-line utilities. If you are forced to use Windows, you’ll probably have to install the Windows Subsystem for Linux. For running specific components of the build, you may not need all of these applications or libs.
The project has two distinct build processes:
build.xml
in the project root folder)build.xml
in the
static
folder)
The extended validation build is designed to provide a range
of extra checks to be carried out before bothering to build the
website. It is controlled by the Ant build.xml
file in the project root folder. It checks that:
It also runs the project diagnostics, to find problems which
are not build-breaking but will require attention.
If all checks in the Extended Validation Build have completed
successfully, then Jenkins will run the static site build. This
is controlled by the Ant build.xml
file in the
static
subfolder.
This is a long and complex process, and it takes a long time to complete. Programmers working on the project need to understand it well so that they can run subcomponents of the build process in order to reproduce build errors rapidly and fix them efficiently.