Memory issue with staticSearch well-formedness test
: Martin Holmes
Minutes: 25
With the addition of a few more poems yesterday, DVPP’s generated site
seems to have passed a size threshold which triggered a memory problem in
the part of the staticSearch build that checks the well-formedness of
the collection. The default value for an XML collection accessed by the
Ant <xmlvalidate> task seems to be 100,000K. Since
that task runs in the context of the Ant instance processing the build
file, it can’t be fixed inside the build file itself; instead, I’ve added
two new global variables in my local .bashrc file, which now
looks like this:
ANT_OPTS="-Xmx16G -Xss16m -Djdk.xml.maxGeneralEntitySizeLimit=0 -Djdk.xml.totalEntitySizeLimit=0"
export ANT_OPTS
The third and fourth values address this problem. You can then run this check at the command line:
ant -diagnostics | grep -i "xml"
Download it at https://xml.apache.org/commons/
XML Parser information
XML Parser : com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl
XML Parser Location: unknown
jdk.xml.maxGeneralEntitySizeLimit : 0
jdk.xml.totalEntitySizeLimit : 0
The values of 0
mean no limit
. Hopefully this will work;
I’ve made the same setting in the Jenkins config.xml file in the
Jenkins job, and that’s also stored in the DVPP repo. We may see more of this
in other projects, so I’m documenting it under various categories.