Maintenance

I've successfully configured my local machine for testing purposes so that eXists can be configured and run alongside each other, available individually on port 8080, and proxied through apache on port 80. The details of the eXist build and configuration are best discovered in my scripts in the repo, but the local configuration details are:

In /etc/hosts:

127.0.2.1	graves2-internal.hcmc.uvic.ca
127.0.3.1	mapoflondon6-internal.hcmc.uvic.ca
127.0.4.1	mariage6-internal.hcmc.uvic.ca

In /etc/apache2/sites-enabled/graves2.conf (one example for graves2, others analogous):

<VirtualHost 127.0.2.1:80>
	ServerAdmin webmaster@localhost
	ServerName graves2-internal.hcmc.uvic.ca
	ServerAlias graves2
	ProxyRequests Off
	<Proxy *>
		Order deny,allow
		Allow from all
	</Proxy>
	ProxyPreserveHost on
	ProxyPass / http://graves2-internal.hcmc.uvic.ca:8080/ nocanon
        ProxyPassReverse / http://graves2-internal.hcmc.uvic.ca:8080
        AllowEncodedSlashes NoDecode
</VirtualHost>

<VirtualHost [YOUR_REAL_IP_ADDRESS]:80>
	ServerAdmin webmaster@localhost
	ServerName graves2-internal.hcmc.uvic.ca
	ServerAlias graves2
	ProxyRequests Off
	<Proxy *>
		Order deny,allow
		Allow from all
	</Proxy>
	ProxyPreserveHost on
	ProxyPass / http://graves2-internal.hcmc.uvic.ca:8080/ nocanon
        ProxyPassReverse / http://graves2-internal.hcmc.uvic.ca:8080
        AllowEncodedSlashes NoDecode
</VirtualHost>

<VirtualHost 127.0.2.1:443>
        ServerAdmin webmaster@localhost
        ServerName graves2-internal.hcmc.uvic.ca
        ServerAlias test
        SSLEngine on
        SSLCertificateFile /etc/ssl/certs/spud.crt
        SSLCertificateKeyFile /etc/ssl/private/spud.key
        ProxyRequests Off
        <Proxy *>
                Order deny,allow
                Allow from all
        </Proxy>
        ProxyPreserveHost on
        ProxyPass / http://graves2-internal.hcmc.uvic.ca:8080/ nocanon
        ProxyPassReverse / http://graves2-internal.hcmc.uvic.ca:8080
        AllowEncodedSlashes NoDecode
</VirtualHost>

The one with an external ip address is optional, but needed if you want external hosts to be able to access the apps (based on their own hosts file being configured to point these domains at your ip). The third stanza allows HTTPS access; in fact Apache provides encryption, using the specified cert, but when it talks to eXist, it does that over an insecure local connection. If you try to have Apache from eXist's 8443 encrypted port for this, you'll get an error because eXist doesn't have a cert set up. You can still use the full :8443 port to access eXist directly over an encrypted connection. In both cases when using https, your browser will complain about a self-signed cert, of course.

With this approach, I've had Mariage, MoEML and Graves all running side-by-side on my machine. I've also finished and commented the script to roll out new eXists on Peach, and I'm waiting for RE to look it over before we actually test it. eXist 3.1 is out today and I built the latest dist from that tag.

This has been a relatively long process to figure out how best to configure a Jetty/eXist instance to run happily alongside others, on a test domain, and how to test that setup. This is what I've done:

• Install apache locally from the repos.
• Install mod_jk from the repos.
• Turn on SSL (sudo a2enmod ssl) and set up a self-signed cert (lots of docs on this available).
• Set up test domains in the local hosts file:

  127.0.0.1	localhost
  127.0.1.1	spud
  127.0.2.1	test-internal.hcmc.uvic.ca
  127.0.3.1	moeml-internal.hcmc.uvic.ca
  

• Set up virtual domains in Apache -- example sites-enabled/test.conf:

  <VirtualHost 127.0.2.1:80>
  	ServerAdmin webmaster@localhost
  	ServerName test-internal.hcmc.uvic.ca
  	ServerAlias test
  	ProxyRequests Off
  	<Proxy *>
  		Order deny,allow
  		Allow from all
  	</Proxy>
  	ProxyPreserveHost on
  	ProxyPass / http://test-internal.hcmc.uvic.ca:8080/ nocanon
          ProxyPassReverse / http://test-internal.hcmc.uvic.ca:8080
          AllowEncodedSlashes NoDecode
  </VirtualHost>
  
  <VirtualHost 127.0.2.1:443>
          ServerAdmin webmaster@localhost
          ServerName test-internal.hcmc.uvic.ca
          ServerAlias test
          SSLEngine on
          SSLCertificateFile /etc/ssl/certs/spud.crt
          SSLCertificateKeyFile /etc/ssl/private/spud.key
          ProxyRequests Off
          <Proxy *>
                  Order deny,allow
                  Allow from all
          </Proxy>
          ProxyPreserveHost on
          ProxyPass / http://test-internal.hcmc.uvic.ca:8080/ nocanon
          ProxyPassReverse / http://test-internal.hcmc.uvic.ca:8080
          AllowEncodedSlashes NoDecode
  </VirtualHost>
  

• In these four files in the Jetty instance:

  tools/jetty/etc/jetty-http.xml
  tools/jetty/etc/jetty-ssl.xml
  tools/jetty/etc/standalone-jetty-http.xml
  tools/jetty/etc/standalone-jetty-ssl.xml
  

  change the <Set name="host"> to <Set name="host">test-internal.hcmc.uvic.ca</Set>. (I think only the first two matter for our purposes, but it does no harm to change the others.)
• Start the Jetty instance, and restart apache. Access the jetty app on test-internal.hcmc.uvic.ca.

I still have to test this with a second Jetty running side-by-side on a different domain; I'll do that tomorrow.

After some experimentation and some help from DW on the eXist list, I've managed to figure out the following steps for making your single app (MoEML or whatever) available as the root thing on th Jetty server (i.e. localhost:8080 if you're running it locally), while maintaining access to the other bits and pieces if necessary (the Dashboard, eXide, Monex or whatever else you need. These are the steps:

• In tools/jetty/webapps/exist-webapp-context.xml, find the element <set name="contextPath"> and make sure it looks like this:

  <Set name="contextPath">/</Set>

  i.e. not /exist. It may be that way already, or it may be /exist.
• In tools/jetty/standalone-webapps/exist-webapp-context.xml (same filename, different place), make the same change if required.
• In tools/jetty/webapps/portal/WEB-INF/jetty-web.xml, find the same element and set it to the same thing (again, it may already be correct).
• In webapp/WEB-INF/controller-config.xml, find this:

  <root pattern=".*" path="/"/>

  Comment it out and replace it with two lines like this:

    <!--<root pattern=".*" path="/"/>-->
    <root pattern="/apps/moeml/apps" path="xmldb:exist:///db/apps/"/>
    <root pattern="/" path="xmldb:exist:///db/apps/moeml/"/>
  

  Where "moeml" is the name of your app. The second line makes the other apps (dashboard etc.) available using the "apps" subfolder of the root (which means you can't have anything in your app's controller that handles an "apps" subfolder).

Note that this was done with a fresh build of eXist 3.0 from the develop branch yesterday, with no use of the installer. We've learned to steer clear of the installer.

I've now built all this into a set of files, configure_exist.sh/xml/xsl. The first is a bash script which expects the path to the eXist instance, and (optionally) the name of the app to make the root. The second is an ant script which is run by bash using the ant which is inside the tools folder of the eXist instance; this parses out what it needs to know, and runs XSLT transforms using the Saxon version which is in the lib/endorsed folder. This means that you can run this deployment script even where neither ant nor Saxon are installed (such as Peach). It has a fragile dependency on the version numbers in filenames of ant and Saxon, but those are easy to update.

I've worked the CodeSharing feature into the MoEML XAR today, deciding on having the build process for the XAR just check it out from GitHub and put the bits in the right place, with the controller.xql handling the rest. So far so good; then I set up controller-config.xml so that the root path goes to the MoEML webapp, and the other apps we want are available from there. However, the "root" right now is /exist, not /, and I can't find a way to change that. After trying everything I could think of, I've written to the eXist list.

Created a script in svn/hcmc/exist which cleans the git tree completely, builds exist, signs the jars, creates a fairly minimal dist, and builds a test instance with one of our own xars, then starts it for testing. All that remains to be done is figure out how to change the jetty.xml file and any oother conf files to add the required placeholders.

Still on the quest of the best way to build a working eXist and update it cleanly, we've been testing the dist-zip build target. First, though, I determined how to build with DS's patch for the bug I reported the other day, which is still only a pull request (#1256); CLI for future reference:

git pull
git checkout 1256
git fetch origin pull/1256/head:1256
git checkout 1256

Then we do:

./build.sh clean
./build.sh 
./build.sh jnlp-all

the last of which signs the jars, enabling the Java client to work. This gives us a working build in the repo context, but of course it's cluttered up with lots of unwanted stuff, including source. That's why we wanted to use the dist-zip target, but the result is currently broken due to an obscure timestamping issue which I reported today on the gmane list. We could manually create a clean tree, of course, but it would be easier to have a script do it.

GL's desktop was rebuilt, and the huge TCCD git repo was copied over. However, it had been checked out using git://, and somehow it knew that the old desktop had SSH keys configured on GitHub, so we had to generate new keys (GL couldn't remember the pw for the old keys), add them to ssh-agent, and add the public key to GitHub (the key must be for the email address by which GitHub knows you). Then pulls and pushes would work. All this is quicker than checking out that repo again, of course.

Since it's not trivial to get this done, these are the steps:

• Ask NETS to add the domain to the UVic DNS servers, pointing at the root of hcmc.uvic.ca.
• Ask sysadmin to set up the apache virtual host on HCMC's web cluster.
• Test locally.
• Ask the person with the domain record to log into their registrar and set the DNS IPs to UVic's servers.

For the record, this one points at /home1t/thewali/www/

xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun

sudo xcode-select --reset

<?xml version="1.0" encoding="UTF-8"?>
<project basedir="." default="all" name="test">
    
<!-- This is a test file for figuring out how best to 
     integrate file diffing checks into ant build files. -->
    
<!-- We need ant-contrib to do foreach.   -->
    <taskdef resource="net/sf/antcontrib/antcontrib.properties"/>
    
    
<!--    Two filesets to diff. -->
    <fileset id="files1" dir="${basedir}/files1" includes="*.txt"/>
    
    <fileset id="files2" dir="${basedir}/files2" includes="*.txt"/>
    
    <target name="diffOneFile">
        <propertyregex property="otherFile" input="${inFile}" regexp="files1" replace="files2" global="true"/>
        <echo message="diffing ${inFile} against ${otherFile}"/>
        <trycatch>
            <try>
                <fail message="${inFile} is different from ${otherFile}.">    
                    <condition>
                        <not>
                            <filesmatch file1="${inFile}" file2="${otherFile}"/>
                        </not>
                    </condition>
                </fail>
            </try>
            <catch>
                <echo message="ERROR: DIFF FAILURE..."/>
                <exec dir="." executable="diff">
                    <arg line="-u"/>
                    <arg file="${inFile}"/>
                    <arg file="${otherFile}"/>
                </exec>
                <fail message="Build failed because of differences between  ${inFile} and ${otherFile}. See diff output above."/>
            </catch>
        </trycatch>
        
    </target>
    
    <target name="diffFileSets">
        <foreach target="diffOneFile" param="inFile">
            <path>
                <fileset refid="files1"/>                
            </path>
        </foreach>
    </target>
    
    <target name="all">
        <antcall target="diffFileSets"/>
    </target>
    
</project>