FrancoToile

The website is located at: http://francotoile.uvic.ca

... which is a virtual host that points to: http://pear.hcmc.uvic.ca:8081/francotoile/francotoile/index.xql

the FrancoToile app lives in the webapps-dev directory on Pear. All of the non-essential web interfaces (the admin interface, eXide, the help docs, etc.) have been erased from the app to limit the possibility of unauthorized access.

To connect to the database: see the ODT file for connection information.

The eXist version of the site uses much of the same HTML/CSS from the PHP version, as done by Pat Szpak.

State of the Website

The site is complete aside from the odd tweak and language-related fix that Catherine sends in. All of the features work, the code is documented, and everything is running smoothly.

The Bookmarking System
(taken from: http://hcmc.uvic.ca/blogs/index.php?blog=12&p=8271&more=1&c=1&tb=1&pb=1 )

Adding a Bookmark

When a user adds a bookmark, an AJAX request with the video ID, the timestamp, and any optional comments is sent to plugins/bookmarks/ajax/add.xql. This XQuery script returns a JSON object with the ID (i.e. bookmark/@n) of the new bookmark, the timestamp, and the comments if applicable. The add.xql script actually returns XML that is passed through the eXist JSON serializer:

declare option exist:serialize "method=json media-type=text/javascript";  

If the bookmark can't be added, which happens if the video ID is invalid, then a 500 error is returned.

Upon successful addition, the JSON response object is parsed by the calling Javascript, which adds a list element to the ordered list of bookmarks.

Deleting a Bookmark

Deleting a bookmark (clicking on the 'X' image next to the bookmark in the ordered list) simply removes the bookmark from the DOM. Bookmark data files are never actually deleted from the server.

Saving Bookmarks

Saving a bookmark list requires no AJAX interaction with the server. Rather, all of the bookmark IDs are gathered from the ordered list of bookmarks (each of which has an ID of bookmark-## where ## corresponds to bookmark/@n in the XML data) and then simply appended to the URL of the video page. This URL is given to the user, who can add it to their bookmark collection for later use. The ids are added as comma-separated values of a "bm" GET parameter. For example: player.xql?id=fraq1&bm=54,55,56

Loading Saved Bookmarks

When a user visits a video with a bm query string (see 'Saving Bookmarks' above), an AJAX call is sent to plugins/bookmarks/ajax/load.xql for each bookmark ID in the query string. Upon successful load of the bookmark - success means that the ID corresponds to bookmark/@n and the current video corresponds to bookmark/@videoId - a JSON-serialized XML node is returned (see 'Adding a Bookmark' above), which is then used to populate the unordered list of bookmarks.

Future Data

As stated on the blog ( http://hcmc.uvic.ca/blogs/index.php?blog=12&p=8288&more=1&c=1&tb=1&pb=1 ), the XML data files on Pomme have been moved to an inaccessible location, in the interests of maintaining a single set of up-to-date files (i.e. the ones on Pear). When Catherine is ready to start working on the site again, Pomme should be updated with the latest version of the files from the Pear eXist DB.

The movies at http://francotoile2.uvic.ca used as a base URL: http://francotoile.uvic.ca/media/

When the eXist site was switched to francotoile.uvic.ca, however, those paths became invalid. So, I updated the movie player to use the 'real' base URL on lettuce: http://lettuce.tapor.uvic.ca/~florevid/media/

This is a decent temporary solution, but I will experiment with putting the media files on the Pear file system and then using an XQuery script to stream the videos, rather than using lettuce.

***per Jamie's idea, how about a mod_proxy def. like:
http://hcmc.uvic.ca/$projName that resolves to a /home1t/$projName/media/ folder
so projects serving media can use a URL like:
http://hcmc.uvic.ca/florevid/media/somevideo.mp4

After digging in our heels and grunting through a bunch of Apache rewrite rule combinations, Greg and I came up with a VirtualHost entry for francotoile2.uvic.ca that does the job, in that http://francotoile2.uvic.ca points to http://pear.hcmc.uvic.ca:8081/francotoile/francotoile/index.xql . Getting it to work basically just came down to trying a lot of different combinations of various RewriteRule and JKMount lines until something worked. We gave the VirtualHost stuff to ER at systems who implemented it (and expanded it a bit). Here's the working entry:

<VirtualHost *:80>
  ServerName francotoile2.uvic.ca
  DocumentRoot /home1t/florevid/www
  RewriteEngine On
  RewriteRule  ^/$  /index.xql
  RewriteRule ^(.*)$ /francotoile/francotoile$1 [PT]
  JkMount /francotoile/francotoile/ dev
  JkMount /francotoile/francotoile/* dev
  ErrorLog "|/usr/local/sbin/cronolog
/usr/local/apache2/logs/francotoile.uvic.ca-error_log.%Y%m%d.%X"
  CustomLog "|/usr/local/sbin/cronolog
/usr/local/apache2/logs/francotoile.uvic.ca-access_log.%Y%m%d.%X" combined
</VirtualHost> 

Now we're just waiting on ER to point the Pear site to http://francotoile.uvic.ca instead of francotoile2, which'll wrap up the new version of the site.

Due to a recent change in Google's map JS API (who knows what that change is), I've had to change the map page to be served as text/html rather than application/xhtml+xml. Here's the comment that I put in the HTML source on the eXist version of the site:

    July 05/2011, Jamie Nay:
    This page is served as text/html rather than application/xhtml+xml because the Google Maps JS API uses document.write to write elements to the DOM, which isn't supported by XHTML.
    Google knows about this issue ( see http://code.google.com/p/gmaps-api-issues/issues/detail?id=530 and http://code.google.com/p/gmaps-api-issues/issues/detail?id=2820 ) but won't fix it. Instead they suggest a
    workaround (better known as a hack): passing a callback function when loading the API script.
    
    Unfortunately, neither this:
    
    <script type="text/javascript" src="http://maps.google.com/maps/api/js?sensor=false&callback=initialize"></script>
    
    Nor this:
    
    <script type="text/javascript" src="http://www.google.com/jsapi"></script> 
    <script type="text/javascript"> 
    google.load('maps', '3', {'other_params': 'sensor=false', 'callback': initialize});
    </script>
    
    Seems to work; I've tried both approaches (loading the API directly and using google.load). The bottom line seems to be that, no matter what workaround you try, you just can't load the API properly while serving your
    content as application/xhtml+xml.
    
    Prior to early July/late June 2011, the API was being loaded successfully because I had put in a document.write clone script that I found at: http://weston.ruter.net/projects/xhtml-document-write/
    
    That script added document.write support to pages served as application/xhtml+xml. However, due to what I can only assume is a recent change in the Google API, this fix no longer works and the map does not display correctly under any circumstances when serving with the xhtml mime type. So, the only feasible solution in terms of time and effort spent is to serve this map page as text/html. If there's a pressing need to serve as application/xhtml+xml in the future then this issue needs to be resolved. For now, however, text/html is doing the job.

eXist comes with a "debugger" extension (or "debuggee", depending on which area of the code you look in), which is disabled by default. To enable XDebug in an eXist application:

1. Build and deploy eXist: http://hcmc.uvic.ca/blogs/index.php?blog=11&p=8146&more=1&c=1&tb=1&pb=1

   • Important:When editing local.build.properties, set include.feature.debugger to true

2. Copy:

   (exist source directory)/extensions/debuggee/lib/mina-core-x.x.x.jar

   To:

   (exist deploy directory)/WEB-INF/lib

Part two will cover actually using the debugger to step into a running script, once I figure out how to do that.

Updated the new eXist-based version of the site with a whole bunch of changes and improvements. The site is feature-complete now. I've emailed CC to ask her to look at the site to make sure it all looks good to her. Once she's happy then it will move to http://francotoile.uvic.ca.

Updated the eXist version of the site to make the namespaces for the plugins (bookmarks, i18n, includes) more generic - i.e. without "francotoile" in the ULR - to make them more portable. So, a namespace that looked like this:

http://hcmc.uvic.ca/namespaces/francotoile/i18n

Now looks like this:

http://hcmc.uvic.ca/namespaces/exist/i18n

If you load an element on your page after page load (i.e. via AJAX), basic JQuery event listeners such as this:

$(".my-selector").click(function(event) {
...
});

won't work, because that event listener only listens to elements that appear upon completion of page load. To catch elements introduced later via AJAX, use the live handler:

$(".my-selector").live('click', function(event) {
...
});

That will attach the 'click' event handler to all selected elements, even ones inserted into the DOM after page load. See the JQuery docs for more info: http://api.jquery.com/live/

If you want to pass Unicode characters to a URL in your eXist application, use fn:encode-for-uri ( http://demo.exist-db.org/exist/functions/fn/encode-for-uri ), which is roughly equivalent to PHP's rawurlencode() (a stricter version of urlencode()).

Made a couple of minor changes to the TEI schema for the XML data files.

First, residence:

• Previously: an element containing "text of subjects's present or past place of residence"

• Now: an element with up to three child elements:

  • settlement: contains the name of a settlement such as a city, town, or village identified as a single geo-political or administrative unit.
  • region: contains the name of an administrative unit such as a state, province, or county, larger than a settlement, but smaller than a country.
  • country: contains the name of a geo-political unit, such as a nation, country, colony, or commonwealth, larger than or administratively superior to a region and smaller than a bloc.

nationality, which previously was an element with a string containing "text of subjects's nationality", will use the same structure as residence, for the same reasons.

I made these change for two reasons: to allow for greater organizational structure and separation of data, and to facilitate populating the "location" and "origin" select lists on the search page. The previous formats for residence and nationality, plain text string values (such as "Victoria, Canada"), were limited in their searchability and organization.

Wrote the bookmarking system for the eXist site, which is basically as described in my previous post. Here's a more technical breakdown of the features:

Adding a Bookmark

When a user adds a bookmark, an AJAX request with the video ID, the timestamp, and any optional comments is sent to plugins/bookmarks/ajax/add.xql. This XQuery script returns a JSON object with the ID (i.e. bookmark/@n) of the new bookmark, the timestamp, and the comments if applicable. The add.xql script actually returns XML that is passed through the eXist JSON serializer:

declare option exist:serialize "method=json media-type=text/javascript";

If the bookmark can't be added, which happens if the video ID is invalid, then a 500 error is returned.

Upon successful addition, the JSON response object is parsed by the calling Javascript, which adds a list element to the ordered list of bookmarks.

Deleting a Bookmark

Deleting a bookmark (clicking on the 'X' image next to the bookmark in the ordered list) simply removes the bookmark from the DOM. Bookmark data files are never actually deleted from the server.

Saving Bookmarks

Saving a bookmark list requires no AJAX interaction with the server. Rather, all of the bookmark IDs are gathered from the ordered list of bookmarks (each of which has an ID of bookmark-## where ## corresponds to bookmark/@n in the XML data) and then simply appended to the URL of the video page. This URL is given to the user, who can add it to their bookmark collection for later use. The ids are added as comma-separated values of a "bm" GET parameter. For example: player.xql?id=fraq1&bm=54,55,56

Loading Saved Bookmarks

When a user visits a video with a bm query string (see 'Saving Bookmarks' above), an AJAX call is sent to plugins/bookmarks/ajax/load.xql for each bookmark ID in the query string. Upon successful load of the bookmark - success means that the ID corresponds to bookmark/@n and the current video corresponds to bookmark/@videoId - a JSON-serialized XML node is returned (see 'Adding a Bookmark' above), which is then used to populate the unordered list of bookmarks.

Crib sheet with some useful reminders: http://en.wikibooks.org/wiki/XQuery/eXist_Crib_sheet

It's popped up a few times during my Google searches.

GN, MH and I spent some time yesterday developing a model for the bookmarking system that will be in the new, eXist version of the FrancoToile site. After throwing around a few ideas, we've settled on this:

A "bookmark" is a reference to a single timestamp in a video. In addition to containing the video ID and the timestamp, a bookmark can also contain an optional comment. Each bookmark is stored in a separate XML document in the eXist database. A proposed sample looks like this:

<bookmark id="b3" videoId="fraq1" timestamp="00:05:35.55">
    <comment>Note the usage of 'dans' in this context.</comment>
</bookmark>

Notes on the bookmark attributes:

• @id is a unique, sequential ID, prefixed by "b", generated upon bookmark creation.
• @videoId is the ID of the video file.
• @timestamp is used rather than an utterance ID, so that a user can bookmark an exact moment in the video rather than a general utterance.

A bookmark can never be edited or deleted via the public website, only by someone with access to the database.

When a user adds a bookmark to a video, an AJAX call is made that stores the new document and, upon success, returns the new unique ID of that bookmark. The unique ID is then appended to the query string in the user's URL. This query string method is how bookmark "collections" are created: they're simply a series of bookmark IDs in the URL.

When a user "deletes" a bookmark, they are not, in fact, deleting it, just removing the ID from their collection URL. Similarly, when a user changes the order of the bookmarks, this simply changes the order of the IDs in the URL.

Since bookmarks can't be edited, if a user wants to edit a bookmark - for example, adding or changing a comment - a new bookmark is created to reflect these changes and added to the user's URL. The original bookmark is then removed from the URL.

So, every action related to manipulating bookmarks results in a change to the URL the user used to access the video. When a user clicks on the "save my bookmarks" link, they are simply given the URL, which they can then share with others.

Remember the comma operator when returning a sequence of multiple elements in an XQuery return statement. If you want to return more than one element, then you need to wrap the whole thing in parentheses and separate each element with a comma. Compare the first "if" return with the second "else" return:

<div id="relatedList">
            {
            let $relatedCount := count($relatedNationality)
            return
                if ($relatedCount < 1) then
                <h2>There are no other people from {$nationality}.</h2>
                else
                (<h2>Other people from {$nationality}.</h2>,
                <ul>
                {
                for $related in $relatedNationality
                    let $relatedId := $related/@xml:id
                    let $relatedTitle := $related//tei:title
                    return
                        <li>
                            <a href="player.xql?id={$relatedId}">
                                <img src="images/thumbs/{$relatedId}.png" alt="{$relatedTitle}"/>
                                <span class="relatedListItemLine">{$relatedTitle}</span>
                            </a>
                        </li>
                }
                </ul>)
            }         
        </div>

Be careful using the "//" axis to iterate over a set of elements in a document fragment, since it doesn't work as you may expect. Take this XML, which I'll call $summary:

<p>
<span class="previous">... sens français, </span>
<span class="hi">francophone</span>
<span class="following"> également, mais quelquefois je me s ...</span>
</p>
<p>
<span class="previous">... ment, mais quelquefois je me sens plus </span>
<span class="hi">francophone</span>
<span class="following"> que français ...</span>
</p>
<p>
<span class="previous">... je ... je crois que notre culture </span>
<span class="hi">francophone</span>
<span class="following"> hein .. ...</span>
</p>

To loop through the "p" elements, you cannot use $summary//p since this is a document fragment. The <p> elements do not have a root element to which they belong, thus $summary//p has no proper context to find the 'child' <p> elements in $summary. Instead, use either the descendant-or-self or the self axis to loop through elements in such a fragment:

for $item in $summary/descendant-or-self::p
...

declare default element namespace "http://www.tei-c.org/ns/1.0";
import module namespace kwic="http://exist-db.org/xquery/kwic";
for $result in collection('/db/francotoile/data')//tei:TEI/tei:text/tei:body[ft:query(.,"francophone")]

order by ft:score($result) descending
return
kwic:summarize($result, <config xmlns="" width="40"/>)

Fixed a longstanding bug with the Google map that was causing Firefox to freeze on page load. It looks like Firefox can't handle this line (which loads the Javascript API file) in the <body>; it needs to be in the <head>:

<script type='text/javascript' src='http://maps.google.com/maps/api/js?sensor=false'></script>

Not sure exactly why Firefox freezes when the script is loaded in the body. Ideas welcome.