Today I added two new functions, f:pruneRecord($recId as xs:string) and f:pruneRecords(). These check one or all of the OAI records in the database to see if a there's a parallel correspondence document for it; if not, it's presumably been deleted, and the OAI record is also deleted. Tested (locally) and working.
Although the actual API for OAI requests and responses is relatively simple, the specification allows for a harvester to make large numbers of requests which can require significant amounts of data from the database; this is especially so in the case of our collection, in which we have a great deal of metadata to offer about every document. There is the risk that a single harvester could hit the db with enough data requests to slow the site for other users. For that reason, I'm taking a three-stage approach to designing the OAI features:
First of all, I have a set of routines that generate and maintain individual OAI records for every document in the correspondence. This is basically complete, and I have a full set of OAI records in my development version of the db; I also have routines written which check all the existing documents and refresh the metadata if they've changed, delete obsolete metadata records, and so on. This part of the work is done.
The next stage will be to write the actual query interface which allows a harvester to request this metadata from the db. Once that's done, the db will be able to provide metadata to a harvester in accordance with the specification.
Finally, I need to address the issue of maintaining the OAI records in the live database. The process of checking and regenerating records takes quite a long time at the moment (between ten and thirty minutes, depending on what it has to do). I don't want to be running that kind of intensive process on the live db. In the meantime, I can run it periodically on my local copy of the db, and upload the resulting records, but ultimately we want to have a more flexible system whereby any change to a document results in an update to its associated OAI record. I can do this using triggers in the eXist database, but it will require an update to a more recent version of eXist. This is something I've been planning for a while, and it will bring other benefits, but it's something I'll have to test carefully before we deploy it.
So I'd expect actual OAI functionality to go live by the end of the month, if nothing unexpected intervenes, and then I'll start the port to the new version of eXist. If that goes smoothly, implementing triggers shouldn't be too complicated, and then the OAI records will maintain themselves.