Image Markup Tool blog

HB wrote to ask for a keystroke shortcut for "New (with same categories)", so I've added one. I also updated the version info and date on the Help.

PC found a bug in the interface translation code while translating the Markin program. Fixed the bug in Markin at home, then migrated the fix to IMT and did a new release. I took the opportunity to update the way the site handles version numbers and download links, as I've been doing for the Markin site, to centralize all references to versions, making new releases easier and more reliable.

Version 1.8.1.0 of the Image Markup Tool has now been released. It's available from the IMT website. This version includes a new feature for automatically checking for updates, a small bugfix, and some updates to documentation.

Taking more code from the Markin program, I've implemented the update checking routines into the application. I also customized a Nuvola icon to use for the Update action, generated bitmaps for it, and rebuilt the icon dll to include it. Then I coded the PHP page which parses the version info passed to it, and shows customized update information (only the updates done subsequent to the user's version number).

"Check for updates" now seems to be complete and working, so the next stage is to update the documentation -- I think I could mention the update checking on the "Installing and Unistalling" page, and I'll also need to update the version info on the Help file itself. Then I can document the new feature on the updates page itself, build an installer, and do a new release. Should be done by lunchtime tomorrow.

Added the key update code to TFormStateSaver, and got to the point of compiling it, but haven't yet been able to test it. I'll begin testing in the Markin program at home, then bring any fixes back in. The logic is slightly complicated, but I think it's all functional.

One thing lacking from the application is a regular update-prompt system which would help users migrate to new versions (without pushing them too hard). We've been developing a strategy for this as part of the Markin program, and I think a similar system could work for IMT and Transformer. These are the details of the plan worked out so far:

• I. Checking for updates:
  1. App keeps a record of the last time it checked for an update for this user, in the user's Application Data folder.
  2. On startup, it checks that info. If the last check was more than a month ago, it asks the user whether it should check.
  3. If the user says OK, the check is made.
• II. Mechanism for checking.
  1. The update URL is stored in the application itself; it points to a PHP page.
  2. When a check is invoked, the app calls the update URL, appending a GET parameter which includes the current version number:

     [site].../update.php?userVersion=1.8.0.0

     It passes the URL off to the browser, which makes contact with the server.
  3. The update script runs on the server, and creates a customized page which shows the user's current version, the current release version, and a list of all the changes between the two; it also has a link to the recommended download. The page is created by PHP, which simply customizes existing XHTML by including or not including relevant rows in the bug/update table. The page is maintained by me, like HotPot and IMT update pages. When no version parameter is passed to the page, the whole update report table is shown.
  4. The user can choose to download, or, even if they're not an administrator on the machine, they can print the page with all the update info and pass it to the administrator, who can decide if an update should be done.

There's more detail in the Markin development notes. The next decision is how this object should be coded. The options are:

1. A completely distinct object. This would be a bit inefficient, because the object would have to do parsing of version info, and we already have a TAppVersionInfo object to do this.
2. A new feature of TAppVersionInfo. This would require that TAppVersionInfo be able to determine the application data folder, which would make it dependent (currently) on the rather bloated GenFunctions.pas library. Alternatively, the AppDataFolder function content could be abstracted from there and used directly in TAppVersionInfo. On the negative side again, this would require that TAppVersionInfo be able to save and load a file, which is more bloat for an object that's used several times in some of my apps.
3. A new feature of TFormStateSaver. TFormStateSaver is already quite large and complicated; it has its own TAppVersionInfo instance, and is already saving and loading files from the application data folder anyway. All it would need would be a new item to save in the file (the date of the last check), code to read and write it, a field for the URL of the update page, and a function that would construct and call the URL. The GUI aspects could be controlled from the main application, so GUI strings are not required in TFormStateSaver. There would be a function to determine if it's time to check (isTimeToCheckForUpdate), one to get the URL (getUpdateURL), and a property to record that the user was prompted (updatePrompted), which, when set to true, would cause the object to store the current date.

Based on this discussion with myself, I think the last option is the best, and I'll start implementing it. The PHP on the server side is a different problem, but not too complicated either, I don't think.

With Delphi 2009 due to arrive soon, it's time to start planning the feature set and data structures for IMT 2.0. The overall plan is to make the application much more flexible; it needs to handle documents with multiple images, and to allow much freedom in associating any "annotation" div and any zone on any image. These are my initial ideas:

• An IMT document is rooted on <TEI>, and contains:
  • A <teiHeader>
  • A <facsimile> element, which contains multiple <surface> elements, with each <surface> containing multiple <zone> elements, and each <surface> element containing one <graphic> element linking to an image file
  • A <text> element which contains a single <div>, which in turn contains a <div> for each annotation or transcription block. Each of these <div>s is associated (through @facs or @corresp) with one or more of the <zone> elements.
• Each image/<surface> is presented as a separate tab in the GUI, so you can move from image to image by clicking on the tabs.
• The annotations/transcriptions (we need a new word for this -- "zone-div"?) are presented in the current format, as a scrolling list, which can be filtered by category.
• Any zone-div can be associated with one or more <zone> elements, on one or more <surface>s. In other words, you can create a single zone-div which links to (say) three <zone>s on three different <surface>s. This enables efficient re-use of zone-div data, where features on different <surface>s need the same explanatory information. This breaks the current one-to-one correspondence between an annotation <div> and a <zone>, in IMT 1.8.
• Associations between <zone>s and zone-divs will be handled by giving each zone-div object a list in which to store links to <zone> elements (as opposed to the single pointer that currently exists). The zone-div parent list will have methods for finding associations by interrogating its children, in order to discover which zone-divs link to any given <zone>, or which <zone> elements are linked from any given zone-div.
• The above change will pose problems for the user interface. At present, when you click on a <zone> on the image, the associated annotation can be highlighted and displayed in the annotation window, because there is no ambiguity -- only one associated annotation exists. The same is true in reverse: when you click on an annotation in the annotation window, the associated <zone> can be highlighted. In the new system, not only can one zone-div be associated with more than one <zone>, but one <zone> can also be associated with more than one zone-div, so it's much more complicated to provide an efficient and helpful interface. This might be handled in the following way:
  • If you click on a zone on the image, and only one zone-div is associated with it, that zone-div will immediately be selected in the annotation window (retaining the simplicity of the current interface where the data structure allows it).
  • Similarly, if you click on a zone-div (annotation) in the annotation window, and that zone-div is associated with only one zone on only one surface, then that surface (image) will be shown, and that zone highlighted.
  • However, if you click on a <zone> with which multiple zone-divs are associated, a popup menu will appear, listing those zone-divs (by their tag-stripped titles), so that you can select one of them.
  • Similarly, if you select a zone-div in the annotation window which is associated with multiple <zone> elements, a popup menu will allow you to select which of them should be highlighted (listing them by image filename and coordinates).
• This takes care of navigating existing data, but a much more complicate problem arises with regard to creating such links in the first place. Since zone-divs and <zone>s are now only loosely coupled, we need to consider how a user might want or need to go about adding new <zone>s and zone-divs. These are some of the operations the GUI will have to enable (while, I hope, remaining simple and intuitive):
  • Adding a new <zone> and zone-div together, as currently; both are created at the same time, and are automatically linked. This should probably be the default behaviour, since most people will probably create most of their markup in this way.
  • Adding a new <zone>, but associating it with one or more existing zone-divs. Perhaps this would be Control + Add Zone, and would pop up a scrolling checkbox list of zone-divs in a modal dialog; if you select one or more and press OK, the <zone> is added and linked, but if not, the <zone> creation is aborted. When a <zone> is successfully created, the first zone-div of those associated with it would be selected.
  • Adding a new zone-div, but associating it with an existing <zone>. This might best be done using a right-click on the <zone> itself, and/or a right-click on the zone-div editing area. Another possibility is to have a drop-down list in the zone-div editing area, which has an entry for each associated <zone>, along with buttons to add and delete associations. Selecting an item in this list would foreground the <zone>.
  • Adding an association between an existing <zone> and an existing zone-div. This should probably also be done using a right-click on the <zone> element, and should also be available in the zone-div editing area.
  • Deleting an association between a <zone> and a zone-div. This could easily be done in the zone-div editing area, but it would be harder to do it through a right-click menu on the <zone> element, so perhaps this should be limited to the zone-div editing area.
  • Deleting a <zone> element. This would require pruning of all associations between zone-divs and the deleted <zone> (and might result in orphan zone-divs -- see below). The operation should also probably ask the user whether any zone-div which is ONLY associated with the <zone> which is to be deleted should also be deleted.
  • Deleting a zone-div. This is simpler than the above, because the associations are all stored in the zone-div object; but as above, it might result in orphan <zone> elements. This operation should also ask the user whether to delete any <zone> elements which are only associated with this zone-div.
  • Handling of orphans. If you can delete associations, then you can easily delete all associations between zone-divs and <zone>s, so any given <zone> may be left without any associated <div>, and vice versa. This is not a problem at all, in fact, but it may be something users should be warned about at the point where they create the orphan(s). The software as a whole needs to handle orphans of both kinds without problems.
  • Question: would it ever be necessary to allow the addition of a zone-div without an associated <zone>? It would be possible to create such a thing by adding a normal pair, then deleting the <zone>, but do we need to allow for direct creation of unassociated zone-divs?

One more thing we need to think about is the question of metadata, both at the document level and at the level of <surface>. Where there are multiple images, each will perhaps require distinguishing data at the level of the surface tag, and we'll need to provide a good interface for that. At the same time, as the overall complexity of the document increases, people will probably want to add more complex information in the header. I don't know how we could make that easier, without actually creating an XML editor for it, so perhaps we can leave that as it is.

The most difficult aspect of the rewrite will be retaining, as far as possible, the simplicity of the current interface, so that existing users have little trouble adjusting, and are able to continue to work on simple, single-image documents in the way they're used to working, while making all the new options easy to access.

Finally, validation: do we want to think about that? It would be extremely hard to implement, especially with RNG schemas, but if it could be done, and errors could be retrieved and shown correctly in the GUI, it would be quite useful.

PC reported that the About box was showing two versions on Win98 and WinME, the old 1.7.2.6 version and the new 1.8.0.0. On investigation, I found the application options "Product version" setting was set to the old version; however, I don't use this information when creating the About box, so I suspect that Win9x is incorrectly reading the product version info from the executable. Since the problem doesn't show up on Win2000+, and we don't officially support Win9x, I've just made a silent change of the Product Version to "1.8" (which is what it should be, rather than the full dotted version, which is stored in a different place). Waiting to see if that makes a difference. What seems to be happening is this:

In my app, I output a string between the copyright info and the link, based on the "Comments" field in the executable version info. In IMT, there is no comments field, so there's nothing to output; on Win2000 and above, nothing appears there at all (in fact there isn't even an empty line). However, on Win9x, it seems that if there's no Comments field, Windows just reads the Product Version field instead. I think that's where the information is coming from.

PC pointed out that the licence text file was showing the wrong version number, so I updated that and rebuilt the installer.

PC sent a new Italian GUI file for the IMT, based on the new release, so I rebuilt the installer to incorporate it, and uploaded it to the site.

Finished the DocBook conversion code, and tested round-tripping, then updated all the help/tutorial files, and upgraded all the distro P5 files and HTML output. Built the installer and tested it; found a bug with the category manager, which caused the top category in the list to be obscured by the columns above it. Fixed the bug, tested the app and the installer on both Linux and Windows (VISTA), and updated the Website to make a formal release. I won't announce this to the TEI list until we've hammered it a bit.

I got the version 1.8 file i/o completed and tested, so the new "transcriptional" feature is correctly saving and loading, and dictating the use of @facs or @corresp. Then I started working on the DocBook export/import. I'm mapping the new <desc> element in the <rendition> element (the annotation category data) to a tag like this in the equivalent docbook <member> element:

<property role="transcriptional">true</property>

For speed and simplicity, I'm also storing, for each individual annotation <div> in TEI, an @remap attribute (remap="facs", or remap="corresp") on the db <callout> element.

Next, I need to:

• Update the docbook-to-imt file accordingly, so round-tripping works
• Rewrite the Help/docs XSLT to take account of the new format.
• Convert all the IMT documentation files to 1.8, then to the new docbook format.
• Update the screenshot for the annotation category list, and the category editor screen.
• Update other Help file and tutorial stuff to explain the new boolean/checkbox.

I've elected to store the "transcriptional" boolean in the form of a <desc> tag like this, inside the <rendition> element:

<desc>transcriptional</desc>

I have this correctly saving and reloading from the file; what isn't done yet is the distinction between the use of @facs and @corresp for linking. I'll have to do this by giving the annotation list a method pointer which can be set to point to TAnnCatList.GetTranscriptionalFromCatID method. I'll have to make sure that binding is set up by the TIMTDoc, in its Create method. Then the annotation list can find that information and decide which attribute it needs to use when saving the file.

I had assumed that the decision to link an annotation using @facs or @corresp, depending on the "transcriptional" setting of its category, would be a simple one, but it turns out to be slightly more complicated than I had hoped. Ironically, this is because I have good object-orientated code with separation of concerns; annotations write themselves to the output document, at the behest of their annotation list, and neither they nor their annotation list know anything about the category object except its CategoryID, which is known to the annotation list object. Therefore the neither the annotation nor the annotation list which manages it have any way to know directly whether @facs or @corresp should be used.

When we're reading documens back in, this is no problem; the annotation can check for @facs, and if it's not there, check for @corresp. I already have that code written and working. The problem is only in writing out the files. I really don't want to undermine the clean separation of the objects, so I'll probably have to create some kind of callback function that the annotation list can use to query the category, to find out what kind of thing it is.

An added complication, though, is that there is no useful place to store the category's type (transcriptional or not) in the <rendition> element which is used to store category information. My (rather vague) plan had been to have the category infer its own nature by looking at the attribute used by annotations linked to it, but this is actually a bit circular, and even if it worked, it would leave us in a situation where we couldn't save the state of this boolean without there actually being an annotation in that category. That's obviously unacceptable; documents are regularly saved with categories that don't yet have annotations in associated with them, and we can't afford to throw away category data when this happens. So I'm going back to the drawing board (or the TEI schema).

The next version, 1.8, will have two basic changes: first, the bug reported by JC will be fixed (@facs attributes will begin with a hash), and there will be a boolean field applied to categories which defines them as transcriptional or non-transcriptional. The former is a straightforward change, and I've begun work on it by first updating the schema and ODD file (no changes except in documentation and filenames), and then by writing all the XSLT conversion files for the new version.

The next stage will be to plan for the handling of @corresp vs @facs. I need to figure out whether I should store a value in the category definition itself (a rendition element, which has no @type attribute).

While I was away, JC reported a bug with the file format -- @facs references should be preceded by a hash, of course. Easy to fix, but I'll integrate the fix into the next release, when I add discrimination between transcriptional (@facs) and non-transcriptional (@corresp) associations between annotations and zones.

PC sent over the latest Italian interface file for IMT, and also reported a couple of bugs. There are two or three bits of text which don't translate properly -- one being the Hint property of the UniSynEdit controls, which paradoxically is not Unicode -- and also reported that when you use Insert Tag from the main form menu when working on the Annotation form, nothing happens. Actually, it operates in the context of the form with the focus, so it works on the main form instead. I've worked around this by adding a little drop-down menu to the toolbar of the Annotation window, so users will use that instead of the main window. This needs looking at again, though.

PC also reported a bug relating to the save location and filename of thumbnails, which I've fixed. Released 1.7.2.6. The Hint problems will have to wait, if they can be solved at all -- but they're very minor and the advantage of syntax highlighting etc. outweighs the absence of a hint.

A new version of the Image Markup Tool has been released. These are some of the changes:

• The application now has XML syntax highlighting in the text boxes where blocks of XML are edited (the annotation text field and the teiHeader area).
• The Edit menu now has a new command, Insert tag (Control + E), which provides a simple interface for inserting tags.
• A "PortableApps" install option is now included, for creating a portable installation which saves all settings in the program folder. This is useful if you're installing the program on a removable drive

Spent much of the day figuring out something that will be portable to all our other Windows application projects, but which I'm doing for the first time with this IMT release: building an installer which can install the app both in "normal" mode and in "portable application" mode.

In both cases, the installation is basically the same; the difference is in how the program is run.

The "portable" install is designed to be used on a thumb drive or other removable drive, and its main feature is that settings, window positions, sizes etc. are all saved in the program folder, so that they travel with the drive. In a "normal" install, these settings are saved in the user's Application Data folder. The "portable" option is designed to be compatible with the PortableApps standard 0.9.8 (see the PortableApps website for more information). If you install the program to the PortableApps folder on a thumb drive, the PortableApps menu will automatically find it. However, regardless of which install type you chose when installing the program, you can still run the program in either mode.

One difficulty lies in the fact that a "normal" install ought to be registered for uninstall, and gets its uninstall application dropped automatically into its program folder; if this happens for the "portable" install, then the PortableApps menu system finds the uninstaller and puts it on the menu as if it were a regular application. I solved this by running a batch file to move the uninstaller and its data file, at the end of the process, to another location.

Other issues involved ensuring that state info files get removed when uninstalling from a thumb drive, and making sure that icons and shortcuts work in either case.

I'd intended to complete a new release today, and I did get the Help file updated, version info fixed, and various other bits of prep, but I need to rewrite the installer to support PortableApp installations, and that can't really be rushed. I ran out of time, so I'll finish it tomorrow.

My first approach to a fix, which was to cast the variable value to a number whenever using it, failed; the user reported no difference. This morning, I installed an Italian locale on Ubuntu, and tested the IMT there, but didn't see any difference at all; everything works fine for me. This suggests that the problem was in the locale for Wine itself, rather than for Ubuntu.

This sent me back to the IMT code, and it turned out that I was generating the value for the XSL variable using Delphi's FloatToStr function, which uses the system locale. That looks like the cause of the problem, so I've changed that call; I now set the DecimalSeparator specifically, and then use FloatToStrF(ScaleFactor, ffGeneral, 6, 2), which should give a result with a period for a separator on any locale. I'm now waiting for the user to report back on whether this works or not, because I haven't been able to create a test system that reproduces the problem so far.

Had a couple of bug reports from a user, some of which are due to Wine, and some of which are probably my fault:

• The XSLT Web View transformation fails to calculate zone offsets for this user; they all come out as NaN. I think I may have pinned this down to a problem based on locale; when the $ImageScaleFactor is calculated, his log shows that it has a comma for the decimal separator, and when the number is later used in a calculation, the calculation fails. On my computers (Win and Linux) it succeeds, but presumably my locale ensures a period separator. I'm trying a solution which uses an explicit cast (number()) when the value is used; we'll see if that helps on his system.
• The Annotation window disappears when you move to another desktop and back again, on Wine. This is definitely a Wine bug, but it's presumably related to the fact that the window is set to always-on-top; that's why the bug doesn't affect regular windows. I determined that it was getting a negative Left setting during the desktop cycle, so I've added code which checks for negative values and sets them to 0 when the window is activated. It still disappears, but now you can get it back by clicking the show button off and on again.
• When FormState saved the application's state to the Application Data folder, it was being saved to a folder named only "v.1" instead of "Image Markup Tool v.1". This turns out to be caused by the fact that Wine doesn't correctly read the product name field in the version info record, so I've rewritten that code so that if no product name is found, the filename of the executable is used instead.

I'll do a new release tomorrow, if the XSLT bug is solved.

I've become interested in the Portable Apps idea, especially since we'll need to be travelling and demoing our apps in Finland. Having learned about the PortableApps system, and built drives that run Tomcat stacks, I decided to modify my apps so they'll run in portable contexts. I've done this for Transformer and IMT. When you run the app with a -portable flag on the command line, it stores its stuff in a Data directory which is in the application tree, rather than in the Documents and Settings tree on the hard drive. I also had to create a special launcher for each of the applications, to conform with the PortableApps standard. (In doing this, I discovered that the PortableApps menu doesn't follow its own documentation, so I had to do some hacking.)

Everything seems to be working well on thumb drives with a manual setup. Now all I need to do is elaborate the installers for each of the apps, so the user can choose to install to a thumb drive as a portable app. That'll take a couple of hours of figuring out, but it shouldn't be too complicated.

A user suggested that a Back button would be helpful in the browser-based Help system I wrote for IMT and Transformer, so I've implemented that (changes to XSLT and JavaScript). Started it at home (I'm using the same system for Markin 4), then finished it at work and tested with the IMT Help.

Today I fixed an erroneous tab order in the annotation window, and also added a routine that eliminates placeholder text and places the cursor inside the tag in a new annotation, when you activate the annotation text box. This saves time. I've also tweaked the zooming functionality so that when you zoom, if an annotation is selected, it's centred in the zoomed window. It's now much easier -- in fact it's automatic -- to zoom in on the annotation area you're working on.

Updated the Help file and the online source code repository to include the new dialog box; also created an icon for it, and updated the icon library.

I've now added a dialog box which is invoked by Control + E (or an appropriate edit menu control), which pops up an XML tag insertion dialog box. This allows you to type the name of a tag, or the name followed by attributes with their values (i.e. the interior of an XML open tag). It will then tag the selected text (if any) with a complete tag automatically. It also makes use of the FormState.pas state-saving code to remember up to 16 tags you've used, in order, and show them in the combo box, so you can easily choose tags you've used before.

Nothing beats using the application yourself for hours on end when it comes to discovering optimizations and annoyances. After today's work on Mariage, I've made these changes:

• Fixed a problem with edit commands not working in the new syntax-highlighting TSynMemo controls. This was caused by my forgetting to update onIdle events and TAction events so that they allow for TSynMemo components as well as descendents of TCustomMemo.
• Added a new popup menu for edit commands in the Annotation/Category window. Going to the main menu changes the active window, which causes the active control to change, making it impossible to use that Edit menu when working in the Annotation/Category window.
• Added a main menu command for Clone Annotation, a TAction that already has toolbar buttons, but had no menu item, so might have gone unnnoticed. It's extremely useful.
• Built a new installer and posted it, and rolled out fresh executables to Chicory and Endive.

Upped the version number to 1.7.2.0, and added UniSynEdit components and XML highlighters in the teiHeader editing area and the Annotation Text area. This is undocumented and untested as yet, so I've just built an installer and installed on Endive and Chicory so CC and I can find any bugs in the next few days.

The final steps to release were the predictable grind: trawling through all the documentation, tutorial and help looking for screenshots and explanations that needed updating, adding of licence info to headers of files newly released as open-source (such as my Unicode spin-edit control and find/replace dialog), updates of the Website, and lots of builds and tests of the installer. Got through it, and announced it.

The Image Markup Tool version 1.7.1.0 has been released. You can get it from here.

This is a build-update rather than a version-update, meaning that the XML file format used by the program has not changed. These are the major changes:

• a fix for a bug in the Web View output, which afflicted documents with very large numbers of annotations
• the addition of a feature which enables the user to specify the xml:id attribute of the annotation <div>, which is useful especially if you're integrating IMT documents into larger projects
• the addition of Find/Replace functionality to the GUI

The TIMTDoc.FindNext routine had an endless-loop bug caused by failure to detect wrapping around correctly (in other words, having searched the whole document and got back to the starting point without a result, it should have noticed that and stopped, but it didn't). It was caused by a failure to set the ItemIndex field to -1 when wrapping back around to the teiHeader; the teiHeader doesn't use ItemIndex, because there's only one of it (as opposed to annotations or categories), so I was neglecting to set that field at all, but the test for wraparound depends on both the field type and the item index being the same as the starting point, so it does need to be set to -1. Fixed it.

CT sent me a link to the new, more precise specification of @facs, which aims to prevent my generalized usage of it as linking anything to a facsimile element. @facs will now be limited to "groups elements corresponding with all or part of an image, because they contain an alternative representation of it, typically but not necessarily a transcription of it." That's still pretty vague, but it's intended, I think, to exclude annotation. IMT needs to change to reflect this.

After some consideration, this is what I think should happen:

• Version 1.8 should add a new checkbox/boolean to the category properties, specifying that this category is transcriptional in nature (vague, but so is the above spec!). I haven't yet figured out a way to store this information in the <rendition> element of the TEI file. Perhaps I should just add an @facs attribute to the <rendition> tag, pointing at the <facsimile> element?
• Where this boolean is set, the annotations should be associated with the area using the @facs attribute; otherwise, the @corresp attribute should be used.

If this proves too complicated for the moment, a quick solution would simply be to switch from @facs to @corresp.

Finished the Replace All functionality, and documented the Find/replace dialog box for the Help file. Also found a bug while doing this: creating a new category when there are no categories (the cat list is empty) would result in an index out of range error. Tracked this down and fixed it. The difficulty of this showed some excessive complexity in the codebase; the handling of categories is shared between four forms and several other libraries, because you can get at them in so many ways (using the actions in the main and Annotation windows, and also through the Category Manager itself). This could really use some optimization in the long run.

Build the Help file, so we're now basically ready to go, but I want to do some more testing before I go for a release. This one might even warrant a beta release.

Wrote the code for ReplaceAll, but no chance to test it properly because the debugger crashed Delphi and I ran out of time. I'll finish it tomorrow. Then we're down to testing, debugging and documentation of version 1.7.1.0, with a release possible within a week.

A Vista user sent sample Web View output where all the pixel offsets on the Annotation areas are NaN. This is mysterious. I set up a Vista VM and tested it out with their own data, but can't reproduce the error at all. Still working on it -- the user thinks it might be security software, but I don't see how.

Still the Vista VM will be useful...

This is the last of the major find/replace components, and I have the basics of it mapped out now. I should be able to get it done tomorrow, all being well.

Continuing some work I started at home this morning, I've added a new feature to the FormState library, so that it also (if desired) saves and reloads the current directories of dialog boxes. This will also be very useful for Transformer, which has lots of dialog boxes, and where users typically want to keep the individual directories they last used for each one of them.

The problem with whole-word-only searching seems to be a function of the TURESearch class which is in the JEDI JCL library. Alongside that class, though, is another unicode implementation of TSearchEngine, named TUTBMSearch (for Unicode Tuned Boyer-Moore). Switching to this class seems to have completely solved the problem.

Now that FindAll is basically working (the only caveats are also applicable to FindNext, and are external to FindAll itself), I found it straightforward to implement the GoToSearchHit routine. The list of KWIC items found in FindAll can now be double-clicked, to jump directly to one of the items, highlighted.

Before getting back into the difficulties of Replace, I decided to get my head back into the code by tackling the relatively simpler FindAll functionality, which involves retrieving and storing a KWIC-style display for the user, who can then double-click on individual hits to view them in the GUI. Got this working fairly well, although one bug remains: the WholeWordOnly setting doesn't seem to be passed through, or if it is passed through, it doesn't seem to be honoured by the FindNext function. This needs to be fixed. I should also test the MatchCase parameter.

Fixed the menu bug reported by DP, and also ported the fixes (XSLT, CSS and JS) into the version of the code which is used on the Mariage site. The Mariage code is very similar, except that it uses camelBack notation, in order to match the rest of its code; this raises the issue of whether I should camelBack the IMT code (by which I mean the XHTML, XSLT, JS and CSS used in the Web View).

DP reported a bug in the Web View output that affects pages with huge annotation lists. The annotation menu appears to have a scrollbar, but in fact it doesn't scroll, so you can't get down to the bottom elements in the list; and because it's position: fixed, even though the page itself scrolls to accommodate it, you can't get down to the bottom of the list because the menu doesn't move with the page. The sample page is here.

There's a JavaScript function called Initialize() that does a lot of the positioning and sizing work. It contains this line:

document.getElementById('AnnMenu').style.overflow = 'scroll';

which should make the scrolling work, but I suspect the height of the container element is being extended too far, and the container doesn't have a scrollbar. I see a line commented out, just before that:

//  L.style.maxHeight = ImgHeight + 'px';

That would set the max height of the menu container, which might achieve the desired effect, but if I've commented it out, it's probably because of some unwanted side-effect (probably with IE, which doesn't support CSS max-height).

Testing the IMT under Wine on my EEE PC at home reveal some problems due to the low screen res (800 x 480) of the EEE PC. Since I'm planning to present the project on the EEE PC in June, I need to work on these, and it makes sense to fix them anyway, because they may affect other users working under Wine.

The core problem is that the annotation category window shows up sized to the desktop height (which is 480px less the taskbar height). When this happens, the bottom two controls (OK and Cancel buttons) are chopped off, although oddly the status bar, which is below them, remains. Even though the window manager (IceWM in this case) is set up to allow windows to be bigger than the desktop, when you resize this window to try to get back the controls, you just get a black area with redraw failure artifacts.

There are various possible reasons for this, and these are my approaches to them:

1. The form style of that particular window is unusual; it's fsStayOnTop, ensuring that it appears above the main window at all times. That could be screwing up IceWM or Wine, so I've added an option to hold down the Shift key while starting the app, which should set it back to fsNormal for testing purposes.
2. The original window size was nearly 600 pixels, meaning that the window would have to be aggressively resized in order to fit on the screen. This might well happen before my own code for restoring the form state (including window size) kicks in, so the problem may occur before my code even tries to resize the window. To avoid this, I've reduced the original window height considerably, putting it well under 480px. I should be able to test starting the app with no previous window size settings (having deleted them from the user profile), and see if that successfully avoids the aggressive resizing, and if so, whether it solves the problem.
3. There may be some constraint somewhere in the code which is interacting with the aggressive resizing done by the window manager. I've searched for this, but all I can find are MinHeight settings on two of the panels, one at 100px and one at 200px; these combined do not prevent the window from sizing to 480px, so I don't thinkt that's the problem.

I'll test these changes and see if they have any effect.

On the surface, Replace is just Find + changing the selected text, but of course it's not that simple; the change puts the textbox data out of sync with the back-end document, so there needs to be a save before the next operation is invoked.

This actually applies to some extent with Find alone; if there's dirty data, I guess changes might be lost with the right combination of actions. I think it makes sense, therefore, to check for dirty data before every Find or Replace operation; or perhaps to save a Replace change back to the document automatically. If the latter, we risk confusion, as the user may assume that, since they haven't pressed OK in the edit box, the data isn't saved. If the former, then we risk annoying the user with popups. One final option is that, if an item is already visible, we search in the live text of that item (ie the text in the GUI) rather than the back-end document. This would be the most intuitive approach from the user's point of view, but perhaps the most programmatically fragile because of the need to search and display hits in two distinct ways, depending on whether an item is live in the GUI or not.

This needs some careful thought.

Implemented a simple tracking system so that the main form knows which of the three document forms in the app last received focus, and the Find now works because it knows where to start from.

All three bugs in the previous posting are now fixed. The Find functionality appears to be working pretty well, with the exception of one new bug which has emerged:

When a search hit is found in an annotation title, it often seems to stick on that ann title and not move on when Find is pressed again; it's as if if fails to get the next container correctly, and searches again within the same container. The reason for this is that the code which determines where the cursor is currently located (GetCursorLocation), which is called to find out where to start searching from, cannot yet tell where the focus last was; it can't just ask which form has focus, because obviously what has the focus is the Search form itself, which has invoked the Find action. Therefore, currently, it just checks to see which windows are visible, and if the teiHeader window is visible, it assumes that one had the focus. Of course, if searching has been done, that's not necessarily the case; it may just have been left open. But if so, it always starts the search from that container, working forward, and therefore always finds the same hit if the hit is in an annotation title.

Currently working on a solution.

Found and fixed the bug in the code which checks to see whether a search has looped right around, so now searching stops after traversing the whole document. Then I looked at the search code itself, and found the bug which prevented searches from working. Now searching works, and repeated searches take you through the whole document. These are the remaining bugs with Find:

• Anything that finds a hit in a Category ID causes an endless loop. This is probably just a typo that keeps sending the thing back to the same container.
• Although hits in annotation titles, annotation text, and category explanations are all correctly highlighted, when a hit is found in the teiHeader, the selection is off by one (shifted forward one character). This might be caused by trimming taking place in some place but not in another.
• There's no default action that happens when you press Return in the find/replace dialog box. It should probably be Find, which is non-destructive.

These don't look like showstoppers.

The writing of the search/replace code goes on and on, ever more complicated, to the point where I wonder if I should have taken a different approach. Still, I can see the light now.

Today's progress:

• Completed the FindNext method of TIMTDocument, which is the heart of the single-shot Find and Replace functions. It's very complicated now, with several embedded functions, much of it in the form of Case statements which branch based on the nature of the container we're currently searching, to find the next container that should be searched.
• Added the following methods to the ufrmMain (main form) GUI object:
  • StashDirtyData. This was complicated, because it threw up a new problem. Normally, when the header or the category data is being edited, that's happening in a modal form. However, the search functionality needs to be able to throw up those forms with selected text, and keep them onscreen while you search again, so they have to be showable as non-modal; and that means they can all have dirty data in them. Working around this involved a lot of changes to the way the header and category editing happens, in a number of different objects.
  • Find, FindAll, Replace, ReplaceAll, and GoToSearchHit. These all implement the function signatures the ufrmReplaceDialog needs to call. Only Find has been implemented so far, and that has bugs (see below).
  • aFindExecute, and action to set up the search dialog and show it.
  • GetCursorLocation, which records the position of the cursor (what container it's in, and the details of any selection) so that this can be used as the starting point for the search.
  • ShowSelection, which should show and select any hit that's found. Again, this is complicated Case statement stuff, depending on where the hit is found.
• Began testing, and discovered two bugs: First of all, the searching fails (a known hit is not found), and secondly, the check to make sure that you don't loop endlessly around the document is also failing, so the search is endless. These shouldn't be too hard to figure out, though.

This has been remarkably complicated, but it's because the document structure is complicated, and it's mapped across lots of different GUI components and forms. I don't seem to have broken any existing functionality so far, but the possibility of having three different components of the document open for editing at the same time, potentially all with dirty data, is going to take some care and attention. We'll have to make sure that the StashDirtyData function is called when saving files, or before closing a file.

Long embedded case function for returning the next container in the document to search is now finished. The actual search function body can now be written, checking document elements one by one till it finds a match.

I'm now about half way through implementing the search/replace. I found some things that needed improving in the code already written. For instance, I was failing to check for well-formedness when writing nodes back to the document which had changed due to string replacements. Then I realized I wasn't tracking successful replacement instances (as opposed to find hits); where replacement would cause a broken doc, the replacements are not done, so find hits are not equal to replacements. That's all fixed now.

Finally, I've mapped out the function for doing a Find Next, which is complicated because it can be upwards or downwards, and has to iterate through the many different document components appropriately. I've not yet added the MatchCase and WholeWordOnly booleans to the function signature -- mustn't forget that. I should be able to call SearchContainer to do the actual search, so that there's the potential for rudimentary regexp functionality in the future.

Another few hours should get me to the working-GUI point, where I can really start hammering it for bugs. It's way more complicated than you would think, especially if you're going to protect the user from destroying their document structure (which I am).

Updated the screenshot and all associated references in the documentation to explain the id editing feature. This also acted as a check to ensure that none of my changes have affected the app's ability to import/export docbook files, or the help file compilation system that exports the docbook to HTML.

Previously, the @xml:id attributes of <zone> elements (and the corresponding @facs attributes of annotation <div>s) were assigned automatically by the program. This is less than ideal, because users (especially sophisticated users) are likely to want to use memorable strings for these attributes, so that they can more easily link to or refer to them from elsewhere in the document, or from other documents. Remedying this deficiency has been on the roadmap for a long time.

I've implemented what I think is the simplest and most reliable approach to editing the ids. First, I had to put in place auto-generation of valid ids when a new annotation is created, so that the id is a property of the annotation itself (rather than just created and used during i/o processes, as before). Next, I added getter and setter routines at the level of TIMTAnnList and TIMTAnnotation, which check for validity and uniqueness. Then I updated the i/o code to ensure that these are being saved and loaded correctly. Finally, I added a simple and unobtrusive GUI method for setting the ids, in the form of a little button on the right of the annotation title. This will not cause much confusion for regular users, and won't change any other functionality, but it's easily accessible when needed.

Initial testing suggests everything is working OK. Now I need to update the screenshots in the documentation.

Implemented a the ReplaceAll bits of FindAll, and tested them. That seems to be working fine. Next is individual find/replace operations.

Progress today:

• Added the SearchContainer functionality to the TimtSearchList object.
• Gave TIMTDocument a TimtSearchList object so it can search itself, and added a FindAll function that populates the search list.
• Did GUI testing and debugging to confirm that the FindAll functionality works. A list of hits, with detailed info about them, is stored correctly.
• Began retro-fitting both these functions with the ability to optionally replace the hits as well. This is slightly complicated because of the effect of replacement on the offsets of hits in the same neighbourhood, combined with the option to search both downwards and upwards. I haven't finished this yet. Also outstanding is the mechanism for ensuring items changed by replacement, which may be currently visible in the GUI edit boxes, get updated.

Once this is done, we'll have handled FindAll and ReplaceAll. The next question is what to do about FindNext. We'll probably have to adapt code from Markin to iterate through the containers in the document, searching each one in turn.

I regularly test IMT on Wine under Ubuntu to see whether the bugs with dialog boxes have been fixed. Testing with the latest version at home, I found that the IMT worked fine using the latest Wine version (0.9.55), set to Windows ME emulation. This morning I tested this thoroughly and confirmed it does indeed work; then we tried running it on OSX under DarWine (also using Wine 0.9.55). The latter failed, unfortunately, but not with all dialog boxes (as used to be the case). The only problem was the Open Picture dialog box, which renders a preview; this seemed to send DarWine into a tailspin.

All this is very hopeful, so I've updated the Website to include detailed instructions for installing Wine/DarWine and running the application under it.

While the blog was down yesterday, I began working on adding find/replace functionality to the application. The Find/Replace dialog box is one I've already created for my Markin program, and which I'll open-source so it can be used in IMT. The key issue is figuring out how to find hits, store info about them, and jump to them. We have to create a special unit to handle this functionality, for clarity/separation, and it should include:

1. An enumerated type for containers: contTypes = [ctHeader, ctAnnTitle, ctAnnText, ctCatId, ctCatDescription].
2. A class which can store our current position/a hit position -- maybe TimtSelection. The data is presumably this:
   • Current container type (header, annotation title, annotation text, category id, or category description))
   • Item index (0 for header, annotation number for anns, cat number for cats).
   • Offset of cursor.
   • Selection length. Obviously this will normally be the same as the length of the search text, but we will presumably also store the original cursor position and selection, and restore it after a search/replace operation; and in the long run, there may be regexp functionality.
   • SelText. This is the text itself, and may help with a sanity check if the document has changed since the text was found.
3. A list class for storing these objects (for when FindAll is used).
4. A method for searching text (taking into account the Down, WholeWord and MatchCase) from the cursor position, and reporting back the correct offset and length (taking into account the preceding text length, if going downwards).
5. A method for finding the next container to search, based on the current container (this can be modelled on my similar code from Markin).
6. A method for making visible a container in which a hit is found, selecting the hit, scrolling it into view, and setting the application and form focus appropriately. This method will have to accept a TimtSelection object, do a sanity check on the data, show the appropriate container, and select the text; if the selection turns out not to match the SelText, it might also show an error message, or try to do another Find to correct the problem.
7. Implementations of all the functions required by the TmdhReplaceDialog Setup routine. Most of these will be wrappers that just call methods detailed above.
8. GUI elements in the Main and AnnDlg forms -- presumably an action, and associated menu and toolbar items.

Spent time working out the details above, then started coding the unit; I've done about a third of it.

Paolo C. provided an updated Italian interface file over the weekend, so I've rebuilt the installer to include it. No version change.

Updated all pages on the site, simplifying a lot of them, and added all new source code to the repository. I'm also now providing downloads of older versions of IMT going back to 1.3.0.5, in case people have projects they don't want to port forward.

Everything works as expected on XP (old and up-to-date) and on Vista. The only oddity is that the image scrolling is much finer-grained (slower) on Vista; that'll need looking at after Vista SP1 is released.

Completed any which were ripe for work, one of which was improving mousewheel behaviour on the main image; made that much more sophisticated, and documented it in the Help and tutorial. About 5 todos remain, but they're mostly irrelevant, or blue-sky future ideas.

Discovered that although my app has unicode-capable spin edit controls, it still wasn't correctly loading unicode hints for them. Created a simple test application for the translation code, to try and narrow it down. Figured it out and fixed it.

Worked through the Help files, examples and documentation, updating text, screenshots etc. Tedious but necessary. Tomorrow I can build and test and installer.

Replaced the old Web View dialog with Settings, and implemented the new behaviour for copying the schema file to the save folder when saving a file. This is slightly complicated because we now have to have an overwrite prompt for two files, so I'm using the ufrmOverwriteConfirm library used elsewhere in IMT.

For the new Settings dialog box, I needed an appropriate icon, and I found a generic one in the openclipart.org collection. Trying to add it into the nuvola.dll library which has all my icons, I discovered that the script that renders svgs into 12 different bitmaps fails on GIMP 2.4. After some research, I discovered why: variables must now be declared before being set. After adding declarations, it worked on Windows and on Linux. So I was able to add the new toolbar icon to the dll, and rebuild the dll.

The Web View XSLT code was using the wrong x and y coordinates for the Zoom functionality (lrx and lry instead of ulx and uly). Fixed.

D.P and A.C. on the TEI-L list come out resoundingly in favour of @facs; only C.T. really disagrees, so we're going with @facs.

I'm still in a debate with Con T. and others on the TEI list about which attribute is appropriate for linking <div>s to <zone>s in IMT. This has taken quite a bit of time over the last little while, and I can't release the new version till it's settled. I favour @facs, and so far the debate seems to be tending that way too.

The release plan calls for a new option to copy the schema to the same location when saving a file. We already have a generic Preferences dialog (for app interface changes), and a Web View Preferences dialog (with only one item); I've decided to replace the latter with a new Settings dialog which will contain both the Web View option and this new one. I created the dialog box; now I need to find an icon for it (some sort of Configuration/Settings/Options icon), and build that into the dll. Then I can implement the new functionality.

Rewrote the XSLT to create the Web view, and tested it. In the process, I found a couple of small inconsistencies in file handling (basically inadequate error-trapping), and fixed them.

Fixed these bugs.

Import/export from DocBook is now working. Next is the Web View code, which will be a little easier because it's all one way.

Wrote and tested the file loading converters (XSLT) for versions 1.0 through 1.6, converting old files to the new IMT 1.7 format so they can be loaded.

Next is the DocBook stuff and the Web View output code.

Rewrote i/o code in the app, and it will now save and reload the new format successfully. However, there's an error when importing an old file for some reason. That'll need debugging tomorrow.

The final release of P5 has some new elements in the transcr module which can replace the use of SVG in IMT files. This will make life much simpler, so I will be converting the application over to this new format for version 1.7. The first stage in doing this was to generate a schema from ROMA, and to write a conversion file (in XSLT) to get us from version 1.6 files to the new output format. I've now done that, and sent a sample file to Conal T, who's one of the prime movers in the facsimile proposal, to see if he's happy with the structures I'm using.

Once I get the OK on this, I can move forward with the conversion. These will be the steps:

• Create the rest of the conversion XSLT files, to handle older IMT version files.
• Create a range of sample files in the new format.
• Rewrite the application file-reading code to read in the new format.
• Test the read capabilities.
• Rewrite the file-saving code so that it produces the new format.
• Test round-tripping files carefully.
• Rewrite the Web View output code so that it works with the new format.
• Rewrite the DocBook import/export code to do the same thing.
• Update the documentation.
• Build and installer.
• Test, test, test...
• Release.

Paolo Cutini reported one bug when sending in his latest translation file (which I included in t new installer build and posted); and I found another when checking his. They're both cosmetic:

• The Hint properties of TSpinEdit controls are not written back to the control when they've been translated, and the translation file is loaded. This is probably related to the fact that they're strings, not WideStrings; the hard solution would be to create a TTntSpinEdit which uses WideStrings, and the soft solution would be to disable hints on those controls, and make sure that their associated labels have good hints.
• If you load a translation file, then restart the app, then go back to the Preferences screen and clear the translation file, the original English is not restored. That's because the BackupAppRef in which the original English texts are stored is not created unless you go to the Preferences screen and load another translation; we need to create one when the app starts up, before another interface is loaded, and use that when reverting to "no translation file".

These are documented on the app's bugs page, and this is set as a task for November (the next scheduled release).

All day on this -- completed the thumbnail functionality, documented it, updated help and tutorial, built and tested installer, updated Website, updated source code archive, and announced the release to the TEI list. In the process, killed my machine with Delphi, and updated the NVidia driver in an attempt to stop it happening again. Long day...

After some research, I discovered that saving to PNG format would require the TPNGImage library, as well as an additional unit posted here. That's not too big a hurdle, but TPNGImage is not licensed under a standard open-source licence; it just has some simple terms and conditions. I'm not sure whether it's compatible or not with MPL 1.1, so I'm deferring the feature which would save thumbnails to PNG for a future version.

This is what remains to be done for version 1.6:

• Create the dialog box for the thumbnail function. This should be highly configurable: at the top, we need a radio button pair for "Entire image" vs "Selected area only", which should be pre-selected based on whether there's a current selection or not; then there should be an output size area, with linked width and height elements; and finally OK and Cancel buttons. When OK is selected, a Save Image dialog box should allow you to choose between JPEG and PNG, and then the thumbnail function should be invoked with the values chosen.
• The thumbnail function needs to be given an Action and a Menu item (on the File menu).
• The thumbnail function needs to be documented in Help (and possibly in the tutorial).
• The remaining Help needs to be updated and rebuilt.
• The installer needs to be built and tested, especially on VISTA.
• The Website needs to be updated.
• The source code archive needs to be refreshed.

This could all be achieved in six or seven hours of work, so we could have this done by the end of the week.

Got the following changes done to the Image Markup Tool for version 6:

• Generated a new schema, removing my old custom code from the ODD file; the only customization is now the SVG element.
• Finalized the markup format for the <rendition> and <appInfo> elements.
• Updated the conversion XSLT files to convert old versions to the new version, and tested them.
• Rewrote the file i/o code in the application itself, to read and write the new formats.
• Updated the template file used to create a new document.
• Updated and tested the docbook import/export XSLT files. These are slightly complicated, because much markup has to be lost, especially from the <teiHeader>, but there's not much we can do about that. The real test will come when I edit the Help files in IMT.
• Cleaned out old schema files, odd files, etc.
• Updated all the tutorial examples and documentation, using this process to debug and tweak the conversion code.
• Updated the Web View XSLT code, and tested it.
• Updated the installer info.

There was little useful response to the issue of changes in <rendition> on the TEI list, so I'm going to have to make an executive decision to change the header encoding myself. This is what I think should be used:

<rendition xml:id="Transcription">
  <label>Transcription de texte</label>
  <code lang="text/css" rend="ellipse">color: #ff0000</code>
</rendition> 

Since I've already converted some files to an intermediate format, I've worked out a strategy for making the change:

1. Change the app so it writes out the new code format.
2. Load and re-save all of the files which have been changed so far (all three files in the tutorial folder are candidates, and some in test_data).
3. Rewrite the app loading code to read the new format.
4. Test file loading with the files previously saved as 1.6.
5. Update the conversion XSLT files to add the new rendition change.
6. Load a variety of known-good 1.5 files to check they load correctly, then save and reload them to ensure the 1.6 round-tripping is good.
7. Update the docbook conversion files so that they also use the new system, and test round-tripping some files to and from docbook.

Once docbook i/o is working, we need to start testing the help building system, which is based on mdh_docbook_to_html.xsl in the mdhHelp folder. Changes should not be required, but it should be tested.

Created the image for the menu/toolbar icon in Inkscape. However, when I tried to export it using my ScriptFu script in the GIMP to create all the BMP files, the script failed to run, presumably because of problems with the latest beta of the GIMP, which I'm running on my work machine. I'll have to try doing the operation at home, where I have older versions of the GIMP.

Created new XSLT for import/export of DocBook files in 1.6. The only change so far is the date/@value to @when change, but there will be more when the issue of rendition is sorted out. That's still being discussed on the TEI list.

Also made the change to @when in the Pascal object code.

Used my ODD file to generate a new schema from Roma, and compared it with the previous version. Apart from the date/@value attribute changing to @when, which I expected, another change has rendered <hi> tags inside <rendition> tags invalid in the header, which screws up my Image Markup code. Wrote to the TEI list about this; it seems like a silly change. I can't release another IMT until it's either undone, or if it's permanent, until I've found another way to encode that information and updated the tool to handle it.

However, I did write the four XSLT conversion files to transform older data files to the 1.6 format, and tested the last of them; that's out of the way. If the problem above is not fixed by changes in TEI, then I'll have to add to these files.

The thumbnail feature discussed in a previous post is an excellent idea, so I adapted the SaveScaledBitmap routine in mdhGraphics.pas to create SaveScaledBitmapRegion, which takes more parameters so that a region of the source image can be specified, as well as a target width. This works well in testing; now I need to create a GUI for it. I imagine this: "Create thumbnail -< from complete image", and "-< from selected area", then a dialog box where you can specify the output size. Then one of the two routines above is invoked, either the former or the latter respectively.

As suggested in the preceding post, it is possible to specify the TKernelResampler for the bitmap in the editing environment, meaning that scaling the bitmap is much better. It doesn't seem any slower either, on my machine, but we should test this on some slower machines to see how well it does with less horsepower to call on.

Now that we know TKernelResampler does a much better job than the default resampler, look at the possibility that the actual editing view, when resizing/scaling the image, could also use that resampler; it might be a bit slower, but it would be a whole lot clearer. Adding this as a task for November.

While working on new images for the Mariage site, I noticed that the quality of the IMT's "web view" scaled-down graphic was much inferior to the results obtained from scaling the graphic in PhotoShop. Greg and I started looking through the Graphics32 documentation, and discovered that the addition of one line of code could cause IMT to invoke a TKernelResampler when doing the downsampling; this is slightly slower, but produces a vastly improved result.

I've changed the version number to 1.6 for this, which means that I need to write a converter XSLT file; it also means that I need to generate a new schema from Roma, and check all the differences (I currently know of only one, the @value attribute having changed to @when on the <date> element, but there are bound to be others). This should be a priority now.

Greg also suggested a new "make a thumbnail" feature, which we kicked around till we came up with this:

If you click on Make Thumbnail when there's nothing selected, it defaults to selecting the whole image; otherwise, it works with the selection. A dialog box pops up which shows you the current Left, Top, Width and Height values, which are editable; changes are reflected in the actual selection. When you've tweaked those, you can specify the output size for your thumbnail; it can be proportional or not. Then you click OK, and get a file save dialog, then a thumbnail image is saved. This would be EXTREMELY handy for the Mariage site.

New releases of P5 required an update of the schema, which I've now done, generating the new schema from ROMA. ROMA is still a little flaky; it seems to delete information almost randomly from the project description (especially info inside tags), and it also displayed only the appInfo tag as a customized addition, rather than both appInfo and appDetail. However, when I generated the schema, it seemed to include appDetail, and files validate against the new schema.

IMT will need a new release soon, for this and the VISTA updates, but we'll wait until Transformer gets finished (there are a couple of tweaks to do on that), and release both on the same day.

Added and tweaked a couple of functions in GenFunctions.pas, then tested on our Vista test machine. These relate to detection of Windows Vista and OS version numbers. All our Delphi projects use these functions.

Built and tested the installer, updated a range of different files, fixed a couple of bugs and typos in licence docs and the tutorial, updated the Website, and released version 1.5.

I made some useful progress with the search functionality:

• All types of searches were returning multiple results, cascading through the hierarchy, and where one node contained multiple hits, it was returned once for each hit. Both of these problems are now solved; any node containing a hit is returned only once, and any exact phrase search returns only the bottom-level node containing the hit.
• The KWIC display was limited to the preceding and following sibling on an exist:match hit. This is no longer the case; it now takes the complete text of the parent, and creates a truncated intro and outtro to the hit.
• I've had a preliminary shot at thumbnails for hits on image markup annotations. I'm able to correctly retrieve the image and the dimensions, but for some reason my attempt to display only the hit area for the annotation is failing (I'm seeing a portion which is the right size, but is just the top left of the image). Nearly there, though!

Once the new icons were in place, I had to re-do all screenshots for the tutorial, help and example documents. Then I worked through the tutorial and help files, and updated them, adding information about the new features and the new schema. In the process, I removed the SVG schema from the schemas folder -- it's not needed for validation, because all its definitions are built into the main schema by ROMA. That simplifies things considerably. I also updated documentation to include more detail about the Nuvola icons.

On reflection, I don't think this is a big deal, so I'm changing it to abandoned. It's curious, but no show-stopper, and we're busy right now.

The annotation window seems to restart at a height of 613px, even if it was closed at a smaller height, and the state storage XML file shows the correct height. Perhaps it's set to a minimum somewhere, or perhaps the height isn't being reloaded correctly. Check this out.

Update: did a little more debugging on this, and the program is correctly reading the window height from the file, and attempting to set it; the result seems to be that if the height is greater than 613, it defaults to 613. There must be something which is forcing a minimum height of 613 on this window, but it's not in my code; there are no constraints in the DFM, and although there are min height constraints on some of the form components, they don't seem to add up to anything like 613...

Took the basic icons I'd created yesterday and worked on them till they look better. Built all the bitmaps, then replaced the originals in the dll, and rebuilt the dll.

For future reference, this is the best way to proceed:

• Start with a document 60px by 60px in Inkscape.
• Set grid units to px, and space the lines every 5 px.
• Leave one grid width empty around the drawing.
• Group objects carefully, ending with a final group containing everything.
• Build the bitmaps on OSX (its GIMP is faster and less buggy).
• Test the results of each bitmap build operation. Look out for misplaced text; if there is any, convert the text to paths.

Tested this lot, and found they're too crude and blocky. Built another set of vectors based around the Nuvola document image; I'll build the bitmaps from these and update the dll at home tomorrow.

Thickened up a couple of lines on the eight icons, and then added five of them into the dll, and rebuilt the dll. It now includes what we need for the IMT, so we can test the icons in context and see if they're acceptable.

Created Annotation, Annotation Clone, Annotation Delete, Annotation New, Annotation Add, Annotation Remove, Annotation List, and Annotation Window. Only five or six of these will be used, most likely.

A couple of problems were caused by the changes in the IMT 1.5 file format, and I fixed those easily. The rest are caused by the fact that, while previously we were using a simple "docbook_" prefix for all xml:id attributes in the DocBook output, that was changed in the reverse XSLT (producing the Docbook files) to be the xml:id of the document itself (derived from its filename), because our Help documentation works by XIncluding multiple files and all xml:id attributes need to be unique in the combined file.

This is logged as fixed, but I need to do some more rigorous testing to be sure, so I've left the task on outstanding till I've done that.

The icons built yesterday for annotations don't work at any size below 32 pixels; they're too complicated. A better approach would be to model them on the annotation icons in the Markin program (a yellow box with a black border and the letter A, with the top left corner folded down). This should be fairly easy to do in SVG. We probably don't need the Add icon -- we can use the simple Annotation icon.

It would also be helpful to find a square-ish speech bubble that can be used for comments and similar functions. Rounded bubbles render less effectively into small bitmaps because of anti-aliasing issues.