Image Markup Tool blog

I have a pilot thing working which parses an XML file, displays it in a syntax-highlighted editor, and simultaneously displays its DOM tree in an expanding tree widget and a list of its xml:ids (with the first 30 characters of the element's text) in a list view. This is pretty good, but I know in my heart that what I really need is a model/view architecture whereby the QDomDocument is the model, and these three views are all synchronized so that when you click on (say) an xml:id in the id list, the dom tree opens and highlights the corresponding element, and the editor scrolls to, and highlights, the same element. QT's model/view stuff hurteth my brane mightily, but I managed to understand it for long enough to write the little SVG resource applet, so I'm sure I can do it again.

My previous little demo app came up against a huge roadblock when any reference to a QTreeWidget would cause an application crash with a segmentation fault. As far as I can see, this was not due to anything in my code, so eventually I went back to the drawing board and rewrote the app using the UI designer in QT Creator. This is now working (parsing an XML string into a DOM tree and displaying it in a QTreeWidget). Something in the process of creating the QTreeWidget in the UI class seems to protect against the seg fault.

I'm now working on having the app build an index of xml:id attributes, from a persistent DOM model, and figure out how easy it is to keep the index and the DOM model aligned.

I've set up a simple project for learning about DOM functionality in QT. I haven't really got very far yet, but I have learned that by default, the QXmlDocument object ignores nodes consisting only of whitespace (obviously not written by humanists), and that some kind of workaround is required when parsing an XML stream. That's the only unpromising thing so far...

MJ came by today and we discussed the framework of the application, and made some preliminary decisions. This is my attempt to capture what ended up on the whiteboard.

The text side of the application (the TEI <text> element, in other words), will probably be handled as a DOM structure. It doesn't make much sense to try to model it in any other way, since it is directly editable (in that any element along with its children can be called up and altered by the user). We will need to keep an updated index of all the links between elements and <zone>s, though; each link would consist of a source element's @xml:id, the target zone's @xml:id, and the category of the link. It appears that categories will be applied, not to <div>s or <zone>s as before, but to links. This is necessary because the same <zone> may be linked from multiple elements for multiple purposes, and the same element may be linked to multiple <zone>s for multiple purposes.

In the interests of simplicity, I think it will make sense to write out the index of links into the TEI file directly. This can be done in categories, something like this:

<linkGrp type="myCategory">
  <link targets="#elementA #zoneB" />
  <link targets="#elementX #zoneY" />
</linkGrp>

I'm not sure yet where we should be storing the <linkGrp> elements; the header is the natural place, but none of its components seems to provide a good home for them.

These sets of elements can be rapidly read by the app when loading an existing file. However, for completeness, when saving, we should also supply the appropriate @facs and @corresp attributes on elements linking them to <zone>s, so that a processor not aware of the linkGrp system can still parse the relationships appropriately (although it would be unaware of the category information).

The index of all these relationships needs to be able to update itself in response to changes in the DOM, and it also needs to provide a brokering role enabling the GUI to display the right <zone> when an element is selected, or the right element when a <zone> is selected. In both cases, when it knows that there are multiple possible candidates, it should pop up a selection menu of some kind. It will also need to be able to display a list of every element which is a descendant of <text> and has an @xml:id, in the form of its <@xml:id> followed by some characters of its text content, so users can identify what they want to link to.

On the other side of the application, this is the basic model. (After our discussion, I did some more reading and realized we had been wrong about one Qt relationship, that between QGraphicsView and QGraphicsScene, so this description re-works that relationship.)

• At the top is HcmcFacsimile, a container for all the graphical constituents of the model. This maps to a TEI <facsimile> element.
• HcmcFacsimile contains a QVector<HcmcSurface>, which manages all the surfaces in the project.
• HcmcSurface maps to the TEI <surface> element. It descends from QGraphicsScene. It contains one HcmcPixmapItem, a descendant of QGraphicsPixmapItem; this corresponds to a TEI <graphic> element. HcmcSurface also contains a QVector<HcmcZone>. It needs its own unique @xml:id.
• Each HcmcZone corresponds to a TEI <zone> element, and descends from QGraphicsPolygonItem. This is the most challenging class to write, because it needs three states (invisible, view, and edit). It will also need a unique <@xml:id>, which the indexing broker will have to be aware of.
• HcmcSurface will render itself through a QGraphicsView, tied to a panel in a tabbed panel control.

Just spent an hour trying to get an app to compile using my MdhXmlEditor class. Although I was declaring the header file for the class in the header file for the main window, it couldn't find the class. Eventually I discovered that I had to manually add the header and cpp files for the two libraries I'm using to the QT .pro file, like this:

# -------------------------------------------------
# Project created by QtCreator 2010-03-10T14:41:22
# -------------------------------------------------
QT += xml \
    xmlpatterns
TARGET = createImtFiles
TEMPLATE = app
SOURCES += main.cpp \
    mainwindow.cpp \
    ../imagemarkuptool/trunk/projXmlEditor/mdhxmleditor.cpp \
    ../imagemarkuptool/trunk/projXmlEditor/mdhxmlhighlighter.cpp
HEADERS += mainwindow.h \
    ../imagemarkuptool/trunk/projXmlEditor/mdhxmleditor.h \
    ../imagemarkuptool/trunk/projXmlEditor/mdhxmlhighlighter.h
FORMS += 

There's no equivalent of Delphi's "Add file to project", but obviously the pre-processor needs to know about this stuff; it can't rely on the compiler to find all the header files.

Did some reading and thinking, to come to this conclusion:

• Each image and its annotation areas should be handled using an MdhSurface class.
• The display of the image and zones should be handled using a QGraphicsScene descendant, which is a member of MdhSurface.
• The image might be rendered as a background using QBrush, or it might be a QGraphicsPixmap item (item 0 in the array held by QGraphicsScene).
• The zones should be handled either by QGraphicsRectItem descendants (if we handle only rectangles), or QGraphicsPolygonItem descendants (if we handle polygons -- see below).
• The actual view should be handled by a QGraphicsView. I'm not sure yet whether that should be part of MdhSurface, or external to it; and I'm not sure at what level we should create an actual widget to be instantiated on a panel. Probably the widget would be MdhSurface, and would have the QGraphicsView as a member, but I'm still fuzzy on that.

I've also figured out a way of encoding polygonal zones in P5. Here's a brief description I sent to the TEI list for some feedback:

The current spec for <facsimile> and its children allows only rectangular zones on a surface:

<surface>
  <zone ulx="10" uly="10" lrx="50" lry="50"></zone>
  <zone ulx="60" uly="100" lrx="90" lry="200"></zone>
</surface>

specifying two rectangular zones on the surface.

Digging around a bit, I noticed that <zone> can contain <desc>, <desc> can contain <dim>. <dim> "contains any single measurement forming part of a dimensional specification of some sort." So I'm thinking that I can do polygons like this:

<surface>
  <zone ulx="10" uly="10" lrx="50" lry="50">
    <desc>
      <dim type="point">10,10</dim>
      <dim type="point">30,30</dim>
      <dim type="point">50,50</dim>
    </desc>
  </zone>
</surface>

The standard attributes on <zone> define the bounding rectangle for the polygon (enabling processing by any agent that only handles rectangles), while the points of the polygon (triangle in this case) are defined in the <dim> elements for IMT and any suitably-aware processor to handle.

Created a more informative error message from the QXmlStreamReader error feedback for when well-formedness checking fails. Also replaced the red-cross image used to indicate the line where an error occurs with a capital X drawn in red. This is simpler, and removes the dependency on an image. For cross-platform compiles, the executable ends up in different places, so it's convenient to avoid this kind of dependency until we absolutely need it.

Today I finished the routines that get and set character formats, so the user will be able to customize the syntax highlighting. I've also implemented well-formedness checking using the QXmlStreamReader component, and I'm able to put a little X on the line where the check fails. I have a couple more bits and pieces to do, and after that I'll clean up the code and add more documentation. Then it's on to the image rendering stuff.

Refactored all the xml editor code to make it a standalone component, and renamed the classes with prefix Mdh. Started on the process of allowing write access to the highlighter character formats, so the user could set these. It should work pretty well, but I need to find a way to enforce the use of values from an enum in a method parameter.

Adapted some code from KDE to get line numbering working, in an editor that now descends from QFrame. Before I go any further, I really need to separate out all the editor code from the test project code, and build a proper set of methods for it, so you can set the font and character attributes of the editor and of all the special character formats used for highlighting. Then I can move on to well-formedness checking.