Professional Development

Posting time spent on the Pro-D expenses claim for my TESOL conference attendance.

Working through an example project.

Put an hour into some more QT background reading (on, among other things, QTs pointer wrappers, QActionGroup, QCache, and QNetworkAccessManager).

The library has a new repro stand (one of these). I went over for an orientation this morning.

From a post by LB, I learned about tree-tagger, which I've downloaded, installed, and briefly tested. It seems to work very well, producing POS-tagged text from input. I also grabbed LB's aftertag.pl PERL script, which is supposed to turn the tree-tagger output into good TEI.

Unfortunately, the PERL script doesn't want to run -- there are several errors:

syntax error at aftertag.pl line 7, near "<"
syntax error at aftertag.pl line 7, near ";) "
Unmatched ) in regex; marked by &lt;-- HERE in m/&lt;!--) &lt;-- HERE  { # tag found<br / at aftertag.pl line 10.

Figured that perhaps the HTML entities were the product of the publishing engine on LB's blog site, so replaced them with angle brackets in line 7. That left me with the line 10 error, which I still haven't figured out. I'll come back to this, because it will be really useful.

Continuing work on my pilot application, I've been trying various methods of getting a directory listing and displaying it with SVG icons representing the files, and checkboxes for selecting items in the list. After some testing with QDir and QListView, it seems that I want QDirModel and QTreeView, with a QAbstractItemView descendant to handle the display of rows. Meanwhile, I'm still struggling with the way C++ (or Qt?) uses pointers for objects. I need to do some methodical programmed study in parallel with my exploratory hacking -- hopefully there'll be time for that before Christmas.

I've sorted all my SVG icons and renamed the according to group_identifier-qualifier, as in for example search_find-again.svg or edit_select-all.svg. There are 179 of them. When I get a chance, I'll build them into a resource file in QT; I think that's just going to be the simplest way to use them. I can't get QT's resource editor to run today, for some reason, but it worked the other day, so I'll come back to this.

I've made a start on the first sub-application in preparation for IMT, which will be the SVG icon infrastructure (making SVG icons available to QActions through a convenient lookup system). This is not a bad thing to start with. I don't have anything working yet because I'm still struggling with the basics of C++, but I've made some early decisions and started to figure out how best to organize my code and files.

Finally figured out how to get a Windows executable compiled on one Windows machine to run correctly (with SVG icons) on another Windows VM without QT installed. Here are the steps:

• Build for release, and take the exe in the release folder of the project.
• Copy the following files from c:\Qt\[release_date]\qt\bin\ (NOT the \bin\ which is one folder further up the tree):
  • mingwm10.dll
  • QtCore4.dll
  • QtGui4.dll
  • QtSvg4.dll
  into the same folder as the executable. (Make sure you don't get the versions with "d" included; these are the debug versions.) You may need other dlls depending on what you're using.
• Take the \plugins\imageformats folder from the same location, and copy it into a \plugins\ folder alongside the executable. (This is all that's needed for my simple SVG test app; you may need other plugins too.)
• Create a text file called qt.conf in the same folder as the executable, and put this in it:

  [Paths]
  Plugins = plugins/
  

  This tells the QT code to look for plugins in this subfolder of the main app. Otherwise it will look for a full QT installation on the target machine, and not find it. Plugins seem to be late-binding, and the app is able to fail gracefully (by not displaying SVG icons) if it can't find the plugins.

This works for an exe compiled on 64-bit Windows 7 or on 32-bit Windows XP, running on a separate 32-bit XP system.

I now have a clean XP VM provided by Greg, with only QT on it, for this kind of task. Now I need to see if I can make a Linux app portable in the same way; then it's Mac time.

Today I was looking at QStringList and QMessageBox, and also taking a preliminary look at the XML stuff. I've also been fighting with the deployment issues again. The situation is that the application compiled on 64-bit Win7 will run on 32-bit XP, but the SVG icons don't work. I've tried every permutation I can think of for the deployment dlls, but nothing works; although the app gives no error. My plan now is to create another simple XP 32 VM, install QT on there, and compile the app there; perhaps that'll give a working executable on a similar system with no QT installed. The SVG icons are going to be important to us, so I'd like to get this sorted ASAP. We should also try the same thing on a Mac, to see what dependency issues we need to prepare for there.

Tried using one of Greg's stripped-down VMs for this, but after doing Windows Updates to get it up to speed, it would no longer boot. Curses.

This week I've set aside afternoons for learning some more about QT, and answering some of the key questions we set ourselves. I built a little GUI app today, and confirmed that SVG images can be used for icons; I also cross-compiled it on Windows 7, and it works there too. Testing the app on XP, though, I ran into the same dependency issues as before; with a suitable collection of DLLs I could get it to run, but I couldn't get any SVG images to show up. I think deployment on Windows is going to be a rather difficult issue; we need to get it solved before we go any further. However, I'm hopeful that we can have a fully SVG-based GUI, which is great.

ØE came by on an arranged visit to talk about many things, including Cocoon/eXist, which they're planning to use for a project soon, and the idea of an annotation server, something we're also planning for the PKP project. We talked about a framework whereby the concerns would be well separated, with the document publication engine and the annotation system living on separate servers and communicating over a stateless API, and determined that most likely, you would want the annotation server to pull content from the publication server and integrate it into its own pages. We will collaborate on a schema fragment for storage and/or transmission of annotations, and on the nature of the APIs on each side.

Got the QT IDE installed on Windows, and built an application, in order to find out about dependencies, redistribution etc. It seems that you basically have to redistribute three core dlls, along with (possibly) some MS VC++ dlls, in order for apps to function, but you're allowed to redistribute the QT dlls; as far as the MS VC++ dll(s) are concerned, we'd have to see under what circumstances they're needed, where they're likely to be missing from a system, and whether it's practical to redistribute them. QT seems to work OK on Windows 7, although it's not supported formally on that platform.

Several things emerged from my brief trawl through examples and help today:

• The default text editing component is rudimentary, but a WebKit browser control can be used to provide sophisticated editing.
• QT has a similar system of Actions associated with widgets as Delphi does.
• The QT QIcon object seems to handle SVG, and will handle scaling automatically if necessary (at least, scaling downwards), as well as the generation of disabled variants. This promises to make action icons simpler than they are in Delphi.

Yesterday, I'd tried and failed to build the Phonon Music Player example project; the error was "cannot find lphonon". A quick Google led to my creating a symbolic link from a phonon library in /usr/lib; I had libphonon.so.4, but it was presumably looking for libphonon.so, so I did sudo ln -s /usr/lib/libphonon.so.4 /usr/lib/libphonon.so, and was then able to build the project and listen to audio files successfully.

At home, I got QT creator working with no problem at all, but installing on the work machine had a couple of glitches; first, I'd forgotten which of a list of libraries I'd installed prior to installing QT, so I sent myself that list from home; then when starting the app, it thought the examples were not installed (couldn't solve this, but I can still open all the examples manually, so no worries); and finally, trying to compile an app resulted in an error stating that no default QT version was set, so I had to go into the Tools / Options dialog, add a new QT version, and browse to the directory /home/mholmes/qtsdk-2009.03/qt, which it identified as QT 4.5.2. After setting this as the default version, example apps compile and run OK.

After spending last week in DH's excellent workshop on text-analysis, I bought a copy of Minitab to use on a text-analysis project related to the Colonial Despatches, which will probably result in a paper. Took a while to get an account set up on the retailer's site, demonstrate my educational status, make the purchase, then fill in and submit the Pro-D cheque req form, but I now have the software working on my Windows XP VM.

Worked through examples of compiling dictionaries based on the del.icio.us API and building recommendation functions from them. So far so good...

Started working through the examples in Collective Intelligence, learning Python syntax as I go, and trying to figure out similarity metrics. The math is beyond me, especially when it comes to the more complicated metrics like Jaccard Coefficients or Tanimoto Scores, but given that I can disentangle the arithmetic of the algorithms, I don't really need to understand why they do what they do. I might work on a Delphi application in parallel to the Python, so that I'm forced to re-code the examples in a way that makes me understand them.

Looking at how best to script Inkscape to rasterize svg source files.

Inkscape can be scripted via Python, but for the project at hand this might be more than we need. I suspect that we can write a much simpler bash script that will do everything by calling inkscape from the CLI and using built-in functions. A bit of reading here: showed me that I can do something like this:
/Applications/Inkscape.app/Contents/Resources/bin/inkscape -D -e ~/Desktop/11.png --export-area-canvas ~/Desktop/11.svg and produce a png that retains the canvas bounding box. If I drop the --export-area-canvas bit I lose the bounding box and have a zero-margin png. I can adjust the width and height with -w and -h in the usual way: pass it a width and let Inkscape do the math to arrive at a suitable height.

Took delivery of a camcorder. Did some testing. Most pleased with results.

Shoots 1920x1080, comes with dock and much cabling. Looks like it isn't really happy with panning - probably a result of running it at 60 fields/sec. Running it at full res, but 30 frames/sec seems to be better - not perfect, but much better. Martin suggested that it's the CCD that can't keep up. Quite possible, given the price of the camera; CCDs are expensive, the camera wasn't.

Full res panning/zooming - in fact any kind of movement - creates very jittery results. I found several sets of plans online for building your own steadycam, but I like these best. I'll try to build one at home. If it works out I'll bill for hours/materials.

Editing the video couldn't be easier. The camera produces H.264 video in an mpg container that just works. Mount the device as a volume on your machine, browse volume to the file you want, drag it to desktop, import/move/copy file in to Final Cut/iMovie and you're ready to edit. Haven't tried it on Linux/Windows yet, though I expect it to be just as easy - as long as I can find an editor for those platforms that handles MPEG-4 natively.

I'm looking at the possibility of moving away from Delphi over to Trolltech's QT platform for desktop application development, so I've spent while looking into their documentation and licensing. While it seems QT is absolutely everything we need (built-in Unicode, scripting, great widgets, cross-platform, WebKit and ECMAScript integration, SVG support, etc.), there are some serious drawbacks. One is that I'd need to learn C++, but actually that might be interesting and a good investment from the Pro-D point of view. The other drawback is that, while you can use QT for open-source development under the GPL, and you're allowed to MPL your code if you want, you can't (they say) then move that code to commercial projects; and the commercial licensing is flat-out prohibitive ($6600 US for one developer working on the three platforms). That would pretty much preclude our commercializing one of our HCMC projects (as we previously did with HotPot and Quandary), which would be a real shame.

So I'm still thinking. The platform looks SO inviting, but the price is almost insane. They obviously don't want small-scale or low-profit developers (such as the educational software crowd) to consider the platform at all as a commercial proposition.

The next version of Firefox, currently in Alpha 2, has some fabulous new features, including integrated video support (HTML 5) and presumably audio as well. We need to get working with this to do some bug reporting and find out how it can help us avoid (e.g.) Flash in future. I've got it installed on a VM, because it's pretty unstable, and doing that involved updating the VM twice, but once that's all squared away we can start creating some test materials to find out what it can actually do. (For instance, does it have some codecs built in, and if so, which are they? Can it use any codecs on your machine?)

Installed the new NetBeans 6.0 and tested my current projects with it. Seems fine -- several improvements over 5.5.1.

In a number of my applications, mainly IMT and Markin, I've been building a large library of interface icons based on the Nuvola GUI icons. All of them are in SVG format, and for Delphi use, I convert each one into 12 different BMP files, for three states (normal, hot, disabled) and four sizes (16, 24, 32, 48). These would be very useful for our Java GUI apps, but they need to be in transparent PNG formats, so I began investigating how best to do the conversions. I do my Delphi conversions using a script I wrote in the GIMP, one image at a time. However, for the entire library, which is now 145 images, that would be very tedious, and it would help to have a conversion tool which could later be adapted to provide other formats where required.

I started by learning that Inkscape has basic command-line functionality which enables me to load an SVG file and convert it to another size. I decided to call this functionality from a simple Delphi app, to give me more flexibility than batch files will provide. Having got the Delphi app to create PNGs at all four sizes, the next stage was to figure out how to create the disabled and hot images. At first, I thought ImageMagick would be the solution to this. I discovered that IM can do the lightening needed to produce a "hot" image like this:

convert  minor.jpg   -sigmoidal-contrast 4,0%   minor_04.jpg

Next, I began to look for ways to produce the grayscale disabled images. IM can do this, too:

convert  test.png  -modulate 100,0  grey_brightness.png

These examples come from this excellent site.

So ImageMagick looks like the best tool to do the conversions. Initially, I decided it would be simplest to make copies of all the existing images in the hot and disabled folders, and then process those in-place. After some hacking around, I found I could only get Delphi to do that by using XCOPY instead of COPY. That's now working. I'm now reconsidering that approach, though, and I think it might be simpler to go through each of the images in each of the size folders, and create output hot and disabled images in their appropriate folders. using ImageMagick. That's for anotehr day -- ran out of time today.

Once this is done, I'll need to look at ways to assign images to actions in Java apps in an efficient and centralized way, as I'm able to do in Delphi, and figure out how Java might be able to use disabled and hot images as well as normal ones.

Revisiting my Apparatus app, I hooked it into the preferences object, and tried adding support for JFileChooser. This proved more complicated than I expected, because the JFileChoosers are usually created on the fly, so they're not owned by any other component in the hierarchy; they're just instance variables in the JFrame class. We had to use some reflection code to iterate through all the variables in the class to find any which are JFileChoosers. In order to make this work, they have to be public rather than private, but that's not a big deal.

As part of the teiJournal project, I needed to write a simple text-transformer that would turn CSS rulesets into XSLT attribute-sets (documented on that blog). I decided to do this in Java to get more practice with basic apps, and take advantage of the opportunity to abstract more of the code I learned writing the Apparatus app into general-purpose utility classes.

I created a class called MdhPrefs in ca.uvic.hcmc.utils. The purpose of this class is to save and reload a range of key info for a JFrame descendant, so that the JFrame can initialize an instance of the object when it starts up (after initComponents() is called) and get preferences for a wide range of settings loaded automatically.

How to use it:

1. Create a private variable in the JFrame descendant class:

   private MdhPrefs mdhPrefs;

2. Initialize it after initComponents() is called in the class's constructor:

   mdhPrefs = new MdhPrefs(this);

3. In the JFrame descendant's formWindowClosing event, make a call to save the data:

   mdhPrefs.saveData();

At the moment, this class saves only window size and position, and the position of any splitter bars, but it will be extended to include a lot more information. The load/save code is recursive, so it should catch all components which are descendants of the JFrame.

This kind of reflection looks to be very powerful; it's similar to the code I've written in the past for Delphi, but (so far) a bit cleaner, because the component hierarchies seem more straightforward. A set of classes for translating the interface could be created in the same way.

I abstracted generally-useful routines such as file loading/saving and resource loading into a new package called ca.uvic.hcmc.utils.

Also, with Stew, took a close look at JGoodies, and ran some of the demos. You could write a seriously attractive GUI with this stuff, and it's BSD-licensed in a very open manner so it could be used in commercial code too.

After writing a little application in NetBeans to process Devonshire MS collation data into TEI format, I'm continuing my Java education by learning a few more things that aren't strictly relevant to that project. With Cocoon, eXist and Juxta written in Java, it makes sense to develop skills in this area, and with Java 1.6 and the latest NetBeans IDE, the tools are now mature and the platform is pretty solid.

Today I was learning how to manipulate resources such as images as part of the application. In the IDE, and when running the app in its debugger, there seems to be no actual JAR file, so images can only be loaded from the file system; however, when you build the JAR, any images are bundled into the JAR, and then you need to use different code to get at them. In order to have an app that runs correctly in the IDE and as a JAR file, therefore, you need to allow for both contingencies, so I have code that looks like this:

File iFile = new File("images/smiley.png");
        if (iFile.exists()){
            setIconImage(new ImageIcon("images/smiley.png").getImage());
        }
        else{
            setIconImage(new ImageIcon(getClass().getResource("images/smiley.png")).getImage());
        }

to set the icon for the application's main form.

I've also been researching the possibility of using SVG images instead of regular bitmaps as application widget icons (on toolbars etc.). This project makes use of Batik to render SVG images in a scalable fashion in a Java UI. This is really worth looking at. Right now, my Windows apps use 12 different versions of each glyph image to allow four sizes and three states; this system might allow me to use just one image, and let the user scale it exactly how they like.