I’ve started using Maven to manage my java projects, and overall I’m very happy with it. It seems to be more mature than ivy, with better documentation, and the vast majority of tasks that I need “just work” (just don’t ask me about jni–that’s another post).

Today, (and yesterday, and a good portion of the night in-between) I ran into a nasty bug in a library that I didn’t know my code depended on. It isn’t particularly important what I was working on, but just for context: I needed to strip a lot of text content out of nodes in the complete wikipedia revision history dump, so I was using Sax to parse the xml stream, filter out the stuff I wanted filtered out, and save the stuff that, well, I wanted saved. Being that the input was all of wikipedia, there were a fair number of unicode characters in there. As it turns out, the 2.6.2 xercesImpl has some sort of bug that allows xml with certain characters to be read without throwing exceptions, but when you try to write the chars that were actually read, you end up trying to write characters that aren’t valid in xml. Even if I’d known that in advance, my response would have been something like “ok, so what? I’m not using xercesImpl, and certainly not a version that old”.

Well.

You see, in addition to using Maven, I’ve also been using the Google Collections and JSR305 libraries, so I just drop those <dependency> entries into the pom for all my new projects–I just assume that I’ll need them, and I usually do.

Unfortunately, JSR305 1.3.8 depends on jaxen 1.1.1, which depends on xercesImpl 2.6.2 (jaxen also needs this dependency via xom 1.0, for what that’s worth).

Because that dependency was already present in my build path (via mvn eclipse:eclipse) and in the generated jar (via <addClasspath> and <classpathPrefix> in the maven-jar-plugin configuration section), I never realized that my sax code actually had a direct dependency on xerces as well. This all came to a head when, 3.53gb into my 2.8tb run, these rather unhelpful exceptions started popping up:

[cc lang=”bash”] java.io.IOException: The character ‘?’ is an invalid XML character

   at org.apache.xml.serialize.BaseMarkupSerializer.characters(Unknown

Source)

   at com.stottlerhenke.tools.wikiparse.ContentStripper.characters(ContentStripper.java:195)
   at org.apache.xerces.parsers.AbstractSAXParser.characters(Unknown

Source)

   at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown

Source)

   at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown

Source)

   at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
   at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
   at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
   at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
   at com.stottlerhenke.tools.wikiparse.ContentStripper.parse(ContentStripper.java:96)
   at com.stottlerhenke.tools.wikiparse.ContentStripper.main(ContentStripper.java:379)

[/cc]

<rant> “?” is not unicode – it fits just fine in asci tables everywhere – so please don’t tell me that it’s an invalid unicode character :) (0xd800 is an invalid unicode character, and that would have been much more helpful) </rant>

Many hours later I was able to find a sample of the actual input that was causing these problems, and I was able to reproduce the issue with an input slightly smaller than 2.8tb. Once that was done, I set out to make a minimal test case. Rather than bother with a new maven project, I just hacked it out in emacs (not using google collections, etc. because, clearly, I wanted it minimal). To my surprise, everything worked, and worked fantastically! But how? I didn’t even supply an xml api on the classpath, yet it ran just fine!

In truth, I did supply an xml api – xercesImpl.jar, and many other libraries – via my environment’s $CLASSPATH. (Figuring that out was another adventure, but I digress.) Once it became clear that I was indeed using a broken library it was simply a matter of explicitly specifying the dependency on a new version of xercesImpl, and rebuilding.

The moral?

Know your dependencies! This should come along with knowing your language’s built-in APIs well. It wasn’t clear to me that the SAX packages I was using were not part of the core java API, so it didn’t strike me as odd that I didn’t need to specify a classpath entry or a pom dependency before I could use sax.

If you suspect something strange, you can see the dependency tree in the generated html documentation you get when running mvn site.

I just upgraded my workstation to Jaunty (Ubuntu 9.04) and the key repeat delay and speed dropped to a frustrating level.

gnome-control-center can be used to fix this, but it requires that the gnome-settings-daemon be running, which forces it’s opinions on many other aspects of my environment (I run Enlightenment dr17).

Poking around a bit, and help from #e on freenode, revealed that xset can be used to fix the key repeat settings.

[cc lang=”bash”]

Look at the current settings:

$ xset q Keyboard Control: auto repeat: on key click percent: 0 LED mask: 00000000 auto repeat delay: 660 repeat rate: 25 auto repeating keys: 00ffffffdffffbbf

                    fadfffefffedffff
                    9fffffffffffffff
                    fff7ffffffffffff

lets speed things up a bit…

$ xset r rate 250 40 [/cc]

I’ve had some minor upgrade issues with the blog lately, and I am only about halfway through updating everything. In the meantime, I’m afraid things will look a bit messy. (Syntax highlighting is currently broken, and there are probably other formatting / data issues as well. I think I have to restore the uploads directory, for one, so there probably won’t be any images in the posts for a while.)

I started writing for another blog this week:

Brewed, Bottled, Cultured and Sweetened is a blog about beer, coffee, wine, cheese, chocolate, etc… that I’m writing with an old friend from Dagit.o

Update: updated the script with improved (functional) error output. added notes about xhost.

There is an annoying bug in the sequence of code that manages the wacom rotation / sleep / resume and stylus calibration right now. (Where “right now” is Ubuntu Intrepid, with the 0.8.2-1 wacom drivers.)

This is a document bug over at the ubuntu launchpad, and the poster there does a fine job of describing the intricacies of reproducing the bug, so I’ll only give a brief explanation here to help get indexed.

If you rotate the screen any amount, even returning to the original rotation, and then sleep the machine, when it wakes up, the stylus will not be calibrated properly – the cursor will be off to the side of the stylus point. It doesn’t seem to matter how it was calibrated when the machine slept, nor does it matter what rotation you’re in when you put the machine to sleep.

There is one straightforward workaround: When you wake the machine, run wacomcpl, click on stylus, click calibrate (the mouse should now be under the stylus again), and exit wacomcpl. This is incredibly cumbersome, but at least it’s better than restarting X, which is what I have been doing.

Further inspection (based largely on the thread of comments on that launchpad bug) reveals that the problem is actually related to bad values for the TopX, TopY, BottomX and BottomY settings on the wacom devices after a resume. By resetting these to their proper values for the current rotation, we can reestablish the proper calibration. First off, we need to know the proper values, and the easiest way to get them is with xsetwacom:

[cc lang=”bash”]

!/bin/bash

wacomSettings

echo “TopX=” xsetwacom get stylus TopX echo “TopY=” xsetwacom get stylus TopY echo “BottomX=” xsetwacom get stylus BottomX echo “BottomY=” xsetwacom get stylus BottomY [/cc]

Now, we’ll run this for each rotation, and save the results. You should end up with something like the following:

[cc lang=”bash”] |rogue on bach |AC 70% | @ 00:02:26 ~| $ xrotate 1 && wacomSettings xrandr to left, xsetwacom to 2 TopX= -46 TopY= -3 BottomX= 18605 BottomY= 24518 |rogue on bach |AC 70% | @ 00:02:28 ~| $ xrotate 2 && wacomSettings xrandr to inverted, xsetwacom to 3 TopX= 58 TopY= -46 BottomX= 24579 BottomY= 18605 |rogue on bach |AC 70% | @ 00:02:35 ~| $ xrotate 3 && wacomSettings xrandr to right, xsetwacom to 1 TopX= -173 TopY= 58 BottomX= 18478 BottomY= 24579 |rogue on bach |AC 70% | @ 00:02:41 ~| $ xrotate 0 && wacomSettings xrandr to normal, xsetwacom to 0 TopX= -3 TopY= -173 BottomX= 24518 BottomY= 18478 [/cc]

(Note that my bash prompt looks like & command lines above are indented, and the output is left-aligned)

That gives us enough information to script the calibration when we resume. For example, when resuming to a “normal” rotation, I need to run:

[cc lang=”bash”] xsetwacom set stylus TopX -3 xsetwacom set stylus TopY -173 xsetwacom set stylus BottomX 24518 xsetwacom set stylus BottomY 18478 [/cc] (Wrap that in a bash script and give it a shot!)

Here’s the full script that gets the current orientation and then calibrates the common wacom devices:

[cc lang=”bash”]

!/bin/bash

#

waCalibrate.sh: recalibrates the wacom stylus

#

Author: Rogan Creswick

License: just be nice

Set LOG to something reasonable:

(The file does not need to exist, but the directory does)

LOG=/home/rogue/calibration.out XSETWACOM=/usr/local/bin/xsetwacom

#

Calibrates the wacom devices {stylus, eraser, cursor} with the

given offsets:

#

Usage:

calibrate

# function calibrate {

${XSETWACOM} --display :0.0 set stylus TopX $1 >> ${LOG} 2>&1
${XSETWACOM} --display :0.0 set stylus TopY $2 >> ${LOG} 2>&1
${XSETWACOM} --display :0.0 set stylus BottomX $3 >> ${LOG} 2>&1
${XSETWACOM} --display :0.0 set stylus BottomY $4 >> ${LOG} 2>&1

${XSETWACOM} --display :0.0 set eraser TopX $1 >> ${LOG} 2>&1
${XSETWACOM} --display :0.0 set eraser TopY $2 >> ${LOG} 2>&1
${XSETWACOM} --display :0.0 set eraser BottomX $3 >> ${LOG} 2>&1
${XSETWACOM} --display :0.0 set eraser BottomY $4 >> ${LOG} 2>&1

${XSETWACOM} --display :0.0 set cursor TopX $1 >> ${LOG} 2>&1
${XSETWACOM} --display :0.0 set cursor TopY $2 >> ${LOG} 2>&1
${XSETWACOM} --display :0.0 set cursor BottomX $3 >> ${LOG} 2>&1
${XSETWACOM} --display :0.0 set cursor BottomY $4 >> ${LOG} 2>&1

}

function fixCalibration {

# get the current orientation:
ORIENTATION=`xrandr --verbose --query | grep " connected" | awk '{print $5}'`
echo "Orientation: ${ORIENTATION}" >> ${LOG}

case "${ORIENTATION}" in
normal)
    calibrate -3 -173 24518 18478   
    ;;
left)
    calibrate -46 -3 18605 24518
    ;;
right)
    calibrate -173 58 18478 24579
    ;;
inverted)
    calibrate 58 -46 24579 18605
    ;;
*)
    calibrate -3 -173 24518 18478
    echo "ERROR!! unknown orientation! ${ORIENTATION}" >> ${LOG}
    ;;
esac

}

case “$1” in

resume|thaw)
date >> ${LOG}
fixCalibration 
whoami >> ${LOG} 
    ;;
*)
echo "not a resum|thaw event: $1" >> ${LOG}
    ;;

esac [/cc]

Stick that in /etc/pm/sleep.d/40wacomCalibrate (or some similarly named file), make it executable by all (chmod a+x /etc/pm/sleep.d/40wacomCalibrate) and it should be run when the system resumes.

Update: I found that the logging of the old script didn’t work, so I’ve updated the script to reflect that. There were also some problems with how I was testing the first script, and the actions I was taking didn’t actually trigger the bug. (The bug seems to be quite state-dependent, and markovian assumption was wrong.) To get this to work, root will need to have access to the display that xsetwacom uses. The simplest way to do this is to add xhost + to you x startup. (I put it in my ~/.xsession just before exec enlightenment-start).

Ubuntu 8.10 configured most everything properly, as mentioned in the previous post in this series, but it did not result in a functional pen.

The tablet screen is a wacom digitizer with a pen that has two buttons (eraser and a finger button), and the tablet can differentiate between touching and hovering. The linux wacom driver & tools are necessary to get this all working. While I didn’t find a single page with instructions that worked flawlessly, I was able to figure it out from a collection of links:

• The Linux Wacom project:
  • http://linuxwacom.sourceforge.net/index.php/minihowto
  • http://linuxwacom.sourceforge.net/index.php/howto/x11
• The Aliencam blog:
  • http://blog.aliencam.net/articles/aliencams-customized-ubuntu-setup-guide/

First off, you will need the latest version of the linux Wacom driver (8.2.1 at the time of this writing). The driver versions seem to be tied to your kernel versions, so this is quite important. The wacom-tools package that comes with Ubuntu is not sufficient (in fact, you’ll want to uninstall it if you have it already).

Once you have the wacom package downloaded, follow the directions for installing it in the howto (linked above). The wacom package uses a typical configure, make, make install process but there are a few kinks:

• configure (almost?) always succeeds, regardless of the dependencies you have yet to fill. The make step will simply not build all the things you need if this happens, but it won’t fail visibly.
• You’ll need to copy the kernel module into the /lib/modules/uname -r/kernel/drivers/usb/input/ directory manually (creating subdirs if necessary), before running make install. (This is outlined in the mini-howto.)

Once wacom is installed, you can begin working with the X.org configuration. This is fairly clearly explained at the aliencam blog linked above, or you can use my xorg.conf here.

[cc lang=”bash”] Section “Device”

Identifier  "Configured Video Device"

EndSection

Section “Monitor”

Identifier  "Configured Monitor"

EndSection

Section “Screen”

Identifier  "Default Screen"
Monitor     "Configured Monitor"
Device      "Configured Video Device"

EndSection

BEGIN TABLET SECTION

Section “InputDevice”

Driver      "wacom"
Identifier  "stylus"
Option      "Device"    "/dev/ttyS0"    # serial ONLY
Option      "Type"      "stylus"
Option      "ForceDevice"   "ISDV4"     # Tablet PC ONLY
Option      "Button2"   "3"

EndSection

Section “InputDevice”

Driver      "wacom"
Identifier  "eraser"
Option      "Device"    "/dev/ttyS0"   # serial ONLY
Option      "Type"          "eraser"
Option      "ForceDevice"   "ISDV4"     # Tablet PC ONLY
Option      "Button3"   "2"

EndSection

Section “InputDevice”

Driver        "wacom"
Identifier    "cursor"
Option        "Device"      "/dev/ttyS0"    # serial ONLY
Option        "Type"        "cursor"
Option        "ForceDevice" "ISDV4"     # Tablet PC ONLY

Option “Mode” “Absolute”

EndSection

This section is for the TabletPC that supports touch

Section “InputDevice”

Driver “wacom”

Identifier “touch”

Option “Device” “/dev/input/wacom” # USB ONLY

Option “Type” “touch”

Option “ForceDevice” “ISDV4” # Tablet PC ONLY

Option “USB” “on” # USB ONLY

EndSection

END TABLET SECTION

Section “ServerLayout”

Identifier  "Default Layout"
Screen      "Default Screen"

InputDevice “Synaptics Touchpad”

added to get tablet working

InputDevice     "stylus"    "SendCoreEvents"
InputDevice     "cursor"    "SendCoreEvents"
InputDevice     "eraser"    "SendCoreEvents"

InputDevice “touch” “SendCoreEvents”

EndSection [/cc]

After doing that, you should be able to reboot and the pen should be working. You can do things like configure the buttons with xsetwacom (and you’ll need that when it comes time to rotate the screen), but I kept getting this error when I tried to run xsetwacom:

[cc lang=”bash”] $ xsetwacom xsetwacom: error while loading shared libraries: libwacomcfg.so.0: cannot open shared object file: no such file or directory. [/cc]

I made a lucky guess, and fixed the problem with a quick ldconfig:

[cc lang=”bash”] $ sudo ldconfig # that was a lucky guess. [/cc]

Update: There were some issues with the wacom calibration after a sleep/resume cycle if the laptop screen had been rotated during that prior wake cycle (this happens a lot more than it seems, given how complex that description is.) I’ve written up a workaround here.

I finally broke down and bought a Lenovo X61 tablet (with SXGA+ screen!), and it arrived this week. This is the first of a series of posts about getting it up and running with Linux.

First off, some specs:

• Lenovo X61 Tablet PC with XSGA+ (1400x1050) screen (not multi-touch)
• 4 gigs of ram
• 200gb SATA hard disk
• WIFI (Intel PRO/Wireless 4965 AG or AGN)
• Gigabit Ethernet (Intel 82566MM)
• Bluetooth
• Wide-area networking card (3G)
• Fingerprint reader
• Integrated SD card reader (Richoh R5C822 SD/SDIO/MMC/MS/MSPro)
• Intel Audio (82801H ICH8 Family audio controller)
• 1 PCMCIA (type-I?) slot
• 4-cell battery
• Ultrabay Slim (which will be holding my Ultrabay CD-RW / DVD drive from my old T42p)
• Intel Mobile GM965/GL960 video controller. (256mb video ram?)

I’ll flesh that list out more as I can find the details (eg: wireless chipset, video, etc..)

First off, I blew some time poking around in Vista of course :). The handwriting input app is phenomenal in a lot of ways. It works very well, training is well integrated, and it has worked with every input area I’ve tried. It could be better if it had contextual clues, and could tie into things like Eclipse’s intellisense. Overall, though, it is amazing how simple it is to use, and how aesthetically pleasing the handwriting actually is. There is a lot to be said for using a couple extra pixels to make the strokes taper off as you pull the pen away. It has QWAN.

That done, I started to move on to installing Linux. I’m giving Ubuntu 8.10 the first chance, and I thought I’d try using a USB-based install so I wouldn’t have to monkey around with the Ultrabase & drive. If you have an 8.10 system already, you can easily create a bootable usb ubuntu drive with usb-creator and an ubuntu iso. This takes perhaps 45min - 1 hour.

Booting was as simple as going into the ibm bios-like page (by hitting the ThinkVantage button on boot) and telling it to boot from another device, then selecting the usb drive (that I had already inserted). I split the existing 200gb partition in two with the ubuntu installer, keeping Vista in it’s 100 gig sandbox, and leaving the remaining ~100 gigs for Ubuntu to partition further (which it did, as two partitions: one for / and one for swap. /dev/sda5 and /dev/sda6).

I do wish it had said how much space was being allocated to each of those partitions though. The installer didn’t give any indication.

Installation from booting the installer from usb to booting into the installed system took right about 30min. I’m impressed :)

Out of the box:

• The screen is the proper res
• Wireless looks like it might be working (I have to AP to verify)
• Sound works
• the pen does not
• Screen rotation does not work
• closing the lid shuts off the screen, but does not put the laptop to sleep.
  • This was easily fixed in the gnome power-management settings, and hey, it resumes too!
• putting the laptop in tablet mode seems to have no effect (at least it doesn’t shut off the screen ;)
• Some power management is clearly working (screen dims when unplugged)
• bluetooth was detected, but I have to way to test it.
• Dual-booting seems to work just fine, although there are two entries for Vista in the grub menu, and the first boots into the Rescue and Recovery system. Vista also had to do a chkdsk, and reboot before it would load.

More information as I figure it out :)

I desperately needed to find out why the tutorials I’ve been following for an Eclipse PDE task today kept referencing a startup.jar file that I could not locate.

A couple google searches later turned up this link:

http://dev.eclipse.org/newslists/news.eclipse.platform/msg62159.html

The poster in that thread had the same problem (back in Feb. 2007), and found the answer, but none of the content in that thread makes it trivial to locate the answer again.

The responder (with the answer) simply included a link to another mailing list:

http://dev.eclipse.org/mhonarc/lists/cross-project-issues-dev/maillist.html

Notice that that page is not constant. Today, it shows the most recent posts as of October 31st, 2008. In order to figure out what had happened to startup.jar, I had to take into account the OP’s response (“Ok so this is very recent.”), the timestamp on the messages (Mon, 12 Feb 2007) and then navigate the mailing list archives to find that time period, and start reading.

Please don’t put people through this sort of crap. It’s generally not difficult to find permlinks to a given email, or include a quick note with the actual answer. I have the answer now (startup.jar was replaced with org.eclipse.equinox.launcher in 3.3), but there is no way that I can tie that answer to the conversation I’ve linked to above.

For the purposes of Google:

If you’re having this problem:

I’m trying to do some automation, but I’m running into a problem with the 3.3 integration build.

java -cp plugins\org.eclipse.platform_3.2.100.v20070126\startup.jar org.eclipse.core.launcher.Main

doesn’t do anything. It doesn’t say anything. The only information I’m getting is an exit status of 13.

Then you need to use “java -jar plugins/org.eclipse.equinox.launcher_1.0.0.v20070207.jar” (adjusting the version numbers for your installation).

(Edit: If you’re reading this after OSGi R4.2, then there is almost certainly a better way to accomplish the same thing)

I’ve been digging into OSGi a bit over the last week or so inorder to create some Eclipse plugins that will automatically discover eachother, and I’ve been generally impressed with the framework on the whole. The documentation is a bit lacking, but there are some good blog posts about it. (Specifically Neil Bartlett’s introduction to OSGi.)

One thing that bugged me is the repetition needed when you implement the CommandProvider interface to add commands to the OSGi console. CommandProvider defines one method you must supply:

[cc lang=”java”]

public String getHelp()

[/cc]

OSGi then uses reflection to extract each of the methods that starts with an underscore, and supplies those methods to the command environment as new commands. (The underscore is trimmed, and the name of the method becomes the command name.) General practice is to include the name of the method in the return value of getHelp(), along with a description of what the method does, eg:

[cc lang=”java”] public class SampleCommandProvider implements CommandProvider {

public synchronized void _run(CommandInterpreter ci) {

  // do stuff.

}

public String getHelp() {

  return "\trun - execute a Runnable service";

} }[/cc]

This seems like a pain to maintain, so I took a quick look at annotations, and propose a new syntax:

[cc lang=”java”] public class SampleCommandProvider extends DescriptiveCommandProvider {

@CmdDescr(description=”execute a Runnable service”) public synchronized void _run(CommandInterpreter ci) {

  // do stuff.

} }[/cc]

Here we’ve extracted the getHelp() method into a new superclass, so our SampleCommandProvider now extends an abstract class instead of implementing an interface. It also makes use of an Annotation, which we need to define:

[cc lang=”java”] import java.lang.annotation.ElementType; import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; import java.lang.annotation.Target;

@Retention(RetentionPolicy.RUNTIME) @Target(ElementType.METHOD) public @interface CmdDescr { String description(); }[/cc]

Finally, we just need to define the superclass that implements getHelp():

[cc lang=”java”] import java.lang.reflect.Method; import java.util.regex.Matcher; import java.util.regex.Pattern;

import org.eclipse.osgi.framework.console.CommandProvider;

public abstract class DescriptiveCommandProvider implements CommandProvider {

private static final Pattern CMD_PATTERN = Pattern.compile(”^_(.*)”); private String help = null;

public String getHelp() {

  if (null == help){
     help = buildHelp();
  }
  return help;

}

private String buildHelp() {

  StringBuilder helpBuff = new StringBuilder();

  for (Method m : this.getClass().getMethods()){
     if (methodIsCmd(m)){         
        if (0 != helpBuff.length()){
           helpBuff.append("\n");
        }
        helpBuff.append(getDocumentation(m));            
     }
  }
  return helpBuff.toString();

}

private boolean methodIsCmd(Method m) {

  return CMD_PATTERN.matcher(m.getName()).matches();

}

private String getDocumentation(Method m) {

  StringBuilder methodHelp = new StringBuilder();

  Matcher matcher = CMD_PATTERN.matcher(m.getName());
  if(matcher.matches()){
     methodHelp.append("\t"+matcher.group(1));

     CmdDescr description = m.getAnnotation(CmdDescr.class);

     if (null != description){
        methodHelp.append(" - "+description.description());
     }
  }
  return methodHelp.toString();

} } [/cc]

Note that the actual reflection on the class only happens once – all subsequent calls to getHelp() use a cached copy of the documentation.

The venerable Joel Spolsky asked recently why someone hasn’t made a device that clips to the back of a desk and:

* It’s a power strip
* It’s a network hub
* It’s a USB hub
* You clamp it onto the back of any desk

The idea being that:

This would make it easy to plug in laptops, USB peripherals, and all your rechargers at your desk without crawling around on the floor.

He links to a device that does some of this, and runs ~$150/device. At that price, I think a better solution is a docking station–when you get down to it, I don’t want to plug in 4 things every time I sit down even if it doesn’t involve crawling under the desk (power, video, usb, ethernet and possibly audio). I think it’s unlikely that all the features needed above are really necessary when you just show up for a meeting, or hop over to your coworker’s office for a short hacking session. Many conference rooms these days already have tables wired for ethernet / power and svga video to a projector.