I ended up writing a short program to verify that it was reseting itself. I wrote values to the two configuration registers different from the default and then set the mode to continuous. I immediately read back the registers to verify the values written. After than wait a second or so and read them back. At which it was in the default configuration and in idle mode.
I sent an email to SparkFun describing this and I received a response confirming that a tantalum capacitor with an ESR of ~2 Ohms was used and that they are switching to a ceramic part for future production.
Notable for its absence was any mention of what would be done about defective product already sold.
I dug around in my parts collection and turned up a 1206 size 10uF ceramic cap that I had forgotten about. Installing it was not easy as one pad is well heatsinked. I am not happy with the soldering job but it will have to do as I am afraid that more heat will damage the capacitor.
With the new capacitor the HMC5843 is happily churning out data.
Unfortunately my SD card code is acting up so I have yet to record enough data to see if it is any good.