Building OpenOCD/FTDI for Vista64 - a crude guide

[el_nihilo]

[UPDATED 2009.10.20 - To reflect the current patch status of OpenOCD/GIT]
[UPDATED 2009.10.11 - GIT information and better approach to x86_64 detection]


Remember: The PREFERRED way of compiling and running OpenOCD with an FT2232 device on Windows 64 bit systems should be using libftdi/libusb-win32, as the resulting executable below is not GPL compliant. You can find a complete guide on how to build a better version of OpenOCD that uses libftdi here

Preamble

Since I finally managed to get an OpenOCD version working on Vista 64, with the proprietary 64 it version of the FTDI drivers/library, I thought I'd share the compilation process I used with MinGW x64.
I tested this with OpenOCD rev. 2826 and an FT2232 based ShevaPlug.

First of all, let me state that I wholeheartedly support the OpenOCD developers decision to try to enforce the use of the GLP'd libftdi vs. the proprietary FTDI library, but, as you all know, getting libftdi to work in Vista 64 is a bit of a headache, and I know there are quite a few of you longing for the convenience of using OpenOCD on your Vista 64 systems.
Of course, if I manage to get OpenOCD to work in Vista 64 with libftdi, I'll let you know.

I will also request that, if you do manage to compile an OpenOCD.exe version that runs on Vista 64 by following the instructions below, you do NOT publish it on the internet, as this would be a breach of the GPL License (and obviously, don't bother asking me to share my binaries)

This being said, be also warned that this is a very crude guide: We'll be doing some quick and dirty stuff in there and you'll also need to fill in some gaps. Hopefully someone will come along and produce a better version of a compilation guide for Windows x64 and submit the needed MinGW x64 patches to the OpenOCD tree (as I have no plans to do so myself). The goal here is to get an OpenOCD executable that works on Windows x64 as quickly as possible.


Compiling OpenOCD with WPG System64

Alright, that was a long preamble enough, let's jump in.

The toolchain we're going to use to compile OpenOCD is MinGW, because currently that's the only GNU Like toolchain that can produce x86_64 binaries on Windows (will cygwin ever get there? Who knows...).
And of course, we're not going to bother recompiling our own toolchain - instead we're going to use the ready made WPG System64 because:
1. It's up to date
2. It's a doodle to install: No need for an installer - just extract the files to C: and you're done!

I used the "WPG System64 1.99-preview" version (WPG_System64-1.99p.7z), which you can find by following the previous link. Just be warned that it takes 2 GB once extracted, and it is configured to run on the C: drive.

Once extracted, you should have a C:\msys directory. In there you will find a "WPG System64 (Bash shell)" shortcut, which you should run to get a command prompt.

Now, before you can compile OpenOCD, you'll need the FTDI proprietary drivers, which you can download from the FTDI website.
For the purpose of this exercise, we will extract these files in C:\msys\src (create the "src" directory if it doesn't exist), and then rename the "C:\msys\src\CDM 2.04.16 WHQL Certified" to "C:\msys\src\ftd2xx".

STEP 0: Retrieving the OpenOCD source.
As the latest GIT sources includes patches needed for successful MinGW-W64 compilation, this is what you should use. For the time being (see below) yopu will need to install a separate GIT client for WPG System64 but that should change with an updated release). You can find a GIT client for MSYS here. Then download the latest sources with:

Code: Select all

git clone git://openocd.git.sourceforge.net/gitroot/openocd/openocd

You should place the source in C:\msys\src\ (i.e. you should end up with a C:\msys\src\openocd directory)
Do not use the SVN or archives earlier than 0.3.0, as the SVN tree is no longer maintained, and the archives do not contain the required patches.

STEP 1: (This where the quick & dirty part start) Edit openocd/configure.in and change the section (line #683)

Code: Select all

    # And calculate the LDFLAGS for the machine
    case "$host_cpu" in
    i?86|x86_*)
      LDFLAGS="$LDFLAGS -L$with_ftd2xx_win32_zipdir/i386"
      LIBS="$LIBS -lftd2xx"
      f=$with_ftd2xx_win32_zipdir/i386/ftd2xx.lib
      ;;
    amd64)
      LDFLAGS="$LDFLAGS -L$with_ftd2xx_win32_zipdir/amd64"
      LIBS="$LIBS -lftd2xx"
      f=$with_ftd2xx_win32_zipdir/amd64/ftd2xx.lib
      ;;

to

Code: Select all

    # And calculate the LDFLAGS for the machine
    case "$host_cpu" in
    i?86|x86_32)
      LDFLAGS="$LDFLAGS -L$with_ftd2xx_win32_zipdir/i386"
      LIBS="$LIBS -lftd2xx"
      f=$with_ftd2xx_win32_zipdir/i386/ftd2xx.lib
      ;;
    amd64|x86_64)
      LDFLAGS="$LDFLAGS -L$with_ftd2xx_win32_zipdir/amd64"
      LIBS="$LIBS -lftd2xx"
      f=$with_ftd2xx_win32_zipdir/amd64/ftd2xx.lib
      ;;

The reason we do that on x86_64 system, the first case is still used (matches x86_*), which is plain wrong as we need to use the 64 bit version of the FTDI lib!. I will try to submit a patch to address this. Note that the configure script also appears not to be able to report 64 bit x86 CPUs properly (but we can fix that with the --build=x86_64-w64-mingw32 below).

STEP 2: In the command prompt, run

Code: Select all

./bootstrap

STEP 3: [NOT NEEDED IF DOWNLOADED FROM GIT] If you downloaded from SVN, edit openocd/src/openocd.c and remove or comment the line (line #50):

Code: Select all

#error THIS IS AN OBSOLETE VERSION OF OpenOCD ... latest is in GIT

It looks like OpenOCD very recently switched from SVN on berlios.de to GIT on SourceForge (?), but we don't have GIT in WPG System64 (for now - see below ). If you don't remove that line, the compilation process will fail.
Not sure what's going on between the berlios and SourceForge repository (has there been a fork or something?)


STEP 4: [NOT NEEDED IF DOWNLOADED FROM GIT] Edit openocd/src/Makefile.in and change the line (line #300):

Code: Select all

@IS_MINGW_TRUE@MINGWLDADD = -lwsock32

to

Code: Select all

@IS_MINGW_TRUE@MINGWLDADD = -lws2_32

If you don't do that, your compilation will fail on -lwsock32 not found near the end


STEP 5: If you try to run "./configure ..." at this stage, you will find that eventhough the process does find the right library, you still get the ominous

Code: Select all

configure: error: Cannot build & run test program using ftd2xx.lib

A look at the config.log will report the error

Code: Select all

C:\Users\nil\AppData\Local\Temp\ccOBXOer.o:conftest.c:(.text+0x1e): undefined reference to `_imp__FT_GetLibraryVersion'

The problem here is that MinGW does expect DLL translated names from .lib files to begin with __imp__ (eg: __imp__FT_GetLibraryVersion) whereas Visual C++ libraries provide names beginning with __imp_ (eg: __imp_FT_GetLibraryVersion). The problem really is something as stupid as using one or two underscores to declare the names of the DLL functions in the relevant .lib file, but because of that we can't use the .lib provided by FTDI. Instead we need to recreate one.
Now, because the library has been stripped, you would normally need to use the Visual Studio tool command "dumpbin /exports ftd2xx64.dll > exports.txt" and use it to create a .def file with the library members, that you can then use with MinGW's dlltool, but the truth is, with the current FTDI lib, the following command will give you what you want, thus:

Code: Select all

cd /usr/src/ftd2xx/amd64
echo "LIBRARY ftd2xx64.dll" > ftd2xx64.def
echo "EXPORTS" >> ftd2xx64.def
strings ftd2xx64.dll | grep FT_ >> ftd2xx64.def
mv ftd2xx.lib ftd2xx.lib.old
dlltool.exe -d ftd2xx64.def -l ftd2xx.lib

This should create a new "ftd2xx.lib" that MinGW can understand.
Note: Don't forget the "LIBRARY" line in your def file, otherwise any executable created with the FTDI library will fail with the error

Code: Select all

the application failed to start because "(null).dll" was not found

STEP 6: At this stage, you want to make ftd2xx64.dll available to the system (else you're going to get "ftd2xx64.dll not found). The easiest way is to simply copy the DLL file to your C:\Windows\System32\ folder (Yes, I know it's a 64 bit library, but copying it ot C:\Windows\SysWOW64\ doesn't seem to work)


STEP 7: This time, everything should be set for configure to work, so:

Code: Select all

cd /usr/src/openocd
./configure --enable-maintainer-mode --enable-ft2232_ftd2xx --with-ftd2xx-win32-zipdir=/usr/src/ftd2xx --build=x86_64-w64-mingw32

Make sure you specify amd64_w64-mingw32 to either --build or --host, as otherwise the configure script will still report your CPU as 686, and thus will look for the 32 bit version of the FTD2xx libraries (see step 1)
If you want to change the default install locations for openocd (default to /usr/local...) now is probably a good time to do so by specifying additional configure options.


STEP 8: [NOT NEEDED IF DOWNLOADED FROM GIT] And now the fun part begins... If you thought all the above was painful enough to setup, think again. The problem is that openocd sure wasn't designed with MinGW x64 in mind, and there are about 12-16 source files in there that are going to choke on (apparently nonstandard) "%lld" printfs and the like... So you have to patch them one after another.
And if you think I'm gonna help you on this one, sorry but no.
It's time consuming but it should be easy enough to figure out. As far as I'm concerned, I replaced problematic "long" variables with "unsigned long long", "%lld" or "%i" with "%I64d", and removed %hh or %j prefixes here and there.
To be truly platform independent, you'd probably want to use the C99 "PRIi64" "PRIu64" "conventions", but whoever came up with these cumbersome macros should be put in a cubicle without any windows to reflect on how non-intuitive these "conventions" are.
Oh, and for that armv7a.c comparison error, I just used a temporary variable.


STEP 9: After all this, at long last, you should have a working 64 bit version of openocd.exe. At this stage, you want to finalize your installation with "make install"


STEP 10: Time for a test. You should of course have previously installed the FTDI USB drivers. And if you get a firewall exception warning from Vista on the first launch of openOCD, accept it!

Code: Select all

$ cd /usr/local/share/openocd/scripts/
$ /usr/local/bin/openocd -f board/sheevaplug.cfg
Open On-Chip Debugger 0.3.0-dev-svn2826 (2009-10-08-19:11)
$URL: svn://svn.berlios.de/openocd/trunk/src/openocd.c $
For bug reports, read
        http://openocd.berlios.de/doc/doxygen/bugs.html
2000 kHz
trst_and_srst separate srst_gates_jtag trst_push_pull srst_open_drain
jtag_nsrst_delay: 200
jtag_ntrst_delay: 200
dcc downloads are enabled
Warn : use 'feroceon.cpu' as target identifier, not '0'
Info : device: 4 "2232C"
Info : deviceID: 2659753615
Info : SerialNumber: ********
Info : Description: SheevaPlug JTAGKey FT2232D B A
Info : clock speed 2000 kHz
Info : JTAG tap: feroceon.cpu tap/device found: 0x20a023d3 (mfg: 0x1e9, part: 0x
0a02, ver: 0x2)

and from a telnet window (eg putty) to localhost:4444

Code: Select all

Open On-Chip Debugger
> sheevaplug_init
target state: halted
target halted in ARM state due to debug-request, current mode: Supervisor
cpsr: 0x000000d3 pc: 0xffff0000
MMU: disabled, D-Cache: disabled, I-Cache: disabled
0 0 1 0: 00052078
> scan_chain
     TapName            | Enabled |   IdCode      Expected    IrLen IrCap  IrMask Instr
---|--------------------|---------|------------|------------|------|------|------|---------
 0 | feroceon.cpu       |    Y    | 0x20a023d3 | 0x20a023d3 | 0x04 | 0x01 | 0x0f | 0x0c
> nand probe 0
NAND flash device 'NAND 512MiB 3,3V 8-bit' found
> nand list
#0: NAND 512MiB 3,3V 8-bit (Hynix) pagesize: 2048, buswidth: 8,
        blocksize: 131072, blocks: 4096
>

PS: For those interested, I also have a guide on how to compile OpenOCD (with libftdi or ftd2xx.lib) for a Linux x64 system. Can't link to it because of this forum's restrictions though... Oh well.

[xenofears]

Hello! I am Xenofears, WPG System64 is my project.

I will also request that, if you do manage to compile an OpenOCD.exe version that runs on Vista 64 by following the instructions below, you do NOT publish it on the internet, as this would be a breach of the GPL License (and obviously, don't bother asking me to share my binaries)

GPL can be shared as long as the source is shared with it. Did you mean this was a non-free license, to the GPL libftdi? I can distribute GPL licensed binaries with WPG System64 itself, as it is open-source (although right now the source is in a huge messy pile.)

If this has a demand like this, I might make a try for libftdi myself, and OpenOCD, something I am not yet fully acquainted with - but I do need to count my battles, and get the important bits remaining, already outlined, finished first.

(will cygwin ever get there? [to 64-bit] Who knows...).

Nah, they never will. It isn't even really relevant, as Cygwin produces Cygwin binaries, not Win32 (unless a cross-compiler is built.) It is a difference many people don't realize until they try to bring their binaries outside Cygwin, and the DLL portability is a mess.

case "$host_cpu" in
i?86|x86_*)
LDFLAGS="$LDFLAGS -L$with_ftd2xx_win32_zipdir/i386"
LIBS="$LIBS -lftd2xx"
f=$with_ftd2xx_win32_zipdir/i386/ftd2xx.lib
;;
amd64)
LDFLAGS="$LDFLAGS -L$with_ftd2xx_win32_zipdir/amd64"
LIBS="$LIBS -lftd2xx"
f=$with_ftd2xx_win32_zipdir/amd64/ftd2xx.lib
;;

I see reference to $host_cpu. This is usually determined (for 64-bit on Windows) by passing --host=x86_64-w64-mingw32 to configure. But this script is wrong, x86_* should not mean a 32-bit architecture. You can use --host=amd64-w64-mingw32, it will work (that is, Mingw-w64 will function with it, and it appears it will solve this issue.) This is assuming autotools, if it is some proprietary configure, then all bets are off. You can even pass --build=amd64-w64-mingw32 to configure, if that is where it gets host cpu from.

but we don't have GIT in WPG System64, so SVN will have to do. If you don't remove that line though, the compilation process will fail.

We will, very soon.

I will take a look sometime soon when I have the time, see if there is anything I can assist with. Once built, a 64-bit patch to the changes you had to make is easy enough, just use diff -u.

There is probably an easier way to solve the symbol naming, I will see if the Mingw-w64 lead developers have anything to say about it. In fact, I do believe this was a recent issue, underscores were removed on purpose, for a good reason.

Oh - and Putty is included! It's in system64/ssl/bin, and in the path.

UPDATE: try using -fno-leading-underscore

Hope this helps!

[el_nihilo]

Hi xenofears, nice to see you around.

First of all, let me express my thanks for your WPG System64 project. It's making our lives on 64 bit Windows systems so much easier!

Did you mean this was a non-free license, to the GPL libftdi? I can distribute GPL licensed binaries with WPG System64 itself, as it is open-source (although right now the source is in a huge messy pile.)

The problem boils down to using the proprietary & non open-source FTDI library in what is otherwise GPL software. For more information, have a look here.

If this has a demand like this, I might make a try for libftdi myself

Very much appreciated, thanks.
I'm still planning to have a look at it too (haven't had a chance for now). One issue with that is the GPL'd libfdti relies on libusb64 for Windows as well as self-signing the device drivers, which means running in test mode in Vista x64, but I don't see that as that big a problem. Having a full GPL compliant version of OpenOCD on 64 bit Windows system is obviously what we need to push for.

(this whole i?86, x86_*, amd64 mess)

I updated my post for a more satisfying approach now (breakdown of x86_32 and x86_64). Totally agree that x86_* is a mistake.
Not exactly sure how $host_cpu is detected exactly by the scripts, but either of amd64-w64-mingw32 or x86_64-w64-mingw32 returns x86_64 for $host_cpu on my system.
Note that I'm advising the use of --build rather than --host above as using --host returns a warning ("configure: WARNING: If you wanted to set the --build type, don't use --host. If a cross compiler is detected then cross compile mode will be used.)

We will, very soon (have GIT).

That's great news! Thanks.

UPDATE: try using -fno-leading-underscore

I tried that in $CFLAGS, but then I got issue with the regular MinGW libraries. My understanding is really that VC++ and MinGW libraries use incompatible conventions, so as soon as you mix them, you have to convert one or the other. But that -fno-leading-underscore sure is a good trick to know!

[el_nihilo]

At last!

I have an working OpenOCD with libftdi/libusb running on Vista 64, which means we should now be able to push some kind of ready-made binary package for you guys.

That wasn't without a struggle => I need to clean up things a bit before I can post more. Hopefully, I should be able to do that tomorrow.

Note that, because we don't have signed 64 bit drivers for libusb, you will need to run Windows 64 in test mode for OpenOCD to work, but that's not a big deal IMO (and actually, if MS weren't so backwards with regards to security, "test mode" should be the standard operating mode of Vista: if your certificate store is protected from rogue operations, self signed drivers are NOT a security risk dammit!).

Stay tuned for more...

[xenofears]

At last!

I have an working OpenOCD with libftdi/libusb running on Vista 64, which means we should now be able to push some kind of ready-made binary package for you guys.

That wasn't without a struggle => I need to clean up things a bit before I can post more. Hopefully, I should be able to do that tomorrow.

Note that, because we don't have signed 64 bit drivers for libusb, you will need to run Windows 64 in test mode for OpenOCD to work, but that's not a big deal IMO (and actually, if MS weren't so backwards with regards to security, "test mode" should be the standard operating mode of Vista: if your certificate store is protected from rogue operations, self signed drivers are NOT a security risk dammit!).

Stay tuned for more...

Hey, can you pass the source patches, or just the sources, on to me? I need libusb too, configure won't build it on Windows, don't know what kind of source patching needs to be done if any.

In exchange I can give you an easy way (the best way) to get rid of those M$ test mode watermarks.

[el_nihilo]

Hi xenofears,

You can pick up libusb & its drivers from the OpenOCD binary I just posted.
The .def file is included so you can create a .lib using dlltools and use it in your projects.

I didn't patch anything to compile it (well, almost - just changed a couple of files to get _x64 names). I just used the latest libusb-win32 sources with the (freely available) Windows Driver Kit v 7.0.0, as it contains DDK friendly scripts.

You should be able to use the versions I compiled with non FTDI based USB devices if you modify the .inf accordingly. I used the Win7 x86 and x64 build environments, so these drivers should be Win7 compatible as well. Only tested on Vista 64 though.

I'll try to post more as soon as I can.

PS: I don't care much about disabling the Test Mode & version report on the Wallpaper. In fact, I prefer seeing the Windows version I am running.

[Homncruse]

First off, THANK YOU for this guide! I've spent all day trying to get OpenOCD to work with my new Flyswatter from Tin Can Tools. This whole thing was brought about by the fact that I've upgraded my dev setup to Win7 x64, and our previous debug setup relied on a Macraigor usbDemon. Macraigor doesn't support 64-bit Windows, and that's the only point of failure for upgrading to 64-bit for our devs. Getting OpenOCD to work under Win7 64-bit was no trivial task either! I've scoured all corners of the internet, including these forums, and this guide is the first that works almost flawlessly. I say almost, because...

STEP 6: At this stage, you want to make ftd2xx64.dll available to the system (else you're going to get "ftd2xx64.dll not found). The easiest way is to simply copy the DLL file to your C:\Windows\System32\ folder (Yes, I know it's a 64 bit library, but copying it ot C:\Windows\SysWOW64\ doesn't seem to work)

...isn't correct. I placed it into both System32 and SysWOW64 and I kept getting the crappy "DLL could not be found" error with the ./configure command. With an MSys/MinGW setup, place the DLL file into c:\msys\bin, and all will be happy. It seems that the Msys is a completely independent path environment. I managed to compile successfully and have an openocd.exe linked against the native FTDI drivers, with no stupid "Test Mode" enabled required!

Now let's hope the rest of my testing goes a little easier.

[Homncruse]

I may as well contribute my crudeness to this crude guide. If I was any good at shell scripting, I'd make this robust to check for 32-bit or 64-bit, Windows or Linux, etc. In other words, a simplification wrapper around configure. But I don't really know where to begin on that, and a shell script is probably the wrong tool to use for such a task. However, perhaps this will provide enough of a launch point for someone else to do just that

Code: Select all

#!/usr/bin/env bash

# Instructions adapted from http://forum.sparkfun.com/viewtopic.php?f=18&t=17598
# All credit goes to el_nihilo and those who helped him. I just made this script to automate the process.
# This script assumes the latest FTD2xx drivers have been downloaded from ftdichip.com and placed in ./src/ftd2xx

# Create a working 64-bit FTDI library file for use by OpenOCD (the FTDI-supplied one has issues)
cd ./src/ftd2xx/amd64
echo "LIBRARY ftd2xx64.dll" > ftd2xx64.def
echo "EXPORTS" >> ftd2xx64.def
strings ftd2xx64.dll | grep FT_ >> ftd2xx64.def
mv ftd2xx.lib ftd2xx.lib.old
dlltool.exe -d ftd2xx64.def -l ftd2xx.lib
# Store it in /bin so it's easily findable
cp ftd2xx.lib /bin
# Back to OpenOCD root
cd ../../../

# After all that hard work, we can finally run the configure script and resulting make targets!
./bootstrap
./configure --enable-maintainer-mode --enable-ft2232_ftd2xx --with-ftd2xx-win32-zipdir=./src/ftd2xx --build=x86_64-w64-mingw32
make -j all
make -j install

I make no claims as to the correctness of all this stuff (e.g., are make all and make install both required? Can they be combined safely?), and this is all functional as of OpenOCD 0.5.0 from the git head (build string: 0.5.0-dev-00188-g620310b (2010-04-16-14:01)). It works for me, and it's going into my local toolchain build, but YMMV.

[kees]

Perhaps not really on topic coz I use Windows7 x64 but perhaps this is useful for others as well.

What I wanted to do is rather obvious: Build a 64 bit version of OpenOCD and use the FTDI driver. I know it is not the preferred way to do it, but for me it is far easier to just install the signed drivers than to go through the hassle of signing the open source drivers.

I just followed the guide described in this thread, using Mingw64, but no matter what I tried, the configure script kept telling me: "Cannot build & run test program using ftd2xx..." Well, it took me hours to figure out why but finally I was able to find out what went wrong:

The configure script wants to include "ftd2xx.dll" where my system uses the amd64/ftd2xx64.dll. This is done here (in configure):
Line 13166:

Code: Select all

    amd64|x86_64)
      LDFLAGS="$LDFLAGS -L$with_ftd2xx_win32_zipdir/amd64"
      LIBS="$LIBS -lftd2xx"
      f=$with_ftd2xx_win32_zipdir/amd64/ftd2xx.lib

The only change I made is just two characters:

Code: Select all

    amd64|x86_64)
      LDFLAGS="$LDFLAGS -L$with_ftd2xx_win32_zipdir/amd64"
      LIBS="$LIBS -lftd2xx64"
      f=$with_ftd2xx_win32_zipdir/amd64/ftd2xx.lib

And surprise: It worked!!!

Well, there was a tiny little problem after that as well: The cortex_a8.c file wouldn't compile. The compiler flags are set to treat warnings as errors and probably the later gcc compiler is a bit more sensitive. It complained about:
cortex_a8.c: In function 'cortex_a8_halt':
cortex_a8.c:762:3: error: jump skips variable initialization

There is a statement on line 775:

Code: Select all

	long long then = timeval_ms();

this is skipped by a bunch on goto's, so that then is not initialized and the compiler (needless I have to say) complains about that. I move the variable declaration to the start of the function and changed the line to:

Code: Select all

	then = timeval_ms();

That was all that we required to build and install on Windows 7 x64...

Hope this help others too.....
Kees.