- Tue Sep 08, 2009 10:32 am
#80528
First off this is not an outgrowth of my work on the Audio Recorder Project or the SMD Oven. This is just another idea I have been kicking around.
Recent interest in Arduino related projects and tools have been pretty hot from a hobbyist perspective. Hardware keeps changing, new chips come out, and using a familiar tool chain can save a lot of effort in bringing up a new project. In the software world re-inventing the wheel is expensive and frustrating. More often than not the end result is counter productive, and limits growth into new platforms.
I can see a time not too long from now where pretty much all the major players are going to have cheap and fast 32bit SoCs that don't require ANY high speed parallel interfaces off chip. Most of them already do. The challenge I see for FOSS is to encompass as many of these new platforms as possible with the fewest tools possible. This makes building tool chains much less of an issue than it is now. The AVR crowd has had a lot of success with the Arduino libraries and it's straight forward IDE. Interest in it has been trending up even in this sucky economy. Part of it's WIN is that it's modest in scope and provides a lot of value for very little effort.
Some time ago I stumbled on to a few usenet chat threads that discussed porting Arduino to the PIC. At the time the threads were written I don't think many people in the PIC communities realized how the Arduino system worked and why a simple port of the tool chain might be a lot harder than any one thought. All of those discussions kinda devolved into Inet noise not long after the PIC32 hit the market. I'm not sure why.
So to open the discussion I'd like to share some insights about how Arduino works for those of you who don't know, and outline a possible approach to porting the tool-chain to PIC18 and PIC32MX platforms.
Arduino under the hood/bonnet:
The core of Arduino is a GCC based cross-compiler that has been built using a custom libc, libm and crtl.0. For those that don't really know how GCC works these three libraries are critical for getting C code to run. While you can get by without much of the high-level stuff in these libraries there is some of it that is just not possible to do without. Also, a lot of higher level libraries we know and love in C are written in a way that makes them easy to recompile once libc and libm have been ported to a platform. Arduino is built on this premise.
To summarize the tool chain we have:
Arduino IDE <--> avr-gcc(avr-libc) --> Firmware Programming Tool <---> AVR Target Platform
The Arduino IDE is little more than a Java based text editor with some hooks in it to call avr-gcc and process the error messages that gcc might spit back. Porting it to Mac OS, Linux, and Windows was done a long time ago, and there's nothing in the tool chain that is host specific. Damn good job. The entire tool chain is FOSS and is distributed as binaries. Source code is easy to get, and I think a lot of people are modding this code base. Even for people who don't like the Arduino IDE, using your own IDE over the top, such as Eclipse or even XCode is not much of a stretch. The real power of Arduino is the libraries, not the IDE.
Ok so what about the compiler part?
The gcc compiler support for AVR has been mature for quite some time, and most of the variants are fully supported and optimized. On the PIC side it's not such a clear picture:
For PIC18 platforms there is SDCC which I have been using in my projects and that is mature enough to support a PICduino Port. And here the two efforts dovetail. Arduino IDE doesn't really care what compiler it calls. The build command is just a constant in the Java code, and there is a small chunk of code that monitors the compiler output for errors, and feeds that back to the editor. The real meat is the libraries.
The big question mark comes for the PIC32MX. However, there is a lot of evidence that this is largely a clerical exercise that no one has taken up yet. The PIC32MX is based on a MIPS core (M4K) and I have found that there is mature support for this instruction set in gcc. However, no one has ported any libraries suitable for embedded use yet, except the commercial compiler developers. MIPS and Microchip did port, and release a hacked up version of gcc to the PIC32MX. They deliberately hobbled the source. Building it requires proprietary chunks that you can only get from Microchip, and cannot be bundled with the FOSS portion of the compiler. However an experienced and clever monkey CAN build a personal tool chain with some pain and get a working compiler, then sell that code embedded into a PIC.... no issues there. Providing a easy to use tool-chain that you can just download and click-install is out the window, and forget trying to build an Arduino port around it .... it loses the whole point of why Arduino is so popular.... it's easy for a kid to set up and use without getting bogged in the arcane mess that is building a cross-gcc compiler. Read: Lots of ugly sheet metal cuts in that build process.
Ok so why the hacked up version of gcc from MIPS/Microchip? Is there something wrong with the FOSS MIPS M4K code generator that makes it unsuitable for use with the PIC32MX? The short answer is no, I do not believe so.
The longer answer is: I think a key reason for the hack is to delay widespread availability of competing FOSS IDE tools for the PICs without looking like it. Microchip and it's partners don't want to bite the hobbyist market, but they don't want commercial customers to bypass their tool-chain and dev platform products. Heck the Microchip IDE is free.... unless you want a C compiler. Then it gets kind of expensive (for a hobbyist)... and you are locked into their development products. Ok good show for their business model. Personally I don't like their tool chain that much, even though I use it for the back-end of mine right now.
A key element of the MIPS work on the GCC port was adding Procedural Abstraction (PA) to the pre-link and link layer of GCC, and developing a proprietary ctrl.0 library to enforce code-size limits on students and free-ware users of the MPLAB IDE C products. I don't see PA to be a show stopper at the long end of the tail that we all play in. (P.A. reduces executable/memeory footprint size by up to 30% -- not a show stopper for Hobbyists and small run solution developers) If PA becomes an important issue I am sure later versions of GCC will get PA support. I'm not gonna hold my breath. PA is only really useful for embedded platforms, and MIPS M4K support for it is not likely to show up as a priority.
Ok back to the FOSS side of the fence:
The real promise of an Arduino port to PIC is that sketches build for AVRs could run UNCHANGED on any supported platform that has a similar I/O ring.
So how might this be done?
The key, it turns out, is porting avr-libc to PIC. Most of the key chunks to do this already exist in SDCC for the PIC18. This is largely a clerical process not an engineering process. But it's still a lot of work. The next level would be porting the Arduino Core library and bootloader to the PIC, and this might involve a bit of engineering to keep the HAL functionally identical at the Sketch level. This is a lot of work as well, but not nearly as bad as starting over with a new library with no high-level user base.
For the PIC32MX it's a little more involved, since there are no machine specific FOSS header files for the PIC32MX devices yet, but this is a clerical issue of producing these header files... not many variants of the chip yet. Again looking at SDCC's PIC18 libc headers and and AVR-libc for guidance makes this much easier to do. The most difficult part will be porting ctrl.0 and the Arduino libraries to the PIC32MX, since this has to set up the machine context, and for this relatively new machine, there's not a lot of guidance on what needs to happen, since I have only just started digging into the guts of the PIC32MX. So for this part I'm hand-waving.
Building a cross-GCC for the PIC32 is not that big a deal... I'm planning to do a test build to see if I can get the linux libc to work for M4K code, and check it's output. I see no sense in messing with Microchip/MIPS poisoned sudo-FOSS implementation of C32-gcc, but for those that want a gcc based tool-chain there is a clear path to building one through that code base. If anyone here is interested I can point you to some links that outline the process. Another good candidate for non-arduino libc implementations is newlibc which would probably be relatively easy to port to PIC32MX, since it was designed to be embedded and platform agnostic.
Ok so this has gotten much longer than I expected.
Questions? comments? Discussion?
Cheers
Recent interest in Arduino related projects and tools have been pretty hot from a hobbyist perspective. Hardware keeps changing, new chips come out, and using a familiar tool chain can save a lot of effort in bringing up a new project. In the software world re-inventing the wheel is expensive and frustrating. More often than not the end result is counter productive, and limits growth into new platforms.
I can see a time not too long from now where pretty much all the major players are going to have cheap and fast 32bit SoCs that don't require ANY high speed parallel interfaces off chip. Most of them already do. The challenge I see for FOSS is to encompass as many of these new platforms as possible with the fewest tools possible. This makes building tool chains much less of an issue than it is now. The AVR crowd has had a lot of success with the Arduino libraries and it's straight forward IDE. Interest in it has been trending up even in this sucky economy. Part of it's WIN is that it's modest in scope and provides a lot of value for very little effort.
Some time ago I stumbled on to a few usenet chat threads that discussed porting Arduino to the PIC. At the time the threads were written I don't think many people in the PIC communities realized how the Arduino system worked and why a simple port of the tool chain might be a lot harder than any one thought. All of those discussions kinda devolved into Inet noise not long after the PIC32 hit the market. I'm not sure why.
So to open the discussion I'd like to share some insights about how Arduino works for those of you who don't know, and outline a possible approach to porting the tool-chain to PIC18 and PIC32MX platforms.
Arduino under the hood/bonnet:
The core of Arduino is a GCC based cross-compiler that has been built using a custom libc, libm and crtl.0. For those that don't really know how GCC works these three libraries are critical for getting C code to run. While you can get by without much of the high-level stuff in these libraries there is some of it that is just not possible to do without. Also, a lot of higher level libraries we know and love in C are written in a way that makes them easy to recompile once libc and libm have been ported to a platform. Arduino is built on this premise.
To summarize the tool chain we have:
Arduino IDE <--> avr-gcc(avr-libc) --> Firmware Programming Tool <---> AVR Target Platform
The Arduino IDE is little more than a Java based text editor with some hooks in it to call avr-gcc and process the error messages that gcc might spit back. Porting it to Mac OS, Linux, and Windows was done a long time ago, and there's nothing in the tool chain that is host specific. Damn good job. The entire tool chain is FOSS and is distributed as binaries. Source code is easy to get, and I think a lot of people are modding this code base. Even for people who don't like the Arduino IDE, using your own IDE over the top, such as Eclipse or even XCode is not much of a stretch. The real power of Arduino is the libraries, not the IDE.
Ok so what about the compiler part?
The gcc compiler support for AVR has been mature for quite some time, and most of the variants are fully supported and optimized. On the PIC side it's not such a clear picture:
For PIC18 platforms there is SDCC which I have been using in my projects and that is mature enough to support a PICduino Port. And here the two efforts dovetail. Arduino IDE doesn't really care what compiler it calls. The build command is just a constant in the Java code, and there is a small chunk of code that monitors the compiler output for errors, and feeds that back to the editor. The real meat is the libraries.
The big question mark comes for the PIC32MX. However, there is a lot of evidence that this is largely a clerical exercise that no one has taken up yet. The PIC32MX is based on a MIPS core (M4K) and I have found that there is mature support for this instruction set in gcc. However, no one has ported any libraries suitable for embedded use yet, except the commercial compiler developers. MIPS and Microchip did port, and release a hacked up version of gcc to the PIC32MX. They deliberately hobbled the source. Building it requires proprietary chunks that you can only get from Microchip, and cannot be bundled with the FOSS portion of the compiler. However an experienced and clever monkey CAN build a personal tool chain with some pain and get a working compiler, then sell that code embedded into a PIC.... no issues there. Providing a easy to use tool-chain that you can just download and click-install is out the window, and forget trying to build an Arduino port around it .... it loses the whole point of why Arduino is so popular.... it's easy for a kid to set up and use without getting bogged in the arcane mess that is building a cross-gcc compiler. Read: Lots of ugly sheet metal cuts in that build process.
Ok so why the hacked up version of gcc from MIPS/Microchip? Is there something wrong with the FOSS MIPS M4K code generator that makes it unsuitable for use with the PIC32MX? The short answer is no, I do not believe so.
The longer answer is: I think a key reason for the hack is to delay widespread availability of competing FOSS IDE tools for the PICs without looking like it. Microchip and it's partners don't want to bite the hobbyist market, but they don't want commercial customers to bypass their tool-chain and dev platform products. Heck the Microchip IDE is free.... unless you want a C compiler. Then it gets kind of expensive (for a hobbyist)... and you are locked into their development products. Ok good show for their business model. Personally I don't like their tool chain that much, even though I use it for the back-end of mine right now.
A key element of the MIPS work on the GCC port was adding Procedural Abstraction (PA) to the pre-link and link layer of GCC, and developing a proprietary ctrl.0 library to enforce code-size limits on students and free-ware users of the MPLAB IDE C products. I don't see PA to be a show stopper at the long end of the tail that we all play in. (P.A. reduces executable/memeory footprint size by up to 30% -- not a show stopper for Hobbyists and small run solution developers) If PA becomes an important issue I am sure later versions of GCC will get PA support. I'm not gonna hold my breath. PA is only really useful for embedded platforms, and MIPS M4K support for it is not likely to show up as a priority.
Ok back to the FOSS side of the fence:
The real promise of an Arduino port to PIC is that sketches build for AVRs could run UNCHANGED on any supported platform that has a similar I/O ring.
So how might this be done?
The key, it turns out, is porting avr-libc to PIC. Most of the key chunks to do this already exist in SDCC for the PIC18. This is largely a clerical process not an engineering process. But it's still a lot of work. The next level would be porting the Arduino Core library and bootloader to the PIC, and this might involve a bit of engineering to keep the HAL functionally identical at the Sketch level. This is a lot of work as well, but not nearly as bad as starting over with a new library with no high-level user base.
For the PIC32MX it's a little more involved, since there are no machine specific FOSS header files for the PIC32MX devices yet, but this is a clerical issue of producing these header files... not many variants of the chip yet. Again looking at SDCC's PIC18 libc headers and and AVR-libc for guidance makes this much easier to do. The most difficult part will be porting ctrl.0 and the Arduino libraries to the PIC32MX, since this has to set up the machine context, and for this relatively new machine, there's not a lot of guidance on what needs to happen, since I have only just started digging into the guts of the PIC32MX. So for this part I'm hand-waving.
Building a cross-GCC for the PIC32 is not that big a deal... I'm planning to do a test build to see if I can get the linux libc to work for M4K code, and check it's output. I see no sense in messing with Microchip/MIPS poisoned sudo-FOSS implementation of C32-gcc, but for those that want a gcc based tool-chain there is a clear path to building one through that code base. If anyone here is interested I can point you to some links that outline the process. Another good candidate for non-arduino libc implementations is newlibc which would probably be relatively easy to port to PIC32MX, since it was designed to be embedded and platform agnostic.
Ok so this has gotten much longer than I expected.
Questions? comments? Discussion?
Cheers
~Metaforest~
----------------------------------------------------
Any sufficiently advanced technology is indistinguishable from magic - Arthur C. Clark
----------------------------------------------------
Any sufficiently advanced technology is indistinguishable from magic - Arthur C. Clark