As I have mentioned previously, I’m working to port OpenBSD’s net80211 layer, which is an IEEE 802.11-2007 WiFi stack, to Mac OS X.
Haiku, an open-source operating system, managed to do this last year for their own kernel. So why is it taking us so long to do a similar thing for OS X?
To start off, although it appears that the OS X kernel being partly based on BSD should make things easier, it actually makes it more complicated to attempt a port such as this. Since almost half of the OS X kernel consists of FreeBSD code, there are lots of symbols predefined, like ifnet and mbuf. Apple’s own “Kernel Programming Interface” (KPIs) provide access to these data structures and functions through a slightly different API which is designed to prevent direct access. Unfortunately, most of the code from other BSD kernels heavily relies on direct access (most notable being the mbuf handling routines). To top it off, several of these data structures have been slightly modified in the XNU kernel compared to their BSD counterparts, making their usage very risky.
Hence, simply including the source code for net80211 and some compatibility glue code (like Haiku does) would result in massive naming collisions between predefined structures and functions in the kernel vs. those redefined in the compatibility layer.
Apple already includes a port of the net80211 stack for their drivers for the Atheros wireless adapters. Without access to that source code, it is difficult to figure out (at least for me) what approach they have taken. My approach has been to convert the entire net80211 source code to C++ and wrap them in a namespace. This will automatically cause all symbols to be prefixed with the namespace, hopefully avoiding collisions with their kernel counterparts.
This sounds easy enough, but for some things it just doesn’t work. The mbuf handling routines in net80211 make heavy use of direct access to the mbuf structure. This is not possible in OS X (or not easily possible). OS X drivers use the mbuf_t structure and the mbuf KPI to manipulate mbufs. Now, mbufs are used to store all network data coming in and going out of the system. Thus, it is used extremely frequently. The two ways to deal with this were to either:
Re-implement BSD’s mbuf API and structures in our compatibility layer. Bad option in my opinion because of the naming collisions, and more importantly, from a performance point of view, and being badly integrated with OS X’s networking tools.
Wade through the hundreds of thousands of instances where mbufs have been used in net80211 source code, and convert them to their mbuf_t KPI equivalents.
I chose the 2nd option as that was the only one which made sense, despite being extremely lengthy. I was actually recommended this by Apple kernel engineers on the darwin-drivers mailing list.
So, the end result was to use Xcode’s “Find” feature to find every single mbuf instance, and replace it with their mbuf_t equivalent, after understanding its context. This was frustrating. But thankfully it’s done. Here’s an example snippet:
And here’s the list of files which needed to be manually reviewed line by line (compiler errors are not always useful) to make sure every instance was properly converted:
The other issue was that a lot of functionality from OpenBSD (e.g. kernel cryptography) is not present in OS X (it probably is present, but inaccessible from the KPIs). So I included all of those. There were also several undefined symbols which I defined as equivalents. All in all, there is a compatibility layer for the compatibility layer:
For what it’s worth, there probably is a much better and more elegant way to do this. But sadly I am not knowledgeable enough to figure out how Apple/Atheros developers did it. How I wish I could at least get a glimpse of their source code! Ralink probably also uses the net80211 stack (or something similar) in the OS X drivers for their USB wifi adapter range. A few days ago I tried to contact Ralink to get access to their source code (in return for making their drivers Airport compatible), but have not received any reply yet.
Anyway, with most of this mechanical porting done, what remains is to write code to glue the net80211 information/function calls to IOKit’s “Airport” interface, which is referred to as apple80211 in Kernel.framework from Leopard and older. Incidentally, Apple stopped supplying this particular set of header files from Snow Leopard onwards. Recreating the proper headers for Snow Leopard was a challenge which deserves a post of its own. Maybe later.
So that’s a survey of the process and the difficulties so far. This week I’ll be working to finish off the “compatibility layer” for net80211 based drivers to access the hardware, and finally hooking it all up to parts of VoodooWireless so it can talk to Airport. Hopefully this approach will work. As is evident, I have not tested even a single line of all this code yet - simply because there are millions of nuts and bolts that all must fit in together before the whole thing will fly. This month I’m hoping it’ll all start to take shape. If not, well tough luck to us.
Postscript: I mainly wrote this post so that future developers who might want to pick up on this project after I leave, can get an idea of what work is being done and how it’s intended to work. Feel free to write to me at voodoo@mercurysquad.com if you have more technical questions (don’t write to ask “how much longer” or “it doesn’t work”, though).
Thanks for reading!