Proxmark3 community

bushing

Contributor

Registered: 2008-10-14

Posts: 42

Client working under OS X

I have the Linux client -- at least the text-based version -- working under OS X.  Using a codeless kext, we can prevent the HID driver from grabbing the device and can talk to it normally.   Here is a patch against current SVN trunk; note that the "linux" directory should probably be renamed to "unix".

Index: linux/Info.plist
===================================================================
--- linux/Info.plist  (revision 0)
+++ linux/Info.plist  (revision 0)
@@ -0,0 +1,30 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd ">
+<plist version="1.0">
+<dict>
+  <key>CFBundleDevelopmentRegion</key> <string>English</string>
+  <key>CFBundleIdentifier</key> <string>org.proxmark</string>
+  <key>CFBundleIconFile</key> <string></string>
+  <key>CFBundleInfoDictionaryVersion</key> <string>6.0</string>
+  <key>CFBundlePackageType</key> <string>KEXT</string>
+  <key>CFBundleSignature</key> <string>????</string>
+  <key>CFBundleVersion</key> <string>1.0.0</string>
+  <key>IOKitPersonalities</key>
+  <dict>
+    <key>Proxmark3</key>
+    <dict>
+      <key>CFBundleIdentifier</key><string>com.apple.kernel.iokit</string>
+      <key>IOClass</key><string>IOService</string>
+      <key>IOProviderClass</key><string>IOUSBInterface</string>
+      <key>bConfigurationValue</key> <integer>1</integer>
+      <key>bInterfaceNumber</key> <integer>0</integer>
+      <key>idProduct</key><integer>19343</integer>
+      <key>idVendor</key><integer>39620</integer>
+    </dict>
+  </dict>
+  <key>OSBundleLibraries</key>
+  <dict>
+    <key>com.apple.iokit.IOUSBFamily</key><string>1.8</string>
+  </dict>
+</dict>
+</plist>
Index: linux/proxmark3.c
===================================================================
--- linux/proxmark3.c  (revision 262)
+++ linux/proxmark3.c  (working copy)
@@ -46,21 +46,17 @@
 {
   struct main_loop_arg *arg = (struct main_loop_arg*)targ;
   char *cmd = NULL;
+  pthread_t reader_thread;
 
-  while(1) {
+  if (arg->usb_present == 1) {
     struct usb_receiver_arg rarg;
-    pthread_t reader_thread;
-
     rarg.run=1;
-    if (arg->usb_present == 1) {
-      pthread_create(&reader_thread, NULL, &usb_receiver, &rarg);
-    }
-    cmd = readline(PROXPROMPT);
-    rarg.run=0;
-    if (arg->usb_present == 1) {
-      pthread_join(reader_thread, NULL);
-    }
+    pthread_create(&reader_thread, NULL, &usb_receiver, &rarg);
+  }
 
+  while(1) {
+
+    cmd = readline(PROXPROMPT);
     if (cmd) {
       if (cmd[0] != 0x00) {
         CommandReceived(cmd);
Index: linux/Makefile
===================================================================
--- linux/Makefile  (revision 262)
+++ linux/Makefile  (working copy)
@@ -1,7 +1,7 @@
 #COMMON_FLAGS = -m32
-LDLIBS = -L/usr/local/lib -lusb -lreadline -lpthread
+LDLIBS = -L/usr/local/lib -L/opt/local/lib -lusb -lreadline -lpthread
 LDFLAGS = $(COMMON_FLAGS)
-CFLAGS = -I. -I/opt/local/include -Wall -Wno-unused-function $(COMMON_FLAGS)
+CFLAGS = -I. -I/usr/local/include -I/opt/local/include -Wall -Wno-unused-function $(COMMON_FLAGS)
 
 CXXFLAGS = $(shell pkg-config --cflags QtCore QtGui 2>/dev/null) -Wall
 QTLDLIBS = $(shell pkg-config --libs QtCore QtGui 2>/dev/null)
@@ -34,6 +34,17 @@
 proxguiqt.moc.cpp: proxguiqt.h
   $(MOC) -o$@ $^
 
+# must be run as root
+install_kext: Info.plist
+  mkdir -p /System/Library/Extensions/Proxmark3.kext/Contents
+  cp Info.plist /System/Library/Extensions/Proxmark3.kext/Contents
+  chown -R root:wheel /System/Library/Extensions/Proxmark3.kext
+  chmod 755 /System/Library/Extensions/Proxmark3.kext /System/Library/Extensions/Proxmark3.kext/Contents
+  chmod 644 /System/Library/Extensions/Proxmark3.kext/Contents/Info.plist
+  rm -rf /System/Library/Caches/com.apple.kext.caches
+  touch /System/Library/Extensions
+  @echo "*** You may need to reboot for the kext to take effect."
+
 clean:
   rm -f cli flasher proxmark3 snooper *.o *.moc.cpp
 

Index: common/Makefile.common
===================================================================
--- common/Makefile.common  (revision 262)
+++ common/Makefile.common  (working copy)
@@ -14,19 +14,14 @@
 #  Windows yields literal "", on Linux yields an empty line
 ifeq ($(shell echo ""),)
 
-# This is probably a proper system, so we can use uname
-UNAME := $(shell uname)
-ifeq ($(UNAME), Linux)
-# Linux. (Todo: Add MacOS X if appropriate)
 DELETE=rm -rf
 MOVE=mv
 COPY=cp
 PATHSEP=/
 FLASH_TOOL=linux/flasher
-DETECTED_OS=Linux
+DETECTED_OS=Unix
 # You may/should set this in your environment
 ARMLIB ?= /usr/local/lib/gcc/arm-elf/4.3.3/interwork
-endif
 
 else
 
@@ -41,7 +36,7 @@
 
 endif
 
-CROSS   = arm-elf-
+CROSS ?= arm-elf-
 CC  = $(CROSS)gcc
 AS  = $(CROSS)as
 LD  = $(CROSS)ld
Index: Makefile
===================================================================
--- Makefile  (revision 262)
+++ Makefile  (working copy)
@@ -1,6 +1,6 @@
 include common/Makefile.common
 
-ifeq ($(DETECTED_OS),Linux)
+ifeq ($(DETECTED_OS),Unix)
 HOST_BINARY=linux
 else
 HOST_BINARY=winsrc

bushing

Contributor

Registered: 2008-10-14

Posts: 42

Re: Client working under OS X

Actually, any command which tries to download samples (e.g. losamples) will not work, it will claim the response is unknown.  I don't know how to fix this without rewriting huge swaths of code (which I'm willing to do, but I don't have any easy way to test to see if my changes break the Windows client ... and it's not going to be a small patch).

edo512

Contributor

Registered: 2008-10-07

Posts: 103

Re: Client working under OS X

-> Still It would be great it you could do regression testing agains both Windows & Linux, of course. I'm sure some will be willing to test your patch against Windows, I can test on Linux...

bushing

Contributor

Registered: 2008-10-14

Posts: 42

Re: Client working under OS X

Okay, I did some fairly major surgery on the client, the results are here: http://repo.or.cz/w/proxmark3.git

Changes of note:
* Combined winsrc and linux into one dir, client
* Switched from MS command line compiler to MinGW for Windows
* Changed from using .cpp files in most cases to .c files -- they were only cpp files because the MS compiler doesn't support C99, it wasn't actually C++ code
* Code cleanup of the main command.c
* Got rid of all the "translate.h" cruft
* Reworked the USB event processing so that it would actually work under OS X -- I don't understand how this could have worked properly on Linux

I've tested that it builds (and runs!) on XP (using MinGW), Linux and OS X.  The GUI works on Windows using MinGW; it works on OS X using qt4-mac from MacPorts.   The QT GUI *should* work on Linux, but for some reason it segfaults when trying to bring up the plot window -- the client works other than that.  Someone who knows QT will have to look more deeply into that.

Patches greatly appreciated, etc.   If you know how to use git, then you can actually anonymously push to the "mob" branch on that git repo.

bushing

Contributor

Registered: 2008-10-14

Posts: 42

Re: Client working under OS X

I've now committed my changes to the Google Code SVN to merge the Windows and "Linux" clients.  It should now build and run on all three platforms.

On the Mac, I tested it on a machine running SnowLeopard; I used libusb and qt4-mac from MacPorts.   You can run "sudo make install_kext" to install a codeless kext to prevent the HID driver from attaching to the device.

henryk

Contributor

Registered: 2009-07-27

Posts: 99

Re: Client working under OS X

Great work!

Also nice job on the -print-libgcc-file-name, I didn't know about that before. This finally makes having an additional environment variable unnecessary.

The revamped receive code invalidates prior assumptions on the screen handling though, so now commands like hexsamples produce less than desirable effects (a prompt is printed every 6 lines and offsets the output). Previously the USB receive loop, upon receiving a packet, removed the prompt, processed the packet (which maybe printed some things), then printed the prompt again. Now "processing the packet" just means storing the data for later processing, so printing the prompt and actually processing/printing the data are out of sync. (Also, I sense that there's a race condition hiding in there somewhere.)

This should be replaced by a proper queuing system for responses. And the prompt handling should be removed from the USB receive code (it doesn't belong there anyway) and be handled somewhere else. We probably need a pair of functions with the semantics of "I may start printing to the screen now, please remove the prompt" and "I'm done with the screen, you may redraw the prompt" and then bracket all parts of the code that might print in calls to these two functions.

bushing

Contributor

Registered: 2008-10-14

Posts: 42

Re: Client working under OS X

Great work!

*phew* I was really worried I was going to hear that I broke everything for everyone else, thanks

The revamped receive code invalidates prior assumptions on the screen handling though, so now commands like hexsamples produce less than desirable effects (a prompt is printed every 6 lines and offsets the output). Previously the USB receive loop, upon receiving a packet, removed the prompt, processed the packet (which maybe printed some things), then printed the prompt again. Now "processing the packet" just means storing the data for later processing, so printing the prompt and actually processing/printing the data are out of sync. (Also, I sense that there's a race condition hiding in there somewhere.)

I really don't understand how this worked before, and it shows -- thanks for correcting my mistake.  The comment in (the old) command.cpp:

// Entry point into our code: called whenever we received a packet over USB
// that we weren't necessarily expecting, for example a debug print.
void UsbCommandReceived(UsbCommand *c) {

When I ran this code under OS X, all packets got funneled to this function (in effect, we were never "expecting" any responses), which caused commands that did expect a response to fail.  The fact that we were tearing down and recreating the reader thread for each command seemed odd to me, so that's why I made the changes I made.

This should be replaced by a proper queuing system for responses. And the prompt handling should be removed from the USB receive code (it doesn't belong there anyway) and be handled somewhere else. We probably need a pair of functions with the semantics of "I may start printing to the screen now, please remove the prompt" and "I'm done with the screen, you may redraw the prompt" and then bracket all parts of the code that might print in calls to these two functions.

I ran into Roel today at CCC and spoke with him about this for just a little bit, hopefully we can continue the conversation.   There's a lot of work that I would like to see done on the Proxmark software (both the host and device-side) -- much of which I will happily volunteer to do myself! -- but I'd like to get a bit of consensus on how to proceed, since there are a few different approaches that can be taken here.

1) The simplest fix for this immediate problem would be to clear the prompt before printing anything (say, debug output) to the screen, then redraw it.  Anything that involves "bracket all parts of the code that might..." seems a bit fragile.

2) In general, we have a problem where we have a big long list of commands that operate in different ways that are not necessarily obvious from their name or help text.  For example:

Most commands just send a packet to the device; any response will come back as debug "print text" packets from the device.  Some (mainly 'tune') will take a special response packet type, interpret it, and print that.  Others expect that earlier commands will have put data in the sample buffer, and they may either process this data and display some interpretation, or process that data and put it back into the buffer (e.g. any demod function).  Others will send a command that takes the existing sample buffer and uses that as an input.  It's hard to really sort out what does what; we really need some sort of interface that defines what commands operate on what data, and how they report status.

3) We have too many commands, in general. As someone who doesn't really understand RFID, but wants to try to read a HF tag -- do I want to use 'hi14areader', 'hi14read', 'hi14bdemod', 'hi15demod', 'hi15read' or 'hi15reader'?  Why is there no 'hi14breader' or 'hi14ademod'?  etc.  Ideally, we'd have just a few general commands -- detect reader, detect card, snoop traffic between card and reader, read card, send raw commands to card, spoof card.  As a user, I shouldn't have to -- and often cannot -- tell the software exactly what kind of environment to expect (what kind of card or reader, what the existing contents of the sample buffer will be, etc) -- it should be able to make a good guess about this, and perhaps allow for a manual override.  (This is one of the main things that I have no idea how I would even approach, from an RF-processing standpoint).

4) Carrying #3 further, do we even want to keep the same command-response model?  Maybe we want to switch to a 2-paned gui, where there's a command line and a log window (like the Windows client, or an IRC client).   Maybe we want most functionality to be hidden behind big, brightly colored "read / spoof" buttons.   I don't know, I'm just not sure that the current model is the cleanest one.

There are also issues of how analysis work is split between the host and the device, and how exactly the data is sent over USB (do we keep the same current model, and add support for larger (bulk?) packets?   or do we go the other direction and use a "USB Serial" (CDC) model, where we have dumb bytes being sent back and forth at relatively high speed?

I'm starting to ramble, so I'll stop here.  I think it would be good for us to decide on what an ideal interface style would look like (or would we have two -- one for users, one for development?) -- and then work out a path to get there.

Any thoughts on this would be welcome, even from non OS X users.   (Maybe I should start a new thread...)

alex

Member

Registered: 2009-12-31

Posts: 2

Re: Client working under OS X

Hi bushing, I am a little lost, maybe you can share some light on this.
I am trying to get the pm3 recognized by Virtualbox under Snow Leopard.

You can run "sudo make install_kext" to install a codeless kext to prevent the HID driver from attaching to the device.

I 've copied the Info.plist from the SVN into an Folder called libusbshield.kext and moved this one to /System/Library/Extensions/ .
I 've waited for the extcache update and pluged the pm3 into the usb port, but the extension got rejected.

thy alex

bushing

Contributor

Registered: 2008-10-14

Posts: 42

Re: Client working under OS X

Hi bushing, I am a little lost, maybe you can share some light on this.
I am trying to get the pm3 recognized by Virtualbox under Snow Leopard.

I'm afraid I don't understand, can you please provide some more information on your setup?  Are you running SnowLeopard as the host -- if so, what OS are you running under VirtualBox?   If you are running SnowLeopard as the guest (not even sure if this is possible), then what OS are you running as your host?

You can run "sudo make install_kext" to install a codeless kext to prevent the HID driver from attaching to the device.

I 've copied the Info.plist from the SVN into an Folder called libusbshield.kext and moved this one to /System/Library/Extensions/ .
I 've waited for the extcache update and pluged the pm3 into the usb port, but the extension got rejected.

I strongly recommend you use the Makefile to do this, instead of doing it manually.  If you look at the install_kext target, then you'll see that it sets up very specific permissions and ownership, and it also points out that you need to reboot the machine before the kext can take effect.   (Yes, this is unusual for a kext, but codeless kext support seems to have some quirks.)

alex

Member

Registered: 2009-12-31

Posts: 2

Re: Client working under OS X

Happy new year!
@bushing:
Here is my setup, I use:
-OSX 10.6.2  (Host)
-Ubuntu Karmic (VM guest)
-VirtualBox (VM software)

I also installed kext via "sudo make install_kext", rebooted the machine.

My problems:

-If I configure Vmbox for grabbing all USB devices, pm3 will get recognized by the guest & proxmark3 client. Unfortunately, the proxmark client is reporting "unrecognized command d18a4980" and seems unusable.

-My OSX Konsole reports "The IOUSBFamily is having trouble enumerating a USB..."

It would be grate to be able to attach & detach the pm3 to every VM I like.

iZsh

Contributor

Registered: 2010-01-02

Posts: 95

Re: Client working under OS X

I ran into Roel today at CCC and spoke with him about this for just a little bit, hopefully we can continue the conversation.   There's a lot of work that I would like to see done on the Proxmark software (both the host and device-side) -- much of which I will happily volunteer to do myself! -- but I'd like to get a bit of consensus on how to proceed, since there are a few different approaches that can be taken here.

1) The simplest fix for this immediate problem would be to clear the prompt before printing anything (say, debug output) to the screen, then redraw it.  Anything that involves "bracket all parts of the code that might..." seems a bit fragile.

2) In general, we have a problem where we have a big long list of commands that operate in different ways that are not necessarily obvious from their name or help text.  For example:

Most commands just send a packet to the device; any response will come back as debug "print text" packets from the device.  Some (mainly 'tune') will take a special response packet type, interpret it, and print that.  Others expect that earlier commands will have put data in the sample buffer, and they may either process this data and display some interpretation, or process that data and put it back into the buffer (e.g. any demod function).  Others will send a command that takes the existing sample buffer and uses that as an input.  It's hard to really sort out what does what; we really need some sort of interface that defines what commands operate on what data, and how they report status.

3) We have too many commands, in general. As someone who doesn't really understand RFID, but wants to try to read a HF tag -- do I want to use 'hi14areader', 'hi14read', 'hi14bdemod', 'hi15demod', 'hi15read' or 'hi15reader'?  Why is there no 'hi14breader' or 'hi14ademod'?  etc.  Ideally, we'd have just a few general commands -- detect reader, detect card, snoop traffic between card and reader, read card, send raw commands to card, spoof card.  As a user, I shouldn't have to -- and often cannot -- tell the software exactly what kind of environment to expect (what kind of card or reader, what the existing contents of the sample buffer will be, etc) -- it should be able to make a good guess about this, and perhaps allow for a manual override.  (This is one of the main things that I have no idea how I would even approach, from an RF-processing standpoint).

4) Carrying #3 further, do we even want to keep the same command-response model?  Maybe we want to switch to a 2-paned gui, where there's a command line and a log window (like the Windows client, or an IRC client).   Maybe we want most functionality to be hidden behind big, brightly colored "read / spoof" buttons.   I don't know, I'm just not sure that the current model is the cleanest one.

There are also issues of how analysis work is split between the host and the device, and how exactly the data is sent over USB (do we keep the same current model, and add support for larger (bulk?) packets?   or do we go the other direction and use a "USB Serial" (CDC) model, where we have dumb bytes being sent back and forth at relatively high speed?

I'm starting to ramble, so I'll stop here.  I think it would be good for us to decide on what an ideal interface style would look like (or would we have two -- one for users, one for development?) -- and then work out a path to get there.

Any thoughts on this would be welcome, even from non OS X users.   (Maybe I should start a new thread...)

bushing you might want to repost that in the firmware dev->General (USB driver, Framework, Protocol) part,insofar as it would get lost in the middle of OS X issues in here. Especially since I'll probably throw my couple of cents too.

As you know I started to rewrite the proxmark code (in a top-bottom / pc-fpga swipe algorithm way ) and created a branch for it. From what I saw with the current code, here are my first thoughts about what I'm going to change first in my sandbox:

the asynchronous usb messages from the arm code is kind of nice to be able to output debugging messages whenever we want without having to change anything on the pc/client side, but it makes it very confusing/hard to read (because of non-linear reading of the code)/hard to maintain for normal commands. So I think I'm going to have 2 out endpoints : one for the debugging/asynch messages, and one for bulk messages such that, normal commands will have a classic send message/recv reply code. Easier to read IMHO.

I'm going to move any signal computations on the arm side, such as the demod commands for instance. In general I think the client should just be a "stupid" client which just sends order to the proxmark. That would enable more standalone features with the proxmark. Of course we should still have some commands to extract raw data from the proxmark whenever we want to do signal analysis while reversing unknown stuff (it might even be useful to be able to save the data in format compatible with gnuradio. Just a side-though), but once something is known, I think it would be best if it's implemented on the arm side. the flash size might be an issue at some point, but I don't think we're there yet.

The way we do some stuff on the fpga side, and some other stuff on the arm side is very fuzzy. I kind of understand the argument being made in the documentation, but I think there's a way to factorize the code and make it more consistent. I need to put more thought on that, but for the moment, in my top-bottom swipe, I'm far away from reaching the fpga code. I already have quite some work rewriting the whole client code

Anyone is free to give advices or 2 cents, I'll try to pick here and there a common ground between all the points which will be made, and come up with some code and design that hopefully, not only I like, but other ppl will find more pleasant to work with than the current code. (there's seriously some insane nasty bugs in the code I've looked on the client side...). I'll work in a branch anyway, so even if you guys hate what I come up with even more than the current code, it won't really matter ^^

bushing

Contributor

Registered: 2008-10-14

Posts: 42

Re: Client working under OS X

Happy new year!
@bushing:
Here is my setup, I use:
-OSX 10.6.2  (Host)
-Ubuntu Karmic (VM guest)
-VirtualBox (VM software)

Happy new year to you too

Well ... stepping back for a second, the whole point of this thread was that you should not need to run the software under Linux under VBox, the client is now fully functional under OS X.  The codeless kext is there to make it work with libusb under OS X; it should have no effect on VBox. (should not break anything, but will not help either)

I also installed kext via "sudo make install_kext", rebooted the machine.

My problems:

-If I configure Vmbox for grabbing all USB devices, pm3 will get recognized by the guest & proxmark3 client. Unfortunately, the proxmark client is reporting "unrecognized command d18a4980" and seems unusable.

Odd... Unfortunately, I've never tried it with VBox, I can give it a try when I get home in a week, but you really shouldn't need to use it (see above).

-My OSX Konsole reports "The IOUSBFamily is having trouble enumerating a USB..."

That sounds more like a hardware problem; when does that happen?  (If that happens, is it visible in USB Prober.app?)

d18c7db

Contributor

Registered: 2008-08-19

Posts: 292

Re: Client working under OS X

iZsh, this has come up before in this forum, the bit about moving demod commands on the ARM. In most instances there is already ARM code to demod and there is also client side code to also demod (but perhaps using a different algorithm). This apparent duplication serves a purpose in that you can use the client in offline mode to play with pre-recoderd waveforms and demod them without needing the PM3 connected.

iZsh

Contributor

Registered: 2010-01-02

Posts: 95

Re: Client working under OS X

Why not using something more powerful for offline analysis, such as gnuradio (or matlab) for that purpose then?
Or if we really want to insist on re-implementing it ourselves, wouldn't it better to factorize the code in such a way that we use the same source code for both the ARM side and the client side?

robertcu

Member

Registered: 2009-02-22

Posts: 4

Re: Client working under OS X

can some post a more detailed guide to getting the client running under osx