hf list and raw parameter info and other commands

asper · 2015-01-28 11:11:45

I have problems running the hf list commands with the raw parameter, can someone explain me how it works ?
The help is not "helping" too much:

proxmark3>  hf list
List protocol data in trace buffer.          
Usage:  hf list [14a|14b|iclass] [f]          
    14a    - interpret data as iso14443a communications          
    14b    - interpret data as iso14443b communications          
    iclass - interpret data as iclass communications          
    raw    - just show raw data          
    f      - show frame delay times as well

__________________________________________________________________

lf hid fskdemod / data fskhiddemod <-- are them the same ?
__________________________________________________________________

lf io fskdemod / data fskiodemod <-- are the same ?
__________________________________________________________________

a brief explanation of "[clock] [invert<0|1>]" parameters ? (ranges, functions, ecc) The invert can be 0 or 1 ? Or automatically switch 0 <--> 1 ?
__________________________________________________________________

Last edited by asper (2015-01-28 11:53:17)

marshmellow · 2015-01-29 01:32:39

lf hid fskdemod / data fskhiddemod <-- are them the same ?

they are not exactly the same.
lf hid fskdemod reads and demods the tag in the arm directly and reads continuously until the button is pressed.
data fskhiddemod takes the graphbuffer and hid demods it. (for offline mode)

lf io fskdemod / data fskiodemod <-- are the same ?

same explanation as above.

explain [clock] [invert<0|1>] parameters

invert, in most cases it can be set to 1 or it assumes 0 if omitted. (1 meaning yes invert the data)
in the indala demod it automatically tests the demoded data and inverts if needed.

[clock] (by itself) should always refer to the bit clock (or how many samples = 1 bit)
we currently only support demoding the bit clocks that the ATA55x7 supports: 8, 16, 32, 40, 50, 64, 128. (note that 8 is very difficult to demod...)

(not to be confused with the Field Clocks or the carrier clock)

holiman · 2015-01-29 01:52:03

asper wrote:

I have problems running the hf list commands with the raw parameter, can someone explain me how it works ?
The help is not "helping" too much:

proxmark3>  hf list
List protocol data in trace buffer.          
Usage:  hf list [14a|14b|iclass] [f]          
    14a    - interpret data as iso14443a communications          
    14b    - interpret data as iso14443b communications          
    iclass - interpret data as iclass communications          
    raw    - just show raw data          
    f      - show frame delay times as well

It should say

proxmark3>  hf list
List protocol data in trace buffer.          
Usage:  hf list [14a|14b|iclass|raw] [f]          
    14a    - interpret data as iso14443a communications          
    14b    - interpret data as iso14443b communications          
    iclass - interpret data as iclass communications          
    raw    - just show raw data          
    f      - show frame delay times as well

or perhaps

proxmark3>  hf list
List protocol data in trace buffer.          
Usage:  hf list <protocol> [f] 
    f      - show frame delay times as well      
protocols:         
    14a    - interpret data as iso14443a communications          
    14b    - interpret data as iso14443b communications          
    iclass - interpret data as iclass communications          
    raw    - just show raw data without annotations

Does that make more sense ?

holiman · 2015-01-29 01:57:58

New functionality added in my 'lf_recorder' - branch.

proxmark3> lf read b 2 d 8
Sampling config:           
   divisor:    0           
   bps:        2           
   decimation: 8           
   averaging:  1           
OBS, this is sticky on the device and affects all LF listening operations          
To reset, issue 'lf read'          
#db# Sampling config:                  
#db#    divisor:    0                  
#db#    bps:        2                  
#db#    decimation: 8                  
#db#    averaging:  1                  
Waiting for a response from the proxmark...          
Don't forget to cancel its operation first by pressing on the button          
#db# Done, saved 159996 out of 1279968 seen samples at 2 bits/sample                 
#db# buffer samples: a6 a5 55 9a 95 55 5a 95 ...                 
proxmark3> data samples 40000
Reading 40000 bytes from device memory
          
Data fetched          
Samples packed at 2 bits per sample          
Unpacking...          
Unpacked 160000 samples

So, the example above recorded data from 1279968 samples, that is, roughly ten seconds. The amazing thing is, it's still decipherable. At 2 bits per sample, the quantization level is pretty high, so in order for that to make sense, you need a good strong signal to begin with. It then decimates by 8, keeping one sample in eight. However, the sample value is averaged, so if e.g. there's an ASK-encoded signal with 32/fc clock, that should still give ~4 discernible peaks. Even when decimating at 1/32 samples it could be noticeable, but you'd probably need more bits per sample then.

Anyway, the code is up for testing.

Edit: See the example above? We fill the buffer (but only up to 159996 samples). When fetching all 40000 bytes, it gets unpacked to 160000. That means there are 4 'stray' samples on the far right - you'll need to rtrim them before you do any 'data norm' or stuff like that.

Last edited by holiman (2015-01-29 02:01:28)

marshmellow · 2015-01-29 05:55:11

does this work only for lf read or did you apply this to the lf snoop as well?

also is there a reason we only use 8 bits of bigbuf instead of all 32?

holiman · 2015-01-29 08:40:40

As of right now, snoop resets the settings, so only read. And no, all bits are used.

Last edited by holiman (2015-01-29 08:41:22)

asper · 2015-01-29 20:43:12

Thank you for the answers !

About the hf list, I think this one will be better:

proxmark3>  hf list
List protocol data in trace buffer.          
Usage:  hf list <protocol> [f] 
    f      - show frame delay times as well      
protocols:         
    14a    - interpret data as iso14443a communications          
    14b    - interpret data as iso14443b communications          
    iclass - interpret data as iclass communications          
    raw    - just show raw data without annotations

I will update the GUI as soon as I can and probably release a new "unstable" version so that more people can test all the new fantastic added functions

If I find other params "interpretation" problems I will write them here. Thank you again guys !

Last edited by asper (2015-01-29 20:44:00)

gaucho · 2015-01-29 21:05:11

Forgive me if i ask stupid questions about command "hf list raw f" :

Is the command "hf list raw f" a new command?
Is it able to record traffic between a reader and a tag?
which is the sampling frequency of this command?
Is it possible to add the options (in case of raw data sampling) to choose sampling frequency, sampling bits for each sample, offset from trigger?

Thank you

Last edited by gaucho (2015-01-29 21:06:30)

holiman · 2015-01-29 21:26:09

gaucho wrote:

Forgive me if i ask stupid questions about command "hf list raw f" :
Is the command "hf list raw f" a new command?
Is it able to record traffic between a reader and a tag?
which is the sampling frequency of this command?
Is it possible to add the options (in case of raw data sampling) to choose sampling frequency, sampling bits for each sample, offset from trigger?
Thank you

I have been mixing and matching in this thread (the title is '... and other commands'.). So, to summarize work in different branches:

* [Generic_tracing] Moving iso14443b into using 'generic' traceformat. This means that 'hf list <proto>' is used instead of 'hf mf list' or 'hf 14b list' etc.
- Along with this, the new trace-listings are annotated according to protocol. The implemented protocols are iclass, 14a and 14b.
- Also, CRC-checks are made according to protocol
- The mode 'raw' just means that no annotations are written, and no crc-checks are made.
* [lf_recorder] has nothing to do with *tracing*, only *sampling*. So, you can sample and specify
- bits per sample
- decimation (1 in N samples)
- averaging ON/OFF (only makes sense when decimating)
- the basic sampling freq is as usual, default 125khz, 'H'->134, or any divisor
- Also threshold as before
- Offset from trigger, not implemented yet. How would you like to define such a trigger? Or do you mean after threshold has been triggered?

The functionality works for 'lf read', but is not yet final. Let me know if you have further suggestions.

asper · 2015-01-30 07:58:32

Iso15693 cannot be fully emulated/snooped/listed (for now this is possible only using iclass) because the full protocol has differencies from iclass. If someone is able to implement the full protocol (different bitrates, etc) I can provide docs. I think this is what gaucho needs, a full iso15693 implementation in pm3.

gaucho · 2015-01-31 10:29:10

holiman wrote:

- The mode 'raw' just means that no annotations are written, and no crc-checks are made.

Does it means that the analog hf path is simply digitized?
Or some more digital filter is applied on the rf signal?
We need simple digitizing of the data for new systems study, also before to know the used protocol, especially during firmware integration phase. The better solution it could be to have inspectable processing chain, in order to output the processed signal on various points of the processing chain. It seems to be exacly what you started to do.

holiman wrote:

* [lf_recorder] has nothing to do with *tracing*, only *sampling*. So, you can sample and specify
- bits per sample
- decimation (1 in N samples)
- averaging ON/OFF (only makes sense when decimating)
- the basic sampling freq is as usual, default 125khz, 'H'->134, or any divisor
- Also threshold as before
- Offset from trigger, not implemented yet. How would you like to define such a trigger? Or do you mean after threshold has been triggered?

I'm not personally interested in lf_recorder since i'm quite sure that it uses lf analog path, and not hf analog path.
So i think that hf signals are filtered out.
When i talk about trigger delay, of course i talk about number of samples to ignore after the threshold has been triggered. This is needed because the memory on pm is not enough to store the entire communication between a reader and a tag.

holiman · 2015-01-31 19:23:23

Regarding LF-recording, more info, pictures and discussion here: https://github.com/Proxmark/proxmark3/pull/54

holiman · 2015-01-31 20:25:16

gaucho wrote:

holiman wrote:
- The mode 'raw' just means that no annotations are written, and no crc-checks are made.
Does it means that the analog hf path is simply digitized?
Or some more digital filter is applied on the rf signal?
We need simple digitizing of the data for new systems study, also before to know the used protocol, especially during firmware integration phase. The better solution it could be to have inspectable processing chain, in order to output the processed signal on various points of the processing chain. It seems to be exacly what you started to do.

It just means that, if someone implements a protocol on the ARM side, and uses the common tracer-functions to log data, he can use 'hf list raw' to see the data, without implementing a 'list' function again from scratch. So sorry, not what you wanted..

Regarding HF recording, I don't think the USB speed is the big blocker. I think 40Kb could be quite useful, at 4 bits per sample that would be 80K samples. Add some decimation to that, and the captured signal could be useful to see quite a lot of data. Someone just need to implement for HF what's already there for LF. Enio's already done it, no? Is there any code laying around somewhere from that?

Last edited by holiman (2015-01-31 20:25:45)

gaucho · 2015-02-01 11:03:15

@holiman, i think that enio is busy and he had no time to complete the function.
moreover i think that the recorder should be unified between lf and hf since digitizing functions shall accept parameters so that they are the same.
one of the parameters should be "hf path" or "lf path". the others could be: sampling frequency, averaging "on" or "off" (in case of low sampling frequency), bits per sample, trigger level, delay from trigger, used protocol

do you agree with me?

we can not write a new function for each need, reuse is the keyword.

if we were a small software house, we will have written first a specification in Word format, before to start writing lines of code.
Anyway I understand that with opensource projects is hard to coordinate developers.

holiman · 2015-02-01 11:21:00

@gaucho, it's good with some ideas and suggestions. I'll look at enio's code and see what can be used, if it should be modified. I saw that he had put it in lf fpga-path due to lack of major modes, and he was unsure about the fpga code. I'll look at it and we'll see where it goes. no promises...

I don't think lf/hf will share paths. The hard part (for me, at least) is implementing the fpga part. Once that has been done, it's not so difficult to fix the arm-part, trigger delay averaging etc. What do you mean by protocol? It's only sampling.

gaucho · 2015-02-01 14:33:37

i mean that if the protocol is known, giving it to the function it could be possibile to identify the received message and tell that it is in example a INVENTORY message received from the reader. In other terms, if this helps to reduce the amount of data stored on the memory, it should be implemented in the arm, otherwise it could be implemented in the client side, in order to give a clear explanation about the recording to to user. similar to what it is done by wireshark for ethernet protocols. anyway i would leave it there for future implementation, if it is not yet implementable.

you said that lf and hf paths will not share the same path, but i will try to be more specific cause may be i didn't understood your sentence:
hf and lf analog paths uses filters in order to demodulate received signal.
once selected the analog path, the signal arrives to the digitizer (there is only one path for sampling).
the received signal is sampled, and the main difference between recording a low frequency signal or high frequency signal it is just the sampling frequency.
what i'm telling you is that function for RECORDING the samples inside the fpga should receive parameters like sampling frequency, decimation etc. in order for the FPGA to be able to reply to the ARM with the requested data, according to requested format.
On the ARM side, again we need a single "FLOW" composed by 2 functions (one for request data and one to receive data from fpga) FOR RECORDING purposes.
It doesn't matter if it is HF or LF, since it should be just a parameter.
Is it there something wrong in this?
Is it something similar to what it is implemented?

p.s: about the number of parameters of the command on client side, I don't think that the number of parameters (if they are optional and with default value declared on the usage description) are a problem for the user. consider that in most cases when you disconnect the usb cable from the computer, the pm3 looses power and forgets everything.

Last edited by gaucho (2015-02-01 14:45:19)

Proxmark3 community

Announcement

#1 2015-01-28 11:11:45

hf list and raw parameter info and other commands

#2 2015-01-29 01:32:39

Re: hf list and raw parameter info and other commands

#3 2015-01-29 01:52:03

Re: hf list and raw parameter info and other commands

#4 2015-01-29 01:57:58

Re: hf list and raw parameter info and other commands

#5 2015-01-29 05:55:11

Re: hf list and raw parameter info and other commands

#6 2015-01-29 08:40:40

Re: hf list and raw parameter info and other commands

#7 2015-01-29 20:43:12

Re: hf list and raw parameter info and other commands

#8 2015-01-29 21:05:11

Re: hf list and raw parameter info and other commands

#9 2015-01-29 21:26:09

Re: hf list and raw parameter info and other commands

#10 2015-01-30 07:58:32

Re: hf list and raw parameter info and other commands

#11 2015-01-31 10:29:10

Re: hf list and raw parameter info and other commands

#12 2015-01-31 19:23:23

Re: hf list and raw parameter info and other commands

#13 2015-01-31 20:25:16

Re: hf list and raw parameter info and other commands

#14 2015-02-01 11:03:15

Re: hf list and raw parameter info and other commands

#15 2015-02-01 11:21:00

Re: hf list and raw parameter info and other commands

#16 2015-02-01 14:33:37

Re: hf list and raw parameter info and other commands

Board footer