2

I'm developing a web front end for a GNU Radio application developed by a colleague.

I have a TCP client connecting to the output of two TCP Sink blocks, and the data encoding is not as I expect it to be.

One TCP Sink is sending complex data and the other is sending float data.

I'm decoding the data at the client by reading each 4-byte chunk as a float32 value. The server and the client are both little-endian systems, but I also tried byte swapping (with the GNU Radio Endian Swap block and also manually at the client), and the data is still not right. Actually it's much worse then, confirming there is no byte order mismatch.

When I execute the flow graph in GNU Radio Companion with appropriate GUI elements, the plots look correct. The data values are shown as expected to between 0 and 10.

However the values decoded at the client are generally around 0.00xxxxx, and the plot looks like noise rather than showing a simple tone as is seen in GNU Radio. If I manually scale the data by multiplying by 1000 it still looks like noise.

I'll describe the pre-D path in GNU Radio since it's shorter, but I see the same problem on the post-D path, where a WBFM Receive and a Rational Resampler are added, followed by a Throttle block and then a TCP Sink block sending float data.

File Source (Output Type: complex, vector length: 1) =>
Throttle (vector length: 1) =>
Low Pass Filter (FIR Type: Complex->Complex (Decimating)) =>
Throttle (vector length: 1) =>
TCP Sink (input type: complex, vector length: 1).

This seems to be the correct way to specify the stream parameters (and indeed Companion shows errors if I make changes which mismatch the stream items), but I can find no way to decode the data correctly on the other end of the stream.

Marcus Müller
  • 34,677
  • 4
  • 53
  • 94
MidnightJava
  • 1,927
  • 2
  • 18
  • 38
  • Why are you using `throttle` blocks, and why even two of them? Throttle does exactly *nothing* to your signal. It just slows down processing. You should never have two of them in the same sample path. – Marcus Müller Aug 18 '16 at 19:50
  • So: bad news first: GNU Radio is preparing for the 3.8 release, and the TCP sinks are deprecated and will be removed. So you might as well avoid those at all, especially since they are pretty horrible in a lot of ways (I can explain if you mail the discuss-gnuradio mailing list). – Marcus Müller Aug 18 '16 at 19:51
  • But: I don't think this is the problem here. What is your decimation factor? – Marcus Müller Aug 18 '16 at 19:51
  • also, do you check whether you received all the data in your TCP client, or whether multiple `recv()` calls are necessary? – Marcus Müller Aug 18 '16 at 19:52
  • 1
    I'd generally recommend switching to the much-more-awesome ZeroMQ network sinks in GNU Radio, and using ZeroMQ in your client; that takes care of the whole packetization thing, which I think might be the issue here. – Marcus Müller Aug 18 '16 at 19:53
  • Thanks for your answers. The waveform is not mine, so I don't know why he has two throttles. Decimation factor for the resampler is 500. I'm doing frame sync on the client (node.js) by counting bytes in the tcp listener and sending one frame at a time for processing. I print to console each float32 value in the tcp listener, and I see the same sort of values there that I'm seeing in downstream processing. I also tried slipping by one, two, or three bytes when reading 4-byte float32 values, and the values are still wrong. – MidnightJava Aug 18 '16 at 20:34
  • I encountered and worked around the blocking on accept() in TCP Sink, which is probably one of the other problems you were thinking about. I'd be glad to use a ZMQ block, but pub/sub and other stuff that ZMQ does is overkill for what I'm doing on the client. Is there a way to use a ZMQ block to just send a tcp stream, so I can consume it with a tcp (not ZMQ) client? I have a simple node.js function which I wrote to receive the tcp data, and I don't want to make it any more complicated. – MidnightJava Aug 18 '16 at 20:37
  • 1
    No, ZMQ does ZMQ, but honestly, PUB/SUB, or REP/REQ (if your client wants to have the role of "requesting" data) isn't overkill - ZMQ is pretty slim, and basically just takes care of data getting from a to be in one piece, so you don't have to mess with the recv()/send() handling. Yep, the whole accept thing is one of the messes that this code is in (also,it's part of the GRC codebase, and stuff) – Marcus Müller Aug 18 '16 at 20:41
  • OK, thanks. I found some ZMQ bindings for node.js (https://github.com/JustinTulloss/zeromq.node), so I guess I can do this pretty quickly. One more thread to pull on: is there a WebSocket block available for Gnu Radio? My client already supports WebSockets, but I implemented a tcp binding because I couldn't find a WebSocket GR block. – MidnightJava Aug 18 '16 at 21:13
  • No, no Websocket block that I'd know of. But: if you go through chapters 1-3 of http://tutorials.gnuradio.org, you could use any websocket python lib to write one :) – Marcus Müller Aug 18 '16 at 21:18
  • I implemented a zmq pub block and coded a sub connection in node.js. I still have the same data representation problem that I did with TCP Sink, so I need to take a closer look at the data and the waveform. But one aspect that is far worse with zmq is the data is coming at about 100th the rate it was coming with TCP Sink. I want to send the data at the rate it's being pushed through the waveform. I don't see any config parameters to control this in Gnu Radio, and nothing in the python API either. Is there a setting I'm missing? I can't imagine zmq is so much slower than TCP Sink. – MidnightJava Aug 19 '16 at 22:24
  • I had the subscribe socket set up wrong, subscribing to "*" to receive all messages. I changed it to "", and now it updates as expected. Clearly "*" was the wrong value, but I wonder why it worked at all, apparently filtering the published messages by some criteria. – MidnightJava Aug 22 '16 at 13:42

2 Answers2

1

"the historic RFC 1700 (also known as Internet standard STD 2) has defined the network order for protocols in the Internet protocol suite to be big-endian , hence the use of the term 'network byte order' for big-endian byte order."

see https://en.wikipedia.org/wiki/Endianness

having mentioned the network order for protocols being big-endian, this actually says nothing about the byte order of network payload itself.

also note: Sun Microsystems made big-endian native byte order computers (upon which much Internet protocol development was done).

i am surprised the previous answer has gone this long without a lesson on network byte order versus native byte order.

GNURadio appears to assume native byte order from a UDP Source block.

Examining the datatype color codes in Help->Types of GNURadio Companion, the orange colored 'float' connections are float32.

To verify a computer's native byte order, in Python, do:

from sys import byteorder
byteorder

the result will be 'little' or 'big'

CSylvain
  • 11
  • 1
0

It might be possible that no matter what type floats you are sending, when bytes get on network they get ordered in little endian. I had similar problem with udp connection, and I solved it by parsing floats as little endian on client side.

charlieamer
  • 81
  • 3