0

I made a simple Python script that should stream the Binance Bitcoin-USD orderbook. I followed the guide here. I subscribed to their websocket stream and started updating my local orderbook. The problem is that the size of the orderbook keeps growing, and i don't know if that's normal. It started from a length of around 1000 rows, after 13 hours it is at around 4000. Is this normal or am i doing something wrong?

Here is how i'm updating the orderbook:

1) Retrieve a copy of the partial orderbook from the API endpoint https://www.binance.com/api/v1/depth?symbol=BNBBTC&limit=1000

2) Take that data, convert it to a dictionary like the following Partial = {'asks:'{...}, 'bids': {...}}, i do this because a dict is easier to update

3) Take every row in the update and update my local dict with the new data using price as the key. Then i make a loop through the dict and delete every row that has value 0.000000

Code:

#Here is the payload received by the websocket stream
Update = message['data']

#Update bids
for x in Update['b']:
    Partial['bids'].update({x[0]: x[1]})

#Update asks
for x in Update['a']:
    Partial['asks'].update({x[0]: x[1]})

#Remove rows where the value is 0
DelBids = ({k:v for k,v in Partial['bids'].items() if v != '0.00000000'})
DelAsks = ({k:v for k,v in Partial['asks'].items() if v != '0.00000000'})

Where Partial is the dictionary where i'm storing the copy of the orderbook i retrieved from the API call (see point 1). Any kind of advice is appreciated!

San9096
  • 231
  • 4
  • 12

1 Answers1

2

well, the size growing and that's normal. As the price changing, you gonna increase the global range of your order book. But there is a problem with your process. In fact, as I understand your process, you receive the partial order book and update with the price as key your local order book. the problem is that you keep track of order that doesn't exist anymore. In fact, when you receive the partial order book, you have to delete in your local order book all the price in the range of the partial data, if not, you keep old order that doesn't exist anymore. If the partial order book go from 9200 to 10200 for example, you have to delete from your local from 9199,99 to 10000,01 for example. you can see that in the api documentation: "Get a depth snapshot from https://www.binance.com/api/v1/depth?symbol=BNBBTC&limit=1000 . Drop any event where u is <= lastUpdateId in the snapshot. The first processed event should have U <= lastUpdateId+1 AND u >= lastUpdateId+1. While listening to the stream, each new event's U should be equal to the previous event's u+1." the main problem with this kind of local order book, is that you can't trust order price/quantities oustside of the range of the partial order book.

Christophe Tharaud
  • 36
  • 2
  • I thought i only had to remove data from the local orderbook when i received an event where the amount is 0.0, only in that case i was removing data. But when should i remove the data outside of the range? When i make the screenshot or while i'm receiving data from the websocket? That's what i don't understand – San9096 Jun 20 '20 at 10:27
  • 1) retreive a copy from the API client.get_order_book(symbol='BTCUSDT', limit=1000) 2) make your Partial dictionary 3) receive update from the stream bm.start_depth_socket('BTCUSDT', process_message) 4) remove from your Partial asks all values in the range of min/max asks of the stream update receive, do the same for the bids 5) update your Partial with the values of the stream update 6) and remove values with ‘0,00000000’ 7) return to step 3 – Christophe Tharaud Jun 21 '20 at 11:58
  • doing like so, your Partial is up to date in the range of the stream update. For a better Partial order book, you can retrieve your datas from the API from time to time, to have a better updated order book. In fact, if there is a big hyperbolic bull run, the risk is that your update from the stream gonna be on top values of your Partial dictionary, instead of normally in the kinda middle range. Depending on the time to compute your dictionary (but should be fast), you can redo all steps from time to time, to keep the stream data in the middle of the copy from the api. – Christophe Tharaud Jun 21 '20 at 11:58
  • I'm sorry for the late answer! Thank you, it's a lot clearer now! Yes, i thought of using only the api to retrieve the orderbook, but the problem is that i would get easily banned for sending too many requests. The webscoekt is a more real-time solution, the only problem is that it's a bit harder to maintain – San9096 Jun 22 '20 at 08:32
  • @ChristopheTharaud I have no idea how you made up the "removing all price/volume levels in range of min/max of the depthupdate". Then why the update itself contains 0.0000 volumes if such a rule exists? What is your source? – Roni Tovi Dec 08 '21 at 14:59
  • @RoniTovi I think it means there are no more orders a given price. – yash Jul 12 '22 at 22:20