How to get the historical data from bitfinex.com with out a limit?

Question

I am drawing a chart using the data pulled from bitfinex.com via a simple API query. As the result, i will need to render a chart which is going to show the historical data of BTCUSD for the past two years. Docs are available right here: https://bitfinex.readme.io/v2/reference#rest-public-candles Everything works fine except the limit of the retrieved data.

This is my request: https://api.bitfinex.com/v2/candles/trade:1h:tBTCUSD/hist?start=1514764800000&sort=1

The result can be seen over here or you can copy the request to the browser: https://docs.google.com/document/d/1sG11Ro0X21_UFgUtdqrlitcCchoSh30NzGCgAe6M0u0/edit?usp=sharing

The problem is that I receive candles for only 5 days no matter what dates or parameters I use. I can get more candles if i add the limit parameter to the string. But still, I can not get more than 1100-1000 candles. I even get the 500 error from the server:

Server error: GET https://api.bitfinex.com/v2/candles/trade:1h:tBTCUSD/hist?limit=1100&start=1512086400000&end=1516233600000&sort=1 resulted in a 500 Internal Server Error response:\n ["error",10020,"limit: invalid"]. What should be the valid limit? There is no such information in the docs.

The author of this topic has the same question but no solutions are given. The last answer does not make big changes: Bitfinex data api

How can I get the desired amount of data for the two years period of time? I do not want to break my query down into smaller pieces and go step by step. It will look ugly.

krizajb · Answer 1 · 2018-09-19T12:54:45.400

From the looks of it the limit is set to 1000. If you need more then 1000 historical entries you could parse the last timestamp of the response and create another request till you reach the desired end time.

Keep in mind that you can only do 10-90 requests peer minute. So it's smart to make some kind of sleeping mechanism on every request for 6 seconds or something like that.

import json
import time

import requests

start = 1512086400000
end = 1516233600000
timestamp = start
last_timestamp = None

url = 'https://api.bitfinex.com/v2/trades/tBTCUSD/hist/'
historical_data = []

while timestamp <= end and timestamp != last_timestamp:
    print("Requesting "+str(timestamp))
    params = {'start': timestamp, 'limit': 1000, 'sort': 1}
    response = requests.get(url, params=params)
    trades = json.loads(response.content)
    historical_data.extend(trades)

    last_timestamp = timestamp
    id, timestamp, amount, price = trades[-1]

How to get the historical data from bitfinex.com with out a limit?

1 Answers1