0

I'm tring to retrieve a list of files from an API using axios, in Node.js. What I get is what I assume is an ArrayBuffer string. I'm using the following code:

axios.get('http://api.com/filesList').then(response => {
    console.log(response.data);
});

This is what I get (changed numbers and letters a bit to hide potential priv info):

$97cceb28↕$sad28sdj-askd-3294-3jh9→--Account_123456789 �����0*$97cceb282¶2022-11-30.csv"�☺
$97cceb28↕$sad28sdj-askd-3294-3jh9→--Account_123456789 ���0*$97cceb282¶2022-11-26.csv"�☺
$97cceb28↕$sad28sdj-askd-3294-3jh9→--Account_123456789 �໑�0*$97cceb282¶2022-11-19.csv"�☺

I know for a fact that I'm supposed to receive certain information for each file of this list, at least which account is associated to them and their name. I did a little digging and added responseType: 'arraybuffer' to the function and then it output a <Buffer 22 be 01...> kind of response. I then tried a method to convert it to a string but I end up getting the result I had without setting the response type.

So, what kind of string data is this and how do I convert it to an object/JSON? Is there a way to decode this structure?

Thank you all in advance.

coldpumpkin
  • 711
  • 4
  • 14
  • 30
  • What is the `content-type` and `content-encoding` of the response? You seem to be trying to reverse engineer things when the response should be telling you exactly what it is. Or, if you can show us the actual URL, we can then look at the response ourselves. – jfriend00 Dec 01 '22 at 03:13
  • @jfriend00 the `content-type` is `application/octet-stream` and the `content-encoding` is non-existant. The URL is https://api.budgetbakers.com/ribeez/import/v1/all, you need to create an account and upload a file first though. – coldpumpkin Dec 01 '22 at 03:26
  • Is there any doc for the API. I wonder if you have to pass an Accept header to tell it what format you want for the data. – jfriend00 Dec 01 '22 at 03:40
  • @jfriend00 there's no doc and I'm basically replicating what happens in the browser, in order to automate an upload. I could try different terms in the accept header but the fact is, this output is exactly the same when browsing the page and they somehow interpret it to list the files on their page. – coldpumpkin Dec 01 '22 at 03:53
  • In those case, I would suggest looking at EXACTLY what the request headers are when coming from the browser and send ALL of those headers exactly the same, including the user-agent and cookies. It may be deciding that your request is not coming from the browser and thus doing something different. – jfriend00 Dec 01 '22 at 03:58
  • @jfriend00 that is exactly what I did and I get the same result in Node.js as the browser does... I'm kinda clueless to what their script then does to interpret the structure since it's minified with ambiguous var names – coldpumpkin Dec 01 '22 at 16:46
  • 1
    Oh, I didn't know you were already getting the same response. So, the problem is one of reverse engineering how the browser code interprets the data it returns, I guess. It's unlikely anyone here is going to be able to help with that since we have none of the context or code that needs to be reverse engineered. A binary format could be anything. I don't think there's any advice other than step through their receiving code one line at a time and try to learn what it's doing. Since you can see some strings unchanged, it's probably just their own binary format. – jfriend00 Dec 01 '22 at 16:58
  • 1
    If you post the first 200 bytes of the binary buffer like it is here `` so I can see the actual binary values, I'll look and see if it's anything obvious. It would also help if I can see what the user interface displays from this (perhaps a screenshot) so I know what kind of data to be expecting to be in there. If one knows the output, it's easier to reverse engineer where it came from in the binary. – jfriend00 Dec 01 '22 at 18:13
  • @jfriend00 after some more digging and help from a friend, it looks like it's a protobuf buffer, managed to decode it here https://protobuf-decoder.netlify.app/ after converting the buffer to hex. Thank you for your commitment to trying to help me with this, that was really nice of you. DM me and I'll gladly buy you a coffee – coldpumpkin Dec 06 '22 at 11:34
  • @jfriend00 P.S.: Feel free to grab my solution and post it as an answer, I'll mark it – coldpumpkin Dec 06 '22 at 11:58
  • I don't really understand your solution so I'd suggest you post an answer yourself. I appreciate you offering to let me get some credit for helping, but in this case better for you to just post a good description of what solved the problem. – jfriend00 Dec 06 '22 at 23:32

0 Answers0