0

I would like to select requests in a HAR file that match certain properties (e.g. POST requests to URL's matching certain parameters) and then examine the request and response data for this.

How can I do this?

Research

  • I searched on github for code that deals with this problem
  • I searched on kaggle for notebooks that deal with HAR files - and didn't find any - since these notebooks might deal with display issues.

Possible approaches

  • Chrome's network tag can perform certain filters, such as filtering to certain mime types, external requests, and searching the URL. However, if cannot filter by POST.
  • jq is a specialised command line tool to filter json, that can be used for some ad hoc filtering.

The following jq query will find POST requests with url's contain sign, but it's rather difficult to look at the result

cat data.har | jq '.log.entries[] | select(.request.method == "POST") | select(.request.url| contains("sign")) | less'
Att Righ
  • 1,439
  • 1
  • 16
  • 29
  • A HAR file is in JSON format .. Soooo .. A scripting language like PHP / Python, NodeJS etc etc that has native JSON parsing capabilities would be a good start .. – Zak Sep 13 '22 at 16:09
  • A good start... but slightly slow one. – Att Righ Sep 13 '22 at 16:10
  • Would you like us to write it for you? – Zak Sep 13 '22 at 16:10
  • I mean... I'm writing it myself in query as we speak in `jq`. I would prefer you to say "ah yes, you should use the GUI tool", "actually firebug can load and filter har files", or "this command line is specialised in reading har files". My hope is that someone on the internnet has specialist knowledge on this task. – Att Righ Sep 13 '22 at 16:12
  • Then please reformat your question with what you have tried. Where you are going wrong. And what the issue is. Asking for opinions and "what is the best tool for this" is against SO guidelines and will get your question closed. – Zak Sep 13 '22 at 16:14
  • Every question is effectively "recommend a tool for this" the world doesn't exist to dance for you. I could concatenate two pdf files together by writing a pdf parser - suspect that this would be a bad approach. – Att Righ Sep 13 '22 at 16:16
  • Fine. Let's argue for do not closing. HAR is a specialist format, and collection of data using HAR is a common tasks. Specialsed tool for dealing with the data contained within HAR exist such as the chrome network panel and the firefox panel, and many users are familiar with interacting those tools. The existence of these tools suggests that interacting with the json dumps is unwieldy. – Att Righ Sep 13 '22 at 16:19
  • OK for the sake of argument. Chrome's Network panel is written in JavaScript and is just a JSON parsing mega-tool. So, parsing JSON can be unwieldly, but you're in such a niche project, you have to do some discovery and probably going to end up parsing it yourself. `jq` is an amazing tool, however I feel a combination of (PHP or Python or JavaScript) + (bash+jq) is going to be your solution. Since dealing with the JSON will be easiest in jq - But parsing the data in a meaningful way will be easiest in a scripting language. But still, this is an opinion, and does not improve SO community – Zak Sep 13 '22 at 16:31
  • Thanks. The thing that makes me unhappy about `jq` (or indeed scripting) is mostly the display of data as well as the fact that some of the fields in the har format are quite nested, so the queries can be unwieldy. If I end up going "scripty" I'll probably almost write a "har cli" tool, since I'm being quite exploratory. The other thing is that there is quite a lot of data associated with a request and response, so it could be nice to put this in a table. I guess I could use something jupyter and friends to do this "prettily" – Att Righ Sep 13 '22 at 16:37

0 Answers0