0

I need to monitor xhr requests on a website (which I don't control) and get realtime data. For example: tickdata.

My plan: log in with selenium and python after build websocket server. finally, acsess that.

Problem: target xhr data is like crash file. So, after debugging some js files I found out xhr data is crashed for not getting. But, the data I want exists as an array variable in a 3rd js. adding not global object.

Is it possible to get the variable in a running js script?

just like this:

Capture AJAX response with selenium and python

YakovL
  • 7,557
  • 12
  • 62
  • 102
Taira Nakanishi
  • 1
  • Hi, your question is very unclear, due to both lack of details and poor grammar (I've fixed some for you). The website is not yours, is it? If so, what "websocket server" are you talking about? What do you mean by "monitoring xhr"? What "data is like crash file" means? Data can not "crash" (unless there's a problem with hardware), programs crash. By "3rd js" do you mean "3rd party js" or what? Please describe your setup in more detail, for ordinary js any variable can be inspected via dev tools. – YakovL Jan 23 '19 at 12:26
  • Hi,YakovL .Thank you for your reply and forgive my unclear question. Please let me explain further. WEB site is not mine. just one thing that I want. That is getting data. Usually scraping. but, what i want is complete data (not depending on intervals of programs). thus,I came up with sending xhr data(json , plain/text) on chrome dev tool to WebSoket server ,finally pick up on server from client .　 I do not know where driver.execute_script("HERE"). ordinally,xhr data is beautiful. – Taira Nakanishi Jan 23 '19 at 15:04
  • we have to do anything.but,on the website's has garbled on pain text. While,variable in javascript contains beautiful data what I want it!! js scripts.exist 5~8 . One of them, example 3rd js has only target variable. To put it simply, variable(not define global scope )on javascript send to WebSoket server. – Taira Nakanishi Jan 23 '19 at 15:04
  • Hi, frist, use @mentions if you'd like somebody to answer (not to miss your reply). Second, your question definitely lacks the code you tried (at least the most important bits) and it's hard to understand the problem because of that. Finally, I'll mention parts of the question which I still don't understad: *after build websocket server* = after connecting to the site via websocket? *finally, acsess that.* = ?? access what and how? *xhr data is like crash file* = ?? *for not getting* = ?? *in a 3rd js* = ?? (you haven't shown anything, 3d is counting among what?) *adding not global object* = ? – YakovL Jan 26 '19 at 23:55

0 Answers0