How to capture Json from web site using c#

Question

I have a paid subscription to Seeking Alpha and have a cookkie which enables me to get full data from https://seekingalpha.com/symbol/AAPL/financials-data?period_type=quarterly&statement_type=income-statement&order_type=latest_left&is_pro=True

I'd like to collect JSON response using C#

Below is my horrible code

        string cookie = "my super secret cookie string";


        var request = new RestRequest(Method.GET);
        request.AddHeader("content-type", "application/json");
        request.AddHeader("Accept", "*/*");
        request.AddHeader("User-Agent","Mozilla/5.0");
        request.AddHeader("X-Requested-With", "XMLHttpRequest");
        string url = "https://seekingalpha.com/symbol/AAPL/financials-data?period_type=quarterly&statement_type=income-statement&order_type=latest_left&is_pro=True";

                    request.AddParameter("cookie", cookie, ParameterType.Cookie);

        var client = new RestClient(url);

        var queryResult = client.Execute(request);

        Console.WriteLine(queryResult.Content);

How can I get it to return JSON to me? I am getting something but not the JSON I want

https://dotnetfiddle.net/jebgbn Here is a `fiddle` with exact same code as you post in `question`. You can check it returns `json`. Strangely if you try few time it will start returning `html` in some of the response. Then again its returning `json`. — Karan, May 28 '20 at 05:09
Yes. I think I need to authenticate somehow. I have a cookie string, just need to pass it somehow — James Raitsev, May 28 '20 at 05:32

Karan · Answer 1 · 2020-05-28T04:55:07.530

0

Try updating your Access header to application/json.

request.AddHeader("Accept", "application/json");

Accept indicates what kind of response from the server the client can accept.

You can get more info for Accept from Header parameters: “Accept” and “Content-type” in a REST context

edited May 28 '20 at 04:55

answered May 28 '20 at 04:50

Karan

12,059
3
24
40

Good idea, but this is not it. I think I am setting cookig incorrectly. Should it be `request.AddParameter("cookie", cooki1, ParameterType.Cookie);` or `request.AddCookie("machine_cookie", cooki1)`? – James Raitsev May 28 '20 at 04:54
Ok. `Accept = application/json` will required when `server` returning data as multiple formats and default format is not `json`. As per your comment it seems that you're not having issue with `Accept`. – Karan May 28 '20 at 04:57
I changed it to application/json. Response is empty – James Raitsev May 28 '20 at 05:00

score 0 · Accepted Answer · answered May 30 '20 at 15:41

After poking around for a bit I figured it out. For the benefit of all:

    private bool FinancialStatement(string symbol, string statement, string period)
    {
        var target = $"{BASE_URL}{symbol}/financials-data?period_type={period}&statement_type={statement}&order_type=latest_left&is_pro=True";

        var client = new RestClient(target);

        client.Timeout = -1;
        var request = new RestRequest(Method.GET);
        request.AddHeader("Cookie", MACHINE_COOKIE);
        IRestResponse response = client.Execute(request);
        dynamic responseObj;
        try
        {
            responseObj = JsonConvert.DeserializeObject(response.Content);
        }
        catch (Exception)
        {
            return false;
        }

        return response.IsSuccessful;
    }

How to capture Json from web site using c#

2 Answers2