Why am I getting an empty struct from JSON Decoder?

Question

I'm trying to make an API call to AlphaVantage's Income Statement endpoint and then unmarshal the response into my IncomeStmt struct. It's not throwing any kind of error but when I try to print the result, I'm getting &{ [] []} as the value of incomeStmt.

This is the endpoint: https://www.alphavantage.co/query?function=INCOME_STATEMENT&symbol=IBM&apikey=demo

Here is the API response structure: https://www.alphavantage.co/query?function=INCOME_STATEMENT&symbol=IBM&apikey=demo

So far, I've combed through my IncomeStmt, QuarterlyReports and AnnualReports structs and all the fields match the name and types (all strings according to API structure above) from the API. I checked that I'm exporting all the fields correctly and I believe that my getJson function is working properly as I've been able to unmarshal a simpler API response using that function. So, I believe the issue is something with how I've created the structs to unmarshal into.

My expected result is to see the API response printed out in fmt.Printf("Unmarshaled IncomeStmt: %v \n", incomeStmt) from GetIncomeStmt.

Code Below:

type AnnualReports struct {
   FiscalDateEnding                  string `json:"fiscalDateEnding"`
   ReportedCurrency                  string `json:"reportedCurrency"`
   GrossProfit                       string `json:"grossProfit"`
   TotalRevenue                      string `json:"totalRevenue"`
   CostOfRevenue                     string `json:"costOfRevenue"`
   CostOfGoodsAndServicesSold        string `json:"costOfGoodsAndServicesSold"`
   OperatingIncome                   string `json:"operatingIncome"`
   SellingGeneralAndAdministrative   string `json:"sellingGeneralAndAdministrative"`
   ResearchAndDevelopment            string `json:"researchAndDevelopment"`
   OperatingExpenses                 string `json:"operatingExpenses"`
   InvestmentIncomeNet               string `json:"investmentIncomeNet"`
   NetInterestIncome                 string `json:"netInterestIncome"`
   InterestIncome                    string `json:"interestIncome"`
   InterestExpense                   string `json:"interestExpense"`
   NonInterestIncome                 string `json:"nonInterestIncome"`
   OtherNonOperatingIncome           string `json:"otherNonOperatingIncome"`
   Depreciation                      string `json:"depreciation"`
   DepreciationAndAmortization       string `json:"depreciationAndAmortization"`
   IncomeBeforeTax                   string `json:"incomeBeforeTax"`
   IncomeTaxExpense                  string `json:"incomeTaxExpense"`
   InterestAndDebtExpense            string `json:"interestAndDebtExpense"`
   NetIncomeFromContinuingOperations string `json:"netIncomeFromContinuingOperations"`
   ComprehensiveIncomeNetOfTax       string `json:"comprehensiveIncomeNetOfTax"`
   Ebit                              string `json:"ebit"`
   Ebitda                            string `json:"ebitda"`
   NetIncome                         string `json:"netIncome"`
}

type QuarterlyReports struct {
   FiscalDateEnding                  string `json:"fiscalDateEnding"`
   ReportedCurrency                  string `json:"reportedCurrency"`
   GrossProfit                       string `json:"grossProfit"`
   TotalRevenue                      string `json:"totalRevenue"`
   CostOfRevenue                     string `json:"costOfRevenue"`
   CostOfGoodsAndServicesSold        string `json:"costOfGoodsAndServicesSold"`
   OperatingIncome                   string `json:"operatingIncome"`
   SellingGeneralAndAdministrative   string `json:"sellingGeneralAndAdministrative"`
   ResearchAndDevelopment            string `json:"researchAndDevelopment"`
   OperatingExpenses                 string `json:"operatingExpenses"`
   InvestmentIncomeNet               string `json:"investmentIncomeNet"`
   NetInterestIncome                 string `json:"netInterestIncome"`
   InterestIncome                    string `json:"interestIncome"`
   InterestExpense                   string `json:"interestExpense"`
   NonInterestIncome                 string `json:"nonInterestIncome"`
   OtherNonOperatingIncome           string `json:"otherNonOperatingIncome"`
   Depreciation                      string `json:"depreciation"`
   DepreciationAndAmortization       string `json:"depreciationAndAmortization"`
   IncomeBeforeTax                   string `json:"incomeBeforeTax"`
   IncomeTaxExpense                  string `json:"incomeTaxExpense"`
   InterestAndDebtExpense            string `json:"interestAndDebtExpense"`
   NetIncomeFromContinuingOperations string `json:"netIncomeFromContinuingOperations"`
   ComprehensiveIncomeNetOfTax       string `json:"comprehensiveIncomeNetOfTax"`
   Ebit                              string `json:"ebit"`
   Ebitda                            string `json:"ebitda"`
   NetIncome                         string `json:"netIncome"`
}

type IncomeStmt struct {
   Symbol           string             `json:"symbol"`
   AnnualReports    []AnnualReports    `json:"annualReports"`
   QuarterlyReports []QuarterlyReports `json:"quarterlyReports"`
}

type Client interface {
   GetEarnings(symbol string) (*Earnings, error)
   GetIncomeStmt(symbol string) (*IncomeStmt, error)
}

type ClientOpts struct {
   ApiKey     string
   BaseUrl    string
   HttpClient *http.Client
}

type client struct {
   opts       ClientOpts
   httpClient *http.Client
}

func NewClient(opts ClientOpts) Client {
   if opts.ApiKey == "" {
       opts.ApiKey = secretProvider.GetEnv("ALPHA_VANTAGE_API_KEY")
   }
   if opts.BaseUrl == "" {
       opts.BaseUrl = secretProvider.GetEnv("ALPHA_VANTAGE_API_BASE_URL")
   }

   httpClient := opts.HttpClient

   if httpClient == nil {
       httpClient = &http.Client{Timeout: 10 * time.Second}
   }

   return &client{
       opts:       opts,
       httpClient: httpClient,
   }
}

func (c *client) get(u *url.URL) (*http.Response, error) {
   req, err := http.NewRequest(http.MethodGet, u.String(), nil)
   if err != nil {
       return nil, err
   }

   resp, err := c.httpClient.Do(req)

   if err != nil {
       return nil, err
   }

   return resp, nil

}

// getJson will take the http response and unmarshal it, returning an error if unsuccessful
func getJson(resp *http.Response, data interface{}) error {
   defer resp.Body.Close()
   return json.NewDecoder(resp.Body).Decode(&data)
}

func (c *client) GetIncomeStmt(symbol string) (*IncomeStmt, error) {
   u, err := url.Parse(fmt.Sprintf("%s?function=INCOME_STATEMENT&symbol=%s&apiKey=%s", c.opts.BaseUrl, symbol, c.opts.ApiKey))
   if err != nil {
       return nil, err
   }
   resp, err := c.get(u)

   if err != nil {
       return nil, err
   }

   incomeStmt := &IncomeStmt{}

   if err = getJson(resp, incomeStmt); err != nil {
       fmt.Print(err)
       log.Fatal("Error decoding data in getJson fn.")
       return nil, err
   }

   fmt.Printf("Unmarshaled IncomeStmt: %v \n", incomeStmt)

   return incomeStmt, nil
}

Guess: The server returned an unexpected JSON value because the request is bad. Check resp.StatusCode and handle 4xx values. — Charlie Tumahai, Dec 28 '22 at 18:29
@mkopriva Can you explain why I need `.Decode(data)` instead of `.Decode(&data)`? Also, regarding the abstractions, what would be your acceptable alternative? — Zac, Dec 28 '22 at 20:12
The application calls Decode with a pointer to an interface value containing a pointer to struct value. Calling Decode(data) eliminates the unnecessary level of indirection. — Charlie Tumahai, Dec 29 '22 at 00:05
@Zac please ignore the first comment, while unnecessary it's not "bad" as I claimed since it works, so you do not need it. I mixed it up with a similar but different issue commonly popping up here on SO. Apologies. — mkopriva, Dec 29 '22 at 04:23

score 1 · Answer 1 · answered Dec 28 '22 at 20:23

The issue here was that I incorrectly camelCased apiKey in the endpoint URL. Doing so caused my API key not to be passed in the GET request to AlphaVantage. I found this thanks to Cerise's comment above. While the API was returning a 200 status (even with my malformed request!), it prompted me to try to read the response body in a different way with ioutil.ReadAll(). It's important to note that the ReadAll() method is not performant for actual use according to this post here. I used the following only for debugging purposes:

    body, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        log.Fatal("Error using readall")
    }
    fmt.Println(string(body))

which printed the error message that was being hidden all along:

{
    "Error Message": "the parameter apikey is invalid or missing. Please claim your free API key on (https://www.alphavantage.co/support/#api-key). It should take less than 20 seconds."
}

Lowercasing the query parameter apiKey to apikey caused me to get the expected response. Moral of the story is I need to add additional error handling in case the API is not returning the expected response body so that the program doesn't fail silently.

Why am I getting an empty struct from JSON Decoder?

1 Answers1