generating oauth_signature for ETrade API using Python

Question

The E*Trade API allows you to use RESTful to log on to the site and manipulate an account or retrieve quote information. Though I am having trouble generating an oauth_signature that matches their "practice problem" located toward the bottom of https://us.etrade.com/ctnt/dev-portal/getContent?contentId=306a9d46-58c2-4cac-85f6-7717aea056bd

The simple HMAC-SMA1 algorithm has been coded below and reproduces the oauth core 1.0a signature value from here https://oauth.net/core/1.0a/#sig_base_example. Though I cannot get E*Trade signature value to reproduce.

def generate_oauth_signature():
    from urllib.parse import quote_plus
    from hashlib import sha1
    import binascii
    import hmac

    key = quote_plus('7d30246211192cda43ede3abd9b393b9') + \
          '&' + \
          quote_plus('XCF9RzyQr4UEPloA+WlC06BnTfYC1P0Fwr3GUw/B0Es=')
    key = key.encode()
    raw = quote_plus('GET') + '&' + \
          quote_plus('https://etws.etrade.com/accounts/rest/accountlist') + '&' + \
          quote_plus('oauth_consumer_key=c5bb4dcb7bd6826c7c4340df3f791188&oauth_nonce=0bba225a40d1bbac2430aa0c6163ce44&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1344885636&oauth_token=VbiNYl63EejjlKdQM6FeENzcnrLACrZ2JYD6NQROfVI=')
    raw = raw.encode()
    hashed = hmac.new(key, raw, sha1)
    sig = hashed.digest()
    oauth_signature = quote_plus(binascii.b2a_base64(hashed.digest())[:-1])

The function is supposed to yield "%2FXiv96DzZabnUG2bzPZIH2RARHM%3D", but I'm not there yet. Has anyone worked out the hashing for the E*Trade API?

I am aware of the etradepy.py, which is a nice package, but is a little outdated and does not match the current E*Trade website.

I'm in .NET, so I can only help conceptually, not specfic code. — BWhite, Jul 19 '17 at 17:21
Look at the example get request on that page. The signature goes in the get request. You have it the other way around. — BWhite, Jul 19 '17 at 17:36
The signature method takes two parameters, text and a key. The key is "Consumer Secret and Token Secret, separated by an '&' ". So it looks to me like you have that part correct. I don't see any details on either page about what the text to be signed should be. Pretty sure it shouldn't include the URL. Play around with encoding different pieces. — BWhite, Jul 19 '17 at 17:46
My header looks roughly like Authorization: OAuth oauth_token="UedyRLaPYUX0shFt3jYLPUVDhT5MsfOWQc%2F9tX2Szwg%3D",oauth_nonce="7f4352f8d79094648bc5629c39981d31",oauth_consumer_key="fc30012ef9f37fd768e935e84f1e413e",oauth_signature_method="HMAC-SHA1",oauth_timestamp="1500449672",oauth_version="1.0",oauth_signature="cayQmNCuz5dZSxEuwxyuHnhUFgs%3D" — BWhite, Jul 19 '17 at 22:03
I sincerely appreciate you spending some time on my question. It's a funny thing, this oauth.... but this is from the OAuth website here (https://oauth.net/core/1.0a/#anchor13). — jsfa11, Jul 20 '17 at 18:38
The request parameters are collected, sorted and concatenated into a normalized string: Parameters in the OAuth HTTP Authorization header excluding the realm parameter. Parameters in the HTTP POST request body (with a content-type of application/x-www-form-urlencoded). HTTP GET parameters added to the URLs in the query part (as defined by [RFC3986] section 3). The oauth_signature parameter MUST be excluded. — jsfa11, Jul 20 '17 at 18:38
@BWhite Can you post it in a google drive link or something similar or do you want my email? I'm currently using a library too, DevDefined.OAuth Library. I get the access tokens but when I try to make a request for an endpoint I get a 401 Error. The 401 error is ambiguous, it does not tell me what the error is because on the ETrade API 401 Error can mean a lot of things. — NuWin, Mar 16 '19 at 17:44
First, I am using the same lib, so that won't help. Second, start a new question, so we can tackle this properly. Tag it with etrade-api so I see it. — BWhite, Mar 18 '19 at 16:47

score 1 · Accepted Answer · answered Jul 22 '17 at 20:06

1

One problem is that the oauth_token needs to be encoded in the parameter string (it will end up being double encoded). Mine is the following:

oauth_consumer_key=c5bb4dcb7bd6826c7c4340df3f791188&oauth_nonce=0bba225a40d1bbac2430aa0c6163ce44&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1344885636&oauth_token=VbiNYl63EejjlKdQM6FeENzcnrLACrZ2JYD6NQROfVI%3D

answered Jul 22 '17 at 20:06

kingdc

135
8

The strings are encoded by the quote_plus() function in Python3. Then, this is encoded to utf-8 by the .encode() that follows. Here is what I pass to the hashing function; key=7d30246211192cda43ede3abd9b393b9&XCF9RzyQr4UEPloA%2BWlC06BnTfYC1P0Fwr3GUw%2FB0Es%3D raw=GET&https%3A%2F%2Fetws.etrade.com%2Faccounts%2Frest%2Faccountlist&oauth_consumer_key%3Dc5bb4dcb7bd6826c7c4340df3f791188%26oauth_nonce%3D0bba225a40d1bbac2430aa0c6163ce44%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1344885636%26oauth_token%3DVbiNYl63EejjlKdQM6FeENzcnrLACrZ2JYD6NQROfVI%3D – jsfa11 Jul 23 '17 at 16:53
@jsfa11 That's what I meant by double-encoded. Your oauth_token should be singly encoded already, and the quote_plus in your code will encode it again. For example, '=' becomes %3D which then becomes %253D. Your key string is correct but the 'raw' should be: GET&https%3A%2F%2Fetws.etrade.com%2Faccounts%2Frest%2Faccountlist&oauth_consumer_key%3Dc5bb4dcb7bd6826c7c4340df3f791188%26oauth_nonce%3D0bba225a40d1bbac2430aa0c6163ce44%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1344885636%26oauth_token%3DVbiNYl63EejjlKdQM6FeENzcnrLACrZ2JYD6NQROfVI%253D – kingdc Jul 23 '17 at 17:17
What do ya know? Thanks kingdc! I was very slow on this one. Your answer is correct. – jsfa11 Jul 23 '17 at 17:21
Took me a long time to get their example to match back when I wrote mine as well (not in python), it's not a very intuitive process. – kingdc Jul 23 '17 at 17:40
Do you think that you could help again? I got my token and it's secret, then I thought accessing a few quotes is the next easy thing. I've tried a lot. New time stamp, new signature with new token and secret, with and without verification_code. Just isn't working for me. The docs are vague on what is included in the header and oauth for requests. – jsfa11 Jul 24 '17 at 22:41
@jsfa11 Not sure what you mean, what's it giving you? The URL for simple quote should look like: https://etws.etrade.com/market/rest/quote/NFLX,AAPL?oauth_consumer_key=&oauth_nonce=&oauth_signature=&oauth_signature_method=HMAC-SHA1&oauth_timestamp=&oauth_token=. – kingdc Jul 25 '17 at 00:34
I am using the headers to pass the information, as I did when requesting the access token. The timestamp is updated. A new signature is also generated to match the new token and token secret. I get oauth_problem=signature_invalid returned. – jsfa11 Jul 25 '17 at 17:39
started this as a new question – jsfa11 Jul 25 '17 at 18:57

generating oauth_signature for ETrade API using Python

1 Answers1