We are using WSO2 APIM as an API manager. We are upgrading from 2.6.0 to 4.1.0 We have run in to an issue with the refresh tokens.
In the old version, once a token had been generated then all calls for the same scope got that token, as long as it was valid. In the new version each call gets a new token[1]. This is not a problem (the old version was somewhat problematic, in that it could lead to very short-lived tokens being returned).
However, if we use the password flow then the response includes both an access token (to access the APIs) and a refresh token (to get a new access token when the current one expires, without the application needing to save the password). If we make multiple calls to the /token endpoint, using the password flow (for the same user, application and scope) then we get a different access token each time but the same refresh token.
Refreshing the token (calling the /token endpoint with the refresh_token flow) no longer invalidates the old access token (which continues to work for the remainder of it's validity period) but does invalidate the refresh token. Thus the need to use device scopes, if the same user logs in on multiple devices, remains.
However, the refresh token also has a validity. Default validity is 1 hour for access tokens and 23.5 hours for refresh tokens. The problem is that the system does not appear to take the validity of the refresh token into account! The response I get, as a client calling the token endpoint, includes
- access_token - the access token
- refresh_token - the refresh token
- expires_in - number of seconds until the access token expires
It does not, however, include any information about the expiration of the refresh token. As a client system I have to assume the refresh token is valid at least as long as the access token and use it to get a new access token either when mine expires or shortly before that. But it seems the access token is now always valid for 1 hour while the refresh token may expire at any point!
To test this I wrote a small bash script which calls the token endpoint. It then saves the refresh token it gets in a variable and then calls the token endpoint again every 10 seconds until it gets a new refresh token. At this point, it tries to use the old refresh token to call the token endpoint using the refresh_token flow. I know this is the old token, but I was given it 10 seconds earlier with an access token that is nowhere near expired - in reality the calls could come from different systems, so the system that made the previous call would have no way of knowing the refresh token was old. Thus I expect it to work. It does not.
We set the timeouts down for our testing, to 60 seconds for the access token and 90 seconds for the refresh token. After 40 seconds, the refresh token changed and the old one was no longer valid[2]
[root@edge-temy-carc ~]# ./TestRefreshToken.sh
2023-02-27T16:48:44+0200 1e901408-94ab-3b2c-82ce-e290ff676ee2 First call 60
2023-02-27T16:48:55+0200 1e901408-94ab-3b2c-82ce-e290ff676ee2 Same refresh 60
2023-02-27T16:49:05+0200 1e901408-94ab-3b2c-82ce-e290ff676ee2 Same refresh 60
2023-02-27T16:49:16+0200 198bfab8-3fa7-396c-97ba-e4f07f4fc1e4 New refresh 60
Trying original refresh token
{
"error_description": "Persisted access token data not found",
"error": "invalid_grant"
}
My question: How can I, as a client calling the APIs via the API manager, know when the refresh token will expire so I can refresh my access token in time? Or, preferable, how can i configure the APIM so that the refresh tokens are always valid at least as long as any access token they are returned with?
Frankly, this feels like a bug - as a client I have to assume the refresh token is valid at least as long as the access token.
Additional Info
Script used for testing
#!/bin/bash
ACCESS_TOKEN=
REFRESH_TOKEN=
ORIGINAL_REFRESH_TOKEN=
EXPIRES_IN=
RESPONSE=
while true; do
CALL_DATE=$(date --iso-8601=s)
RESPONSE=$(curl -X POST https://api-temy-carc.internal.carus.com/token -d "grant_type=password&username=apiuser&password=1234&scope=device_baz" -H "Authorization: Basic amZXNGkyRkdMY1JTYjk5MWFXTkhnVEcyRVJNYTpnclQ5bFY2RldKY2x5VGh1WDBXSng3ZlNyRXdh")
REFRESH_TOKEN=$(echo "$RESPONSE" | jq -r .'refresh_token')
EXPIRES_IN=$(echo "$RESPONSE" | jq -r '.expires_in')
if [[ -z "$ORIGINAL_REFRESH_TOKEN" ]]; then
ORIGINAL_REFRESH_TOKEN=$REFRESH_TOKEN
echo -e "$CALL_DATE\t$REFRESH_TOKEN\tFirst call\t$EXPIRES_IN"
elif [[ "$ORIGINAL_REFRESH_TOKEN" == "$REFRESH_TOKEN" ]]; then
echo -e "$CALL_DATE\t$REFRESH_TOKEN\tSame refresh\t$EXPIRES_IN"
else
echo -e "$CALL_DATE\t$REFRESH_TOKEN\tNew refresh\t$EXPIRES_IN"
echo "Trying original refresh token"
sleep 1
curl -k -d "grant_type=refresh_token&refresh_token=$ORIGINAL_REFRESH_TOKEN" -H "Authorization: Basic amZXNGkyRkdMY1JTYjk5MWFXTkhnVEcyRVJNYTpnclQ5bFY2RldKY2x5VGh1WDBXSng3ZlNyRXdh" https://api-temy-carc.internal.carus.com/token | jq .
exit
fi
sleep 10
done
I did find this question, the answer to which says
In both password and refresh token grant types, you are renewing the previous access token. With the password grant type, you use the username and password combination while with the refresh grant type, you use the refresh token from a previous token call. Both follow the same approach where a new token will be issued with each token call while revoking the previous one
However, again this doesn't match our experience: If I get a new token using the password flow, the old token is not revoked but continues to work alongside the new one. Only the refresh_token flow seems to revoke the old refresh token.
Revoke doesn't work either. We found this question but
- We are only using APIM not IS and
- As stated there, those blocks are already present in the deployment.toml but don't seem to help
[1]: The WSO docs explain this as
The JWT token persistence behaviour is different to the opaque token persistence behaviour. The JWT token issuer always provides a new JWT token upon a token request and it does not persist a complete JWT access token in the database but only the JTI value of the JWT token. Therefore, there is no way to achieve the same behaviour for JWT tokens as opaque tokens other than customizing the token issuer.
However, if we put the generated JWT into jwt.io then it seems that each also gets a new JTI value also.
[2]: We had, of course, a couple of bugs in the first version of script so we ran it and then stopped it to fix them and ran again. I think therefore the 40 seconds is 90 seconds from an earlier attempt.
