Error Received
I help a developer out maintaining a simple python script that scrapes images from unsee.cc. I have helped to correct URLs and remove minor problematic bits of code. Over the past two days I have received an error while running the script that I am completely baffled with and lack the necessary knowledge to resolve. The error that I am receiving is as follows:
Traceback (most recent call last):
File "/unsee-dl/main.py", line 104, in <module>
main()
File "/unsee-dl/main.py", line 14, in main
asyncio.get_event_loop().run_until_complete(run_downloader())
File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "/unsee-dl/main.py", line 98, in run_downloader
await download_beta(album_ids_beta_version, args.out_dir, args.group_album)
File "/unsee-dl/main.py", line 33, in download_beta
await client.anonymous_login()
File "/unsee-dl/unsee_dl/unsee_beta.py", line 62, in anonymous_login
tokens = content["data"]["getToken"]
KeyError: 'data
Offending Code
The offending block of code creating the error is concerned with logging into a session as an anonymous user and is as follows:
Defining "data"
The keyerror references the variable "data", which is the websocket settings pulled from the server. It is defined in a separate file that is imported and is as follows:
async def download_album(self, album_id):
unsee_name = names.get_random()
ws_params = "?album={}&name={}".format(album_id, unsee_name)
ssl_context = ssl.create_default_context()
# Settings WS
async with self.session.ws_connect(
_UNSEE_WEBSOCKET_URL.format("settings") + ws_params, ssl=ssl_context
) as ws_settings:
await ws_settings.send_str("{}")
data = await ws_settings.receive()
settings = json.loads(data.data, encoding="utf-8")
logging.info("[ws_settings] {}".format(settings))
images_info = []
The problematic block
The actual problematic block involves creating a new session as an anonymous user. The block of code generating the error is as follows:
async def anonymous_login(self):
"""
Login with an anonymous token
"""
gql_body = {
"operationName": "getToken",
"variables": {},
"query": """
query getToken($identity: ID, $code: ID, $refreshToken: ID, $name: String) {
getToken(identity: $identity, code: $code, refreshToken: $refreshToken, name: $name) {
...AuthPayloadFragment
__typename
}
}
fragment AuthPayloadFragment on AuthPayload {
token
refreshToken
__typename
}
""",
}
async with self.session.post(_GRAPHQL_URL, json=gql_body) as response:
content = await response.json()
#!tokens = content["data"]["getToken"]
tokens = content["data"]["getToken"]
self.token = tokens["token"]
Review
So there you have it. In review, it appears that the response from the server is different than what the script is expecting, thus leaving "data" undefined and preventing the script from properly executing. How am I able to discover the changes in order to appropriately query the web socket server and provide definition to data?
Here is an unsee session I created specifically for troubleshooting this, it will be open for the next 24 hours. Test session
The full and official repository containing the project I am working on. Unsee_dl.py