Ansible Cant extract json from URL but Curl can. What am I missing?

Question

Im having some trouble with Ansible URI module when pulling json from a specific web site.

Ansible Code:

 - name: check for Tenable Agent updates.
   uri:
     url: https://www.tenable.com/downloads/api/v1/public/pages/nessus-agents
     follow_redirects: none
     validate_certs: false
     return_content: yes
     body_format: json
   register: tenable_result

Ansible output when run:

TASK [agent-check : check for Tenable Agent updates.] ***************************************************************************************************************************
fatal: [testhost.local]: FAILED! => {"cf_cache_status": "DYNAMIC", "cf_ray": "66e527b55f742542-SJC", "changed": false, "connection": "close", "content": "Bad Request", "content_length": "11", "content_type": "text/plain; charset=utf-8", "date": "Tue, 13 Jul 2021 20:10:30 GMT", "elapsed": 0, "expect_ct": "max-age=604800, report-uri=\"https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\"", "msg": "Status code was 400 and not [200]: HTTP Error 400: Bad Request", "redirected": false, "server": "cloudflare", "set_cookie": "AWSALB=mjbt0/pPG5uYbItH6JZZsMNU31tfIYcjN3EzGUGGwe0yh1IyClNI3QZwwaIsvvxTBsxAtONNkX5ikvZUQxB/3m5RsPKsZKCs9GC/shJpEAmAiRTvDFgzMdSEsBl2; Expires=Tue, 20 Jul 2021 20:10:30 GMT; Path=/, AWSALBCORS=mjbt0/pPG5uYbItH6JZZsMNU31tfIYcjN3EzGUGGwe0yh1IyClNI3QZwwaIsvvxTBsxAtONNkX5ikvZUQxB/3m5RsPKsZKCs9GC/shJpEAmAiRTvDFgzMdSEsBl2; Expires=Tue, 20 Jul 2021 20:10:30 GMT; Path=/; SameSite=None; Secure, __cf_bm=cf4aba5cd769c19f68e8a6b84448295bb7acca50-1626207030-1800-AcDXU5VzEKMNjpyeGluMOFzCeIfWcxvrpS8wIms3LQHysBisfPjp051BUkVGysLvq7pPwEfXNDrNwGV7VV2GPbI=; path=/; expires=Tue, 13-Jul-21 20:40:30 GMT; domain=.tenable.com; HttpOnly; Secure; SameSite=None", "status": 400, "strict_transport_security": "max-age=31536000", "url": "https://www.tenable.com/downloads/api/v1/public/pages/nessus-agents", "x_content_type_options": "nosniff"}

QUESTION: How do I tell Ansible to anticipate this redirect and satisfy whatever the cloudflare needs in order for me to move on to the file & ID data over on tenable.com?

I know the URL works & can output the needed json data because I can parse the Filename & download ID via Curl and jq like so:

$ curl -s https://www.tenable.com/downloads/api/v1/public/pages/nessus-agents| jq '.downloads[] | "\(.file) \(.id)"'
"Agent_plugins_expires_2021-07-15.tgz 13183"
"NessusAgent-8.3.0-x64.msi 13130"
"NessusAgent-8.3.0-Win32.msi 13131"
"NessusAgent-8.3.0.dmg 13132"
"NessusAgent-8.3.0-debian6_amd64.deb 13134"
"NessusAgent-8.3.0-debian6_i386.deb 13135"
"NessusAgent-8.3.0-amzn.x86_64.rpm 13133"
"NessusAgent-8.3.0-es5.x86_64.rpm 13136"
"NessusAgent-8.3.0-es5.i386.rpm 13137"
"NessusAgent-8.3.0-es6.x86_64.rpm 13138"
"NessusAgent-8.3.0-es6.i386.rpm 13139"
"NessusAgent-8.3.0-es7.x86_64.rpm 13140"
"NessusAgent-8.3.0-amzn2.aarch64.rpm 13149"
"nessus-agent-updates-8.3.0.tar.gz 13150"
"NessusAgent-8.3.0-es8.x86_64.rpm 13141"
"NessusAgent-8.3.0-fc20.x86_64.rpm 13142"
"NessusAgent-8.3.0-suse11.x86_64.rpm 13143"
"NessusAgent-8.3.0-suse11.i586.rpm 13144"
"NessusAgent-8.3.0-suse12.x86_64.rpm 13145"
"NessusAgent-8.3.0-suse15.x86_64.rpm 13146"
"NessusAgent-8.3.0-ubuntu1110_amd64.deb 13147"
"NessusAgent-8.3.0-ubuntu1110_i386.deb 13148"
"tenable-2048.gpg 7000"
"tenable-1024.gpg 6998"

Any idea what I'm missing in my Ansible code?

Try [get_url](https://docs.ansible.com/ansible/latest/collections/ansible/builtin/get_url_module.html) instead. — Vladimir Botka, Jul 14 '21 at 19:45
get_url will allow me to save the page to a file on the host. I guess I can then convert the file to json, register, and finaly parse the filename and id's that way. — Michael Zietlow, Jul 14 '21 at 21:12

Ansible Cant extract json from URL but Curl can. What am I missing?

0 Answers0