I'm trying to modify a device in NetBox using subprocess, curl and the NetBox REST API through python.
The issue seems to be with special characters and JSON.
Note: I'm not using requests because we do everything in curl. We are also bypassing security by using the "--insecure" option for curl because this is just a proof-of-concept right now.
The script we are using has two functions:
get: - retrieves all the info for all devices in NetBox and create a file for each device containing it's current 'comments' data. This works just fine.
patch: - grabs the contents of the appropriate device and uploads to NetBox, which should be visible in it's web GUI. This is the function with the issue.
For example, if the file contained
abc
I should be able to change the contents of the file to
now it's more than abc
and then upload the change to NetBox. Looking at the web GUI, I should see the change.
The actual file I am trying to upload to the device comments contains this gibberish, which I typed in manually:
anotyher device needing spellchecking
!@#$%^&\*()\_+-=\[\]\\ \[\]\\ /.,,./
did yuou see that?
My upload_comments function:
Note: the {deviceid} in the url comes from a different section of code and is not shown here and is working fine.
def upload_comments(filename):
with open(filename, 'r') as f:
new_comment = f.read().strip()
print('new_comment: ', new_comment) # verify we have content
patch_command_line = f'''curl
--insecure
--verbose
--request PATCH
--header "Content-Type: application/json"
--header "Accept: application/json"
--header "Authorization: Token xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
--user "xxxx:xxxx"
--url "https://xxx.xxx.xxx.xxx/api/dcim/devices/{deviceid}/"
--data-binary '{{ "comments": "{new_comment}" }}' '''
print(patch_command_line) # verify curl command line
try:
r = subprocess.run(patch_command_line, capture_output=True, shell=True, check=True, timeout=10)
# print("stdout: \n")
print(r.stdout.decode('utf8'))
print(r.stderr.decode('utf8'))
except Exception as e:
print(e)
^
This is the result of running that function:
new_comment: anotyher device needing spellchecking
!@#$%^&\*()\_+-=\[\]\\ \[\]\\ /.,,./
did yuou see that?
curl --insecure --verbose --request PATCH --header "Content-Type: application/json" --header "Accept: application/json" --header "Authorization: Token xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" --user "xxxx:xxxx!" --url "https://xxx.xxx.xxx.xxx/api/dcim/devices/2/" --data-binary '{ "comments": "anotyher device needing spellchecking
!@#$%^&\*()\_+-=\[\]\\ \[\]\\ /.,,./
did yuou see that?" }'
{"detail":"JSON parse error - Invalid control character at: line 1 column 53 (char 52)"}
* Trying xxx.xxx.xxx.xxx:443...
* TCP_NODELAY set
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\* Connected to xxx.xxx.xxx.xxx (xxx.xxx.xxx.xxx9) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
} \[5 bytes data\]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} \[512 bytes data\]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ \[122 bytes data\]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ \[21 bytes data\]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ \[892 bytes data\]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ \[264 bytes data\]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ \[52 bytes data\]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} \[1 bytes data\]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} \[52 bytes data\]
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: C=AU; ST=Some-State; O=Internet Widgits Pty Ltd
* start date: Nov 11 22:51:31 2022 GMT
* expire date: Nov 11 22:51:31 2023 GMT
* issuer: C=AU; ST=Some-State; O=Internet Widgits Pty Ltd
* SSL certificate verify result: self signed certificate (18), continuing anyway.
} \[5 bytes data\]
> PATCH /api/dcim/devices/2/ HTTP/1.1
> Host: xxx.xxx.xxx.xxx
> User-Agent: curl/7.68.0
> Content-Type: application/json
> Accept: application/json
> Authorization: Token xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> Content-Length: 108
} \[108 bytes data\]
* upload completely sent off: 108 out of 108 bytes
{ \[5 bytes data\]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ \[265 bytes data\]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ \[265 bytes data\]
* old SSL session ID is stale, removing
{ \[5 bytes data\]
* Mark bundle as not supporting multiuse
\< HTTP/1.1 400 Bad Request
\< Date: Tue, 22 Nov 2022 17:03:08 GMT
\< Server: gunicorn
\< Content-Type: application/json
\< Vary: Accept,Cookie,Origin
\< Allow: GET, PUT, PATCH, DELETE, HEAD, OPTIONS
\< API-Version: 3.3
\< X-Content-Type-Options: nosniff
\< Referrer-Policy: same-origin
\< Cross-Origin-Opener-Policy: same-origin
\< X-Frame-Options: SAMEORIGIN
\< Content-Length: 88
\< Connection: close
\<
{ \[5 bytes data\]
100 196 100 88 100 108 245 300 --:--:-- --:--:-- --:--:-- 545
* Closing connection 0
} \[5 bytes data\]
* TLSv1.3 (OUT), TLS alert, close notify (256):
} \[2 bytes data\]
Running curl from the command line gives me the correct result as long as there are no special characters in the value section of comments, otherwise I get errors like this:
{"detail":"JSON parse error - Unterminated string starting at: line 1 column 14 (char 13)"}\* Could not resolve host: november
* Closing connection 1
curl: (6) Could not resolve host: november
curl: (3) unmatched close brace/bracket in URL position 5:
11th}'
^
