Answering my own question here in the hope it helps somebody.
In my case I wanted to verify the signature of a JWT token obtained via the AWS Cognito Developer Authenticated identity route. No AWS API Gateway involvement.
Like many posters on various sites I had trouble piecing together exactly the bits I needs to verify the signature of an AWS JWT token externally i.e., server side or via script
I think I figured out out and put a gist to verify an AWS JWT token signature. It'll verify an AWS JWT/JWS token with either pyjwt or PKCS1_v1_5c from Crypto.Signature in PyCrypto
So, yes this was python in my case but it's also doable easily in node (npm install jsonwebtoken jwk-to-pem request).
I attempted to highlight some gotchas in the comments because when I was trying to figure this out I was mostly doing the right thing but there were some nuances like python dict ordering, or lack there of, and json representation.
I've also noticed some questions around about doing the validation using the signers Certificate. I'll amend my gist to show this also.
Hopefully it may help somebody somewhere.