I tried restarting my puppet master but the error still persists.
Also see it from all the puppet agents that try to join.
puppet agent --server puppet.example.com --waitforcert 60 --test --verbose
Error: Could not request certificate: Error 400 on SERVER: header too long