You are definitely missing something. Try this in a fresh scenario and see it work. We are hard-coding response
below, which is exactly equivalent to what happens at run-time, and by the way this is a great way for you to test assertions against different types of JSON (without making any HTTP calls):
* def response = {}
Then match $.orgId == '#present'
Then match $.orgId == '#notnull'
And this gives a failure as you expect:
assertion failed: path: $.orgId, actual: null, expected: '#present', reason: actual json-path does not exist
So if you still are stuck, follow this process please: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue
EDIT: if your response is an empty string but you were expecting JSON, just do this and it will fail the test, refer type-conversion: https://github.com/intuit/karate#type-conversion
* json response = response
But as mentioned in the docs, you should always try to match "full JSON" so this should work:
* def response = ''
Then match $ contains { orgId: '#notnull' }
EDIT: this will be fixed in 0.9.4 https://github.com/intuit/karate/issues/814