API Blueprint and Dredd - Required field missing from response, but tests still pass

Question

I am using a combination of API Blueprint and Dredd to test an API my application is dependent on. I am using attributes in API blueprint to define the structure of the response's body.

Apparently I'm missing something though because the tests always pass even though I've purposefully defined a fake "required" parameter that I know is missing from the API's response. It seems that Dredd is only testing whether the type of the response body (array) rather than the type and the parameters within it.

My API Blueprint file:

FORMAT: 1A
HOST: http://somehost.net

# API Title

## Endpoints [GET /endpoint/{date}]

+ Parameters

  + date: `2016-09-01` (string, required) - Date

+ Response 200 (application/json; charset=utf-8)

    + Attributes (array[Data])


## Data Structures

### Data

- realParameter: 2432432 (number)
- realParameter2: `some string` (string, required)
- realParameter3: `Something else` (string, required)
- realParameter4: 1 (number, required)
- fakeParam: 1 (number, required)

The response body:

[
  {
    "realParameter": 31,
    "realParameter2": "some value",
    "realParameter3": "another value",
    "realParameter4": 8908
  },
  {
    "realParameter": 54,
    "realParameter2": "something here",
    "realParameter3": "and here too",
    "realParameter4": 6589
  }
]

And my Dredd config file:

reporter: apiary
custom:
  apiaryApiKey: somekey
  apiaryApiName: somename
dry-run: null
hookfiles: null
language: nodejs
sandbox: false
server: null
server-wait: 3
init: false
names: false
only: []
output: []
header: []
sorted: false
user: null
inline-errors: false
details: false
method: []
color: true
level: info
timestamp: false
silent: false
path: []
blueprint: myApiBlueprintFile.apib
endpoint: 'http://ahost.com'

Does anyone have any idea why Dredd ignores the fact that "fakeParameter" doesn't actually show up in the response body and still allows the test to pass?

score 1 · Accepted Answer · edited Jun 20 '20 at 09:12

1

You've run into a limitation of MSON, the language API Blueprint uses for describing attributes. In many cases, MSON describes what MAY be present in the data structure rather than what MUST exactly be present.

The most prominent case are arrays, where basically any content of the array is optional and thus the underlying generated JSON Schema doesn't put any constraints on array contents. Dredd just respects that, so indirectly it becomes a Dredd issue too, however there's not much Dredd can do about it.

There's an issue for the problem: apiaryio/mson#66 You can follow and comment under the issue to get updated about this. Dredd is usually very prompt in getting the latest API Blueprint parser, so once it's implemented in the language itself, it won't take long to appear in Dredd.

Obvious (but tedious) workaround is to specify your own JSON Schema with stricter rules using the + Schema section alongside the + Attributes section.

edited Jun 20 '20 at 09:12

Community

1
1

answered Sep 06 '16 at 15:11

Honza Javorek

8,566
8
47
66

Thanks for the answer! I will try it out right away. – Alex Seifert Sep 06 '16 at 15:18
On the same note as this answer, you could try formatting your .apib file to use JSON format. It's a step between what you have and @Honza's answer. – Brydenr Sep 06 '16 at 17:01
Yes, but unfortunately, gavel.js (the validator Dredd uses) has also some old problems with arrays - https://github.com/apiaryio/gavel.js/issues/43 :-( – Honza Javorek Sep 06 '16 at 17:34
Unfortunately, I wasn't able to get it to running before having to leave work. I will try again tomorrow though and hope that I can get it going. – Alex Seifert Sep 06 '16 at 19:14
Great! I'm glad I could help :-) – Honza Javorek Sep 07 '16 at 12:14

API Blueprint and Dredd - Required field missing from response, but tests still pass

1 Answers1