How to export newman report without secrets?

Question

I have to generate a newman JSON report for my APIs test results and then upload it somewhere.

The problem with the default JSON reporter is that it renders all my environment variables even the ones I set as secret.

I created a small mock collection to show that:

Environment snippet:

{
    "id": "1bb352fe-fdcf-4a18-942b-a83270fa6fbd",
    "name": "Comments Fake API Env",
    "values": [
        {
            "key": "commentId",
            "value": "1",
            "type": "default",
            "enabled": true
        },
        {
            "key": "secretCommentId",
            "value": "2",
            "type": "secret",
            "enabled": true
        },
        {
            "key": "Some_disabled_var",
            "value": "",
            "type": "default",
            "enabled": false
        }
    ],
    "_postman_variable_scope": "environment",
    "_postman_exported_at": "2023-02-17T21:00:41.635Z",
    "_postman_exported_using": "Postman/10.9.4"
}

Collection snippet:

{
    "info": {
        "_postman_id": "1d347975-eedd-4f5c-a58d-36cdd7b77794",
        "name": "Dummy comments API",
        "schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json"
    },
    "item": [
        {
            "name": "Gat all",
            "request": {
                "method": "GET",
                "header": [],
                "url": {
                    "raw": "https://dummyjson.com/comments?page=30",
                    "protocol": "https",
                    "host": [
                        "dummyjson",
                        "com"
                    ],
                    "path": [
                        "comments"
                    ],
                    "query": [
                        {
                            "key": "page",
                            "value": "30"
                        }
                    ]
                }
            },
            "response": []
        },
        {
            "name": "Get by Id",
            "request": {
                "method": "GET",
                "header": [],
                "url": {
                    "raw": "https://dummyjson.com/comments/{{commentId}}",
                    "protocol": "https",
                    "host": [
                        "dummyjson",
                        "com"
                    ],
                    "path": [
                        "comments",
                        "{{commentId}}"
                    ]
                }
            },
            "response": []
        },
        {
            "name": "Get by secret ID",
            "request": {
                "method": "GET",
                "header": [],
                "url": {
                    "raw": "https://dummyjson.com/comments/{{secretCommentId}}",
                    "protocol": "https",
                    "host": [
                        "dummyjson",
                        "com"
                    ],
                    "path": [
                        "comments",
                        "{{secretCommentId}}"
                    ]
                }
            },
            "response": []
        }
    ]
}

And if I run newman with JSON reporter

newman run  -e ./environment.json ./collection.json 

I see that the values are rendered

cat newman/report.json | jq '.environment.values'

[
  {
    "type": "any",
    "value": "1",
    "key": "commentId"
  },
  {
    "type": "any",
    "value": "2",
    "key": "secretCommentId"
  },
  {
    "type": "any",
    "value": "",
    "key": "Some_disabled_var"
  }
]

I did some search online and I didn't find any workarounds or fixes for this. I am working on my custom reporter but it is not going well as all what I receive from newman's event emitter is an object with environments already rendered and all variable types are set to any so there's no way that I can filter them out without a hacky way of listing them from my environment.json file before I run newmann.

I believe that this might be a bug or at least a non-implemented feature so I filed an issue and I'm still waiting for an answer from the maintainers

So my question is: How to get a newman report without any secrets?

Answer 1

You have a workaround option tile Postman support your filed an issue

Filter only "enabled": true fields from environment.json by in-house node.js program.

Install dependency for node.js

npm install fs

Save as convert.js file name.

const fs = require('fs');

const environment = fs.readFileSync('environment.json');
const varaibles = JSON.parse(environment);
console.log(varaibles);

const filtered_collection = {
    "id": varaibles.id,
    "values": varaibles.values.filter((x) => { return x.enabled == true }),
    "name": varaibles.name,
    "_postman_variable_scope": varaibles._postman_variable_scope,
    "_postman_exported_at": varaibles._postman_exported_at,
    "_postman_exported_using": varaibles._postman_exported_using
}
const new_varables = JSON.stringify(filtered_collection, null, 4)
console.log(new_varables);

fs.writeFileSync('environment-no-credential.json', new_varables);

Running with converted environment-no-credential.json and missed credential variables.

newman run  -e ./environment-no-credential.json ./collection.json --env-var "varaible1=your_varaible1" --env-var "varaible2=your_varaible2"

Demo

I will shows getting Spotify access-token and display genre with this methods. There are two requests.

Confirmed running collection is passed.

Export after unchecked for credentials variables

*Note - unchecked client_id and client_secret

Those will export with values.enabled is false in environment.json

convert from environment.json to environment-no-credential.json

blue box is with all variables red box is without credential variables

Run with environment-no-credential.json and 'collection.json` and credential variables.

newman run  -e ./environment-no-credential.json ./collection.json --env-var "client_id=<your_client_id>" --env-var "client_secret=<your_client_secret>"

Result

Final conclusion

You can export your environment variables with un-checked enable,

convert from environment.json to environment-no-credential.json

then run newman with credential variables by --env-var option.