Automate Testing Your API with Postman, Newman and Jenkins (Part 1)

This will be a four part series on how to use Postman, Newman and Jenkins as part of your continuous integration. We will consume an API, create test scripts with Postman, and use Jenkins to run our integration tests when our build server deploys code. When a build fails on our integration tests, we will configure Jenkins to send out an email alert to our supporting team.

What is the goal?

Testing APIs is difficult. Instead of spot checking an endpoint's response, we can write integration tests with Postman to examine and validate the response. We also want to automate running these tests with our deployment process. One approach you can take is to have your integration tests run every time a developer checks in code to your repo. Instant feedback!

Why do we want to do this?

Automate your QA! While this is in the right direction, this isn't fool proof. This add another layer of quality control but we need to ensure our tests are thorough and up to date. Hopefully, this added layer will ensure existing functionality still works as expected when we add new features to our API. Having this process automated, will catch unanticipated issues sooner in the development life cycle and will help the developers validate their code is doing what they intend.

What tools will we need?

1. Postman - We will use Postman to make requests to our API and run integration tests.
2. Newman - Newman is the NPM command-line version of Postman's Collection Runner. More on this later.
3. Jenkins - This is our build server that will be used to deploy and run our integration tests.

Overview

1. We will create a few integration tests with Postman.
2. We will create a Powershell file that runs these integration tests via command line.
3. We will add a Post Build step in Jenkins to execute our Powershell script whenever a build is initiated.

Let's begin!

1. Select Your API

   Here, we are going to use a free public API called Open Weather Map.

   You can sign up for an API Key and start consuming their Weather API in a few minutes. https://openweathermap.org/citytes. Feel free to use any API you'd like.

2. Create Integration Tests

   1st Integration Test: Get Weather by ID

   I create a new GET request to Get Weather by ID. Don't forget to add in your API Key in the APPID variable. Otherwise, you won't get a response from OpenWeatherMap. Send your request and verify that you get a response from the API. I have highlighted a few important things in my screenshot below.

   1. Notice the URL is using Postman parameters. Parameters are indicated by surrounding your variables with brackets. ex.{{variableName}}.
   2. The Postman variables are inside my environment called "Demo-OpenWeatherMap." You can read more on using Environments with Postman here.
   3. I have added some raw JavaScript to the Pre-Req tab to setup our test.

    <code>
   var cityIds = [
    5206379, // Pittsburgh
    4440906, // Philadelphia
    4505716, // Baltimore
    4160812, // Key West
    3882428, // Lost Angeles
    5128638, // New York
    5809844, // Seattle
    1723862, // Boston
    4887398, // Chicago
    5506956, // Last Vegas
   ];
   // Get a random number between 0-9
   var randomCityId = Math.floor((Math.random() * 9) + 1);
   
   // Set an environment variable to the value of the random city id from array
   postman.setEnvironmentVariable("CITYID", cityIds[randomCityId])
    </code>
    

   I start by creating an array with 10 known city ids. You can get a list of all available city ids here. I then get a random value from 1-10 and assign it to a local variable. I then assigned this value to a Postman Environment variable called "CITYID." Behind the scenes, this key-value variable is written to a local json file that contains all the variables in our environment. In our case, this is the Demo-OpenWeatherMap environemnt. So every time this endpoint is executed, a (not so) random city is queried.

   Now turn your attention to the Tests tab where I have the following JavaScript.

    <code>
   var jsonData = JSON.parse(responseBody);
   
   //Saving the response "name" value into an environment variable
   postman.setEnvironmentVariable("CITYNAME", jsonData.name);
   
   pm.test("Status code is 200", function () {
    pm.response.to.have.status(200);
   });
    </code>
    

   What we are doing here is parsing the json response into a variable that we can work with. We then save the value of the "name" property from the json response into an Environment variable. The value here is "Philadelphia." Then we create our first Postman test called "Status code is 200," which will fail if the HTTP status code is anything other than a 200.

   2nd Integration Test: Gets Weather By City Name

   We create a 2nd endpoint that Gets Weather By City Name. We don't have any code in the Pre-req tab so move onto the Tests tab.

    <code>
   var jsonData = JSON.parse(responseBody);
   pm.test("City Name matches Request'", function () {
    var contains = false;
    var cityFromResponse = postman.getEnvironmentVariable("CITYNAME");
    //console.log(cityFromResponse);
   
    if(jsonData.name == cityFromResponse) {
    contains = true;
    }
    else
    {
    console.log("Unexpected city");
    }
   
    pm.expect(contains).to.be.true;
   });
   
   pm.test("Contains object: 'weather'", function () {
    var contains = false;
    for (var key in jsonData) {
    if (key == "weather") {
    contains = true;
    }
    }
    pm.expect(contains).to.be.true;
   });
   
   pm.test("Contains string: 'rain' ", function () {
    var contains = false;
    var expected = "rain";
    pm.expect(pm.response.text()).to.include(expected);
   });
   
   pm.test("DOES NOT Contain string: 'snow' ", function () {
    var contains = false;
    var expected = "snow";
    pm.expect(pm.response.text()).to.not.include(expected);
   });
    </code>
    

   Here we have 4 tests. Each of these tests inspects the response and tests different properties of the json response. The first test "City Name matches Request," tests that the city name in the response is the same name we pass to the GET request. If you recall, the Environment variable "CITYNAME" is captured in the "Get Weather By Random City ID" endpoint. What we are illustrating here is one endpoint saves a value and passes that same value to another endpoint. In a real-world app, you could request an auth token with one endpoint and use the token with another endpoint. The 2nd test is checking if an object exists in the json. We want to make sure that "weather" is an object in the response. The last two tests are string comparison tests. We want to see if "rain" is present and "snow" is not. Send your request and verify you get a response. If you don't, check to make sure your API Key was supplied in the request.

   When you can confirm a response is sent back, examine the Test Results tab.

   The first two tests should always pass. If they are not passing, check to make sure you have the Environment variable names spelled correctly and with the right casing. Your response should always have a Weather object. Now the next two tests really depend on the city that was picked and of course, the weather during the time and year you make the request. On this request, it is not raining in Philadelphia, which fails my test. Remember, one of the tests was to verify it was raining by expecting the word "rain" in the response.

Summary

Congratulations on completing Part 1!

We learned how to use Postman's Environment file to parameterize our API Key and how to save and retrieve Environment variables within the JavaScript code. We created two endpoints with Postman and added a few integration tests. We used the pre-req tab on our first request to capture some data for the second request. In our second endpoint, we added a test that checked if the response name matches the value that was sent to the request. We also added a test that verified a response object was available, and two string comparison tests. While theses tests aren't complex, they illustrate how you can use Postman to execute integration tests with whatever business logic you are trying to validate. The reason why we created two endpoints was to show how you can use one endpoint in conjunction with another. You can chain together any number of requests you'd like.

Next, we will look at how Postman can run all the endpoints in our Collection and run through each test within an endpoint using Collection Runner. We will also introduce Newman, which is the command-line version of Collection Runner.

You can read Part 2 of this series by clicking here.