Fun with Outbound REST Events

Posted on | by snhackery

“A good programmer is someone who always looks both ways before crossing a one-way street.”
— Doug Linder

A while back I mentioned that ServiceNow Event Management can be used within ServiceNow itself. I explained how all of that could work, but I never really came up with a real-world Use Case that would demonstrate the value of starting to wander down that road. I have code to generate Events in a lot of places that never get executed, but it is still there, just in case. One place where unwanted things do tend to happen, though, is when interacting with outside resources such as JDBC imports or external REST calls. Here, you are dependent on some outside database or system being up and available, and that’s not always the case, so you need to build in processes that can gracefully handle that. This is an excellent place for Event Management to step in and capture the details of the issue and log it for some kind of investigation or resolution.

So, I thought I should come up with something that anyone can also play around with on their own, that isn’t tied to some proprietary database or internal web service. I started searching for some kind of public REST API, and I stumbled across the Public APIs web site. There is actually a lot of cool stuff here, but I was looking for something relatively simple, and also something that would seem to have some relation to things that go on inside of ServiceNow. After browsing around a bit, I found the US Street Address API, which looked like something that I could use to validate street addresses in the User Profile. That seemed simple enough and applicable enough to serve my purpose, so that settled that.

There are quite a few parts and pieces to do everything that I want to do, so we will just take them on one at a time. Here is the initial list of the things that I think that I will need to accomplish all that I would like to do:

  • Create an Outbound REST Message using the US Street Address API as the end point,
  • Create a Script Include that will encapsulate all of the functions necessary to make the REST call, evaluate the response, log an Event (if needed), and return the results,
  • Create a Client Script on the sys_user table to call the Script Include if any component of the User’s address changes and display an error message if the address is not valid,
  • Create an Alert Management Rule to produce an Incident whenever the new Event spawns an Alert,
  • Test everything to make sure that it all works under normal circumstances, and then
  • Intentionally mangle the REST end point to produce a failure, thereby testing the creation of the Event, Alert, and Incident.

The first thing to do, then, will be to create the Outbound REST Message, but before we do that, let’s explore the web service just a little bit to understand what we are working with. To do that, there is a handy little API tester here. This will allow us to try a few things out and see what happens. First, let’s just run one of their provided test cases:

https://us-street.api.smartystreets.com/street-address?auth-id=21102174564513388&candidates=10&match=invalid&street=3901%20SW%20154th%20Ave&street2=&city=Davie&state=FL&zipcode=33331

The API is just a simple HTTP GET, and the response is a JSON object:

[
  {
    "input_index": 0,
    "candidate_index": 0,
    "delivery_line_1": "3901 SW 154th Ave",
    "last_line": "Davie FL 33331-2613",
    "delivery_point_barcode": "333312613014",
    "components": {
      "primary_number": "3901",
      "street_predirection": "SW",
      "street_name": "154th",
      "street_suffix": "Ave",
      "city_name": "Davie",
      "default_city_name": "Fort Lauderdale",
      "state_abbreviation": "FL",
      "zipcode": "33331",
      "plus4_code": "2613",
      "delivery_point": "01",
      "delivery_point_check_digit": "4"
    },
    "metadata": {
      "record_type": "S",
      "zip_type": "Standard",
      "county_fips": "12011",
      "county_name": "Broward",
      "carrier_route": "C006",
      "congressional_district": "23",
      "rdi": "Commercial",
      "elot_sequence": "0003",
      "elot_sort": "A",
      "latitude": 26.07009,
      "longitude": -80.35535,
      "precision": "Zip9",
      "time_zone": "Eastern",
      "utc_offset": -5,
      "dst": true
    },
    "analysis": {
      "dpv_match_code": "Y",
      "dpv_footnotes": "AABB",
      "dpv_cmra": "N",
      "dpv_vacant": "N",
      "active": "Y"
    }
  }
]

It looks like the response comes in the form of a JSON Array of JSON Objects, and the JSON Objects contain a number of properties, some of which are JSON Objects themselves. This will be useful information when we attempt to parse out the response in our Script Include. Now we should see what happens if we send over an invalid address, but before we do that, we should take a quick peek at the documentation to better understand what may affect the response. One input parameter in particular, match, controls what happens when you send over a bad address. There are two options;

  • strict  The API will return detailed output only if a valid match is found. Otherwise the API response will be an empty array.
  • invalid  The API will return detailed output for both valid and invalid addresses. To find out if the address is valid, check the dpv_match_code. Values of Y, S, or D indicate a valid address.

The default value in the provided tester is invalid, and that seems to be the appropriate setting for our purposes. Assuming that we will always use that mode, we will need to look for one of the following values in the dpv_match_code property to determine if our address is valid:

Y — Confirmed; entire address is present in the USPS data. To be certain the address is actually deliverable, verify that the dpv_vacant field has a value of N. You may also want to verify that the active field has a value of Y. However, the USPS is often months behind in updating this data point, so use with caution. Some users may prefer not to base any decisions on the active status of an address.
S — Confirmed by ignoring secondary info; the main address is present in the USPS data, but the submitted secondary information (apartment, suite, etc.) was not recognized.
D — Confirmed but missing secondary info; the main address is present in the USPS data, but it is missing secondary information (apartment, suite, etc.).

So, let’s give that a shot and see what happens. Let’s drop the state and zipcode from our original query and give that tester another try.

https://us-street.api.smartystreets.com/street-address?auth-id=21102174564513388&candidates=10&match=invalid&street=3901%20SW%20154th%20Ave&street2=&city=Davie

… which give us this JSON Array in response:

[
  {
    "input_index": 0,
    "candidate_index": 0,
    "delivery_line_1": "3901 SW 154th Ave",
    "last_line": "Davie",
    "components": {
      "primary_number": "3901",
      "street_predirection": "SW",
      "street_name": "154",
      "street_suffix": "Ave",
      "city_name": "Davie"
    },
    "metadata": {
      "precision": "Unknown"
    },
    "analysis": {
      "dpv_footnotes": "A1",
      "active": "Y",
      "footnotes": "C#"
    }
  }
]

This result doesn’t even have a dpv_match_code property, which is actually kind of interesting, but that would still fail a positive test for the values Y, S, or D, so that would make it invalid, which is what we wanted to see.

OK, I think we know enough now about the way things work that we can start building out out list of components. This is probably a good place to wind things up for this episode, as we can start out next time with the construction process, beginning with of our first component, the Outbound REST Message.