Fun with Outbound REST Events, Part V

“The best preparation for good work tomorrow is to do good work today.”
Elbert Hubbard

At the end of the last installment in this series, I mentioned two possible options for the next item on which to focus our energies. At the time, I wasn’t really sure which direction would be the preferable choice, but now that it is time to fish or cut bait, a decision needs to be made. Given that the entire purpose of this exercise is to demonstrate the use of Event Management practices on internal ServiceNow processes, I believe it would be good to go ahead and wrap up our example Use Case at this point so that we can devote the remainder of our time exclusively on the Event Management aspects. All that is really left to do in order to to complete our address verification scenario is to add the form validation to the User Profile form. That process will leverage the new Script Include and Outbound REST Message that we have already completed.

There are two different ways that we can go about this: we can use an onSubmit Business Rule on the server side, or we can use an onSubmit Client Script on the client side. The Business Rule route is actually a little easier, as you have access to both the current and previous versions of the GlideRecord, and you can call the Script Include directly, without the need for GlideAjax. My problem with that, though, from a User Experience perspective, is that you have to submit the entire form to the server for processing, which then gets reloaded if you have validation issues. My preference is to validate the form right where it sits on the client side, before the form ever gets submitted to the server. For that, we need to build a Client Script.

In fact, for our purpose, we will need two Client Scripts, one an onLoad script and the other an onSubmit script. The reason that we have to have an onLoad script is because you do not have access to the previous field values on the client side like you do with a server-side Business Rule. We will need to snag those values and stuff them somewhere for safekeeping as soon as the form loads. The ServiceNow platform provides a place for just that sort of thing called the g_scratchpad. You can pretty much throw whatever you want in there and it will be available for use until the form is reloaded. The entire onLoad script is just a few short lines of code.

function onLoad() {
	g_scratchpad.originalStreet = g_form.getValue('street');
	g_scratchpad.originalCity = g_form.getValue('city');
	g_scratchpad.originalState = g_form.getValue('state');
	g_scratchpad.originalZip = g_form.getValue('zip');
}

That’s all there is to that. With those initial values safely tucked away, we can then pull them back out later on and refer to them in our onSubmit script to see if anything has changed. Before we start in on our onSubmit script, however, we need to go back to our Script Include and add just a bit of code. When we first built out Script Include, we did not set it up to be client callable, but we are going to need to do that now so that we can access it via GlideAjax. It’s a simple checkbox on the Script Include form, which triggers a slight change in the prototype for the script. That change is handled automatically for any new script, but since we have been working on ours for a while, checking the box may not actually alter any modified code. In that case, you just need to modify the prototype line yourself to look like this:

AddressValidationUtils.prototype = Object.extendsObject(AbstractAjaxProcessor, {

Also, you will need to go down to the very bottom of the script and insert a closing paren in between the final curly brace and the terminating semi-colon. Once that’s done, we can add a new function that we can call from the client side.

validateAddressViaClient: function() {
	var street = this.getParameter('sysparm_street');
	var city = this.getParameter('sysparm_city');
	var state = this.getParameter('sysparm_state');
	var zip = this.getParameter('sysparm_zip');
	return JSON.stringify(this.validateAddress(street, city, state, zip));
},

This function just grabs all of the parameters passed via GlideAjax and passes them to our existing function, and then turns the response object into a string that can be returned in the XML Ajax response. Now we are all set up to receive calls from the client side.

Being a validation script, our onSubmit script will need to prevent the submission of the form in the event that any validation errors are detected. Without an Ajax call back to the server, that’s easily accomplished by returning false from the onSubmit function. However, making an asynchronous Ajax call means you are leaving your function without the answer in hand, so you don’t know whether or not any issues were detected until the response comes back, which will be in a completely different thread. Although you could utilize the getXMLWait() method to simply wait for the answer, that approach is quite frowned upon in Client Scripts for a variety of reasons, so we do not want to go down that road. Instead, we will use a modified version of this technique.

The approach is to cancel submission of the form before making the Ajax call, and then when the response comes in, submit the form a second time if all is well. Of course, submitting the form again will launch the onSubmit script again, so we need to make it smart enough to know that this is the second submit and to not start the whole process all over again. To accomplish that, we use yet another g_scratchpad property to let the script know that validation has already taken place. Here is the main onSubmit script:

function onSubmit() {
	var submitForm = true;

	var actionName = g_form.getActionName();
	if (!g_scratchpad.isFormValid) {
		submitForm = checkAddress();
	}

	return submitForm;
}

On the first submit, we default the submitForm variable to true, capture the name of the button that was pushed to submit the form so that we can use it when we submit again later, and then check to see if validation has already taken place and passed by testing the isFormValid scratchpad property. In the case of the first submit, isFormValid has not been set to true, so we check the address to see if it needs validated. If it does, then the submitForm variable will be set to false, and the form will not be submitted.

Assuming that it does need to be validated, the checkAddress function will make the Ajax call and cancel the original form submission by returning false. When the Ajax response comes in, if the address is valid, the response function will then submit the form a second time using the saved actionName after setting the isFormValid scratchpad property to true. When the onSubmit function then starts again due to the second submit, it will bypass the address check and simply allow the form to submit due to the isFormValid scratchpad property being set to true.

It’s all a little convoluted, but it works. Here is the checkAddress function:

function checkAddress() {
	var submitForm = true;

	var street = g_form.getValue('street');
	var city = g_form.getValue('city');
	var state = g_form.getValue('state');
	var zip = g_form.getValue('zip');
	if (street || city || state || zip) {
		if (street != g_scratchpad.originalStreet || city != g_scratchpad.originalCity || state != g_scratchpad.originalState || zip != g_scratchpad.originalZip) {
			GlideUI.get().clearOutputMessages();
			var ga = new GlideAjax("AddressValidationUtils");
			ga.addParam('sysparm_name', 'validateAddressViaClient');
			ga.addParam('sysparm_street', street);
			ga.addParam('sysparm_city', city);
			ga.addParam('sysparm_state', state);
			ga.addParam('sysparm_zip', zip);
			ga.getXMLAnswer(processXMLAnswer);
			submitForm = false;
		}
	}

	return submitForm;
}

Just like in the main onSubmit function, we first default the submitForm variable to true, and then we grab all of values for the four address components off of the g_form object. The first thing that we check is if there is even any data in any of the four fields. If they are all empty, then there is nothing to validate. If there is some data there, then the next thing that we check is if it is any different than the values that we squirreled away in our onLoad script. If there are no changes, then again there is no need for validation. But if there is data there and it has changed in any way, now we are going to be making that GlideAjax call. Most of that is pretty standard GlideAjax stuff, but we also clear out any error messages on the screen from previous attempts and set the submitForm variable to false to kill the original form submission.

To process the Ajax response, we have yet another function, processXMLAnswer:

function processXMLAnswer(answer) {
	var response = JSON.parse(answer);
	if (response.result == 'invalid') {
		g_form.addErrorMessage('Unable to verify address entered using the US Address validation service');
		g_form.showFieldMsg('street', 'Unverifiable address', 'error');
		g_form.showFieldMsg('city', 'Unverifiable address', 'error');
		g_form.showFieldMsg('state', 'Unverifiable address', 'error');
		g_form.showFieldMsg('zip', 'Unverifiable address', 'error');
	} else {
		if (response.result == 'valid') {
			g_form.setValue('street', response.street);
			g_form.setValue('city', response.city);
			g_form.setValue('state', response.state);
			g_form.setValue('zip', response.zip);
		}
		g_scratchpad.isFormValid = true;
		g_form.submit(actionName);
	}
}

The response comes back in the form of a string, so the first thing that we have to do is convert it back to an object. If the result property of that object is ‘invalid’, then we leave the form unsubmitted and throw a few error messages up on the screen; otherwise, we are going to submit the form a second time. But before we do that, we check to see if the result property is ‘valid’, in which case we overlay the user’s input with the clean address values returned by the service. After that , we set our isFormValid scratchpad property to true and resubmit the form using the saved actionName.

It’s all a little complicated, having to pass through the onSubmit script twice due to the double submit, but it all makes sense if you really think about it. Of course, we won’t really know if it works until we try it, so let’s pull up a user’s profile and give it a shot.

For our first test, let’s see if we can get a validation error. We can use the test address that we have been using up to this point, but let’s change the state from FL to TX and let’s drop the zip code entirely. Also, let’s put everything in lower case, just for fun. OK, let’s see what happens.

Test using invalid address

Having both form-level and field-level error messages is probably a little overkill, but everything seems to have worked. Now, let’s change the state back to FL and see if that is enough to consider it a valid address.

User profile after address validation

Not only did it pass validation, it also corrected our capitalization and added the missing zip code. That’s actually pretty slick. This wasn’t really the point of this series, but I like this address validation feature. I was originally just looking for something that might have a failure to showcase the Event Management stuff, but this turned out to be a pretty handy little addition that could be useful in a number of other places, such as the Building and Location forms.

This now completes the example working feature that has the potential for failure. From this point on, we can focus on what we came here for, which is to log and process Events that originate in ServiceNow. Next time out, we will start in on the logging.

Fun with Outbound REST Events, Part IV

“Life is trying things to see if they work.”
Ray Bradbury

We got a little side-tracked last time, but we are finally ready to tackle the Script Include that will utilize our new Outbound REST Message. I like to tackle things a little bit at a time and test, so we will start out with just the code to invoke the service and format a response to the caller, and then deal with all of the error handling later on in the process. We will also want to call this from a Client Script, but again, let’s not worry about that part just yet and focus on successfully launching the service and interpreting the results.

To start with, we pull up the list of Script Includes and hit the New button to create a new script. We will call our new script AddressValidationUtils and create a function called validateAddress that takes street, city, state, and zip as arguments. The first thing that we will want to do in our new function will be to establish the object that will be returned to the caller.

var response = {result: 'failure', street: street, city: city, state: state, zip: zip};

Our result property will have three possible values: valid, invalid, or failure. We initially default it to failure, mainly because there will be a lot places where we might have to set it to failure, and then alter it to one of the other two values when appropriate. The remaining properties are defaulted to their original value, and if the address is deemed valid, will be updated with the values returned by the service to clean up any inconsistencies. Now let’s add the code to invoke the service.

var rest = new RESTMessage('US Street Address API', 'get');
rest.setStringParameter('authid', gs.getProperty('us.address.service.auth.id'));
rest.setStringParameter('authToken', gs.getProperty('us.address.service.auth.token'));
rest.setStringParameter('street', encodeURIComponent(street));
rest.setStringParameter('city', encodeURIComponent(city));
rest.setStringParameter('state', encodeURIComponent(state));
rest.setStringParameter('zip', encodeURIComponent(zip));
var resp = rest.execute();

This is all pretty straightforward: we establish the rest object by creating a new RESTMessage using the name of the message and the name of the method. Once the object has been established, we use the setStringParameter method to set the value for each of the required variables. Once again, the system does not encode any of the values, so we need to do that for any user input. We don’t need to do that for any of the credentials, as we already know the value of those and we know that there are no characters in either of those that would cause an issue with the URL. Once all of the variable value have been set, we then make the call to the service using the execute method.

Now we need to add code to interpret the response.

var body = resp.getBody();
if (resp.getStatusCode() == 200) {
	var respArray = [];
	try {
		respArray = JSON.parse(body);
	} catch (e) {
		gs.info('Exception: ' + e);
	}
	if (respArray && respArray.length > 0) {
		respObj = respArray[0];
		if (typeof respObj.analysis == 'object') {
			var validity = respObj.analysis.dpv_match_code;
			if (validity == 'Y' || validity == 'S' || validity == 'D') {
				response.result = 'valid';
				response.street = respObj.delivery_line_1;
				response.city = respObj.components.city_name;
				response.state = respObj.components.state_abbreviation;
				response.zip = respObj.components.zipcode;
				if (respObj.components.plus4_code) {
					response.zip += '-' + respObj.components.plus4_code;
				}
			} else {
				response.result = 'invalid';
			}
		} else {
			gs.info('Invalid Response Object: ' + JSON.stringify(respObj));
		}
	} else {
		gs.info('Invalid Response Body: ' + body);
	}
} else {
	gs.info('Invalid HTTP Response Code: ' + resp.getStatusCode());
}

There is a little more code here, but most of it is defensive, just to make sure that we don’t choke on something unexpected. All of the gs.info statements are just placeholders at this point; eventually we will replace those with Event logging, but I’m not ready to deal with that just yet, so we’ll put that off for now and just write to the system log. Assuming all goes well, we will have the response object in hand, and then all we have to do is check to see if it validated or not. If it did, we pass back their version of four address components, and if it didn’t, then we just indicate that is invalid.

Once get through all of that, all that is left is to return the response to the caller. Here’s the whole thing, all put together.

var AddressValidationUtils = Class.create();
AddressValidationUtils.prototype = {

	initialize: function() {
	},

	validateAddress: function(street, city, state, zip) {
		var response = {result: 'failure', street: street, city: city, state: state, zip: zip};

		var rest = new RESTMessage('US Street Address API', 'get');
		rest.setStringParameter('authid', gs.getProperty('us.address.service.auth.id'));
		rest.setStringParameter('authToken', gs.getProperty('us.address.service.auth.token'));
		rest.setStringParameter('street', encodeURIComponent(street));
		rest.setStringParameter('city', encodeURIComponent(city));
		rest.setStringParameter('state', encodeURIComponent(state));
		rest.setStringParameter('zip', encodeURIComponent(zip));
		var resp = rest.execute();
		var body = resp.getBody();
		if (resp.getStatusCode() == 200) {
			var respArray = [];
			try {
				respArray = JSON.parse(body);
			} catch (e) {
				gs.info('Exception: ' + e);
			}
			if (respArray && respArray.length > 0) {
				respObj = respArray[0];
				if (typeof respObj.analysis == 'object') {
					var validity = respObj.analysis.dpv_match_code;
					if (validity == 'Y' || validity == 'S' || validity == 'D') {
						response.result = 'valid';
						response.street = respObj.delivery_line_1;
						response.city = respObj.components.city_name;
						response.state = respObj.components.state_abbreviation;
						response.zip = respObj.components.zipcode;
						if (respObj.components.plus4_code) {
							response.zip += '-' + respObj.components.plus4_code;
						}
					} else {
						response.result = 'invalid';
					}
				} else {
					gs.info('Invalid Response Object: ' + JSON.stringify(respObj));
				}
			} else {
				gs.info('Invalid Response Body: ' + body);
			}
		} else {
			gs.info('Invalid HTTP Response Code: ' + resp.getStatusCode());
		}

		return response;
	},

	type: 'AddressValidationUtils'
};

Now all we need to do is test it out and make sure that it all works. Once again, we can use that same test case that was provided in their testing tool; we just have to pass the values as arguments to the function call. We can write a quick little stand-alone script for that that looks like this:

var avu = new AddressValidationUtils();
gs.info(JSON.stringify(avu.validateAddress('3901 SW 154th Ave', 'Davie', 'FL', '33331'), null, '\t'));

To actually run that, we can bring up the background script runner, paste it in, and click on the Run Script button.

Test script in the background script runner

… which gets us the following result:

{
	"result": "valid",
	"street": "3901 SW 154th Ave",
	"city": "Davie",
	"state": "FL",
	"zip": "33331-2613"
}

Not only did we validate the address, we also picked up the last four digits of the zip code, which was not a part of the original user input. And of course, we just proved that our new Script Include does, in fact, do what we intended for it to do. Chalk up another baby step along the path that we first laid out when we began this journey. So now what?

At this point, we actually have two different ways to go as far as our next step. Given that we have a working script to call when everything is working as it should, we could move on to the Client Script and see if we can now integrate this with the User Profile form, or we could turn our attention to those gs.info placeholders and start replacing those with calls to the function that logs Events. It all has to be done at some point, so it really doesn’t matter other than the fact that we have to choose. But, I guess we don’t have to choose right now. We can figure that out when it’s time to put together the next installment in the series.

Fun with Outbound REST Events, Part III

“If it’s a good idea, go ahead and do it. It is much easier to apologize than it is to get permission.”
Rear Admiral Grace Murray Hopper (“Amazing Grace”)

In the last installment in this series, we set up an Outbound REST Message and announced the intention of devoting this installment to the creation of a Script Include that would utilize the new REST message. In testing out that new Script Include, however, I ended up making a few tweaks to that REST message, so we should probably go over those first, just to bring everyone up to speed on what the REST message looks like today.

The first thing that I ended up changing was the name of the address variable. The web service uses the word street for that parameter, but for some reason I was thinking that ServiceNow called the property on the sys_user table address, so I wanted to adopt their name and not the one used by the web service. As it turns out, however, ServiceNow uses the word street as well, so I went into the REST message definition and changed it back to street in the end point URL and in the variable test value list.

The other thing that I ended up doing was adding a completely new variable to both the URL and the variable list called authToken. When we tested the GET method, we were using the auth-id from the web site’s testing tool, and we expected to get the 401 HTTP Response that we received. However, when testing the new Script Include with my own auth-id, I got the same thing. After rooting around in the documentation on the service’s web site, I discovered that server-side calls to the service require both an auth-id and an auth-token, both of which you receive when you sign up for their services. Once I figured that out, I added an auth-token parameter to the URL, added an authToken variable, and added a new System Property to store the value of the auth-token called us.address.service.auth.token.

After doing all of that, before getting back to the Script Include, I went ahead and tested the modified GET method using the Test link on the method form. That actually got me valid results, but they were a little hard to read.

Web Service Test Results

Fortunately, there is a handy little trick that you can use in ServiceNow to clean up that big, long string quite nicely. If you right click on the Response field label and select Configure Dictionary, you will be taken to the Dictionary Entry form for the Response field. Down at the bottom of the form, you will find the Related Lists. Open up the Attributes tab and then click on the New button at the top of the list. This will take you to the Dictionary Attribute form.

Dictionary Attribute form

Select JSON view from the drop-down list or pop-up selection and then type the word true in the Value field. Save the form, which will take you back to the Dictionary Attribute form, and from there you can use the back arrow on the form to return to the test results. Now you should see a new little icon next to the field label, and if you click on it, a new pop-up screen will appear with the contents of the Response field formatted for much easier reading.

Formatted JSON response

That’s a much better way to look at that data. The JSON View attribute is just one of many field attributes available on the platform. When you have some time, it’s a worthwhile exercise to go back into that selection list and take a look at all of the various choices. It’s very easy to try one or two out, just to see what they do. Some of them, like this one, can be quite useful.

Well, we never got around to actually working on the Script Include, but at least we are all caught up on the changes that I made to get to this point. Since the Script Include discussion will undoubtedly consume an entire post all on its own, I think this will be a good place to stop for now. We will tackle that Script Include in the next installment in this series.

Fun with Outbound REST Events

“A good programmer is someone who always looks both ways before crossing a one-way street.”
Doug Linder

A while back I mentioned that ServiceNow Event Management can be used within ServiceNow itself. I explained how all of that could work, but I never really came up with a real-world Use Case that would demonstrate the value of starting to wander down that road. I have code to generate Events in a lot of places that never get executed, but it is still there, just in case. One place where unwanted things do tend to happen, though, is when interacting with outside resources such as JDBC imports or external REST calls. Here, you are dependent on some outside database or system being up and available, and that’s not always the case, so you need to build in processes that can gracefully handle that. This is an excellent place for Event Management to step in and capture the details of the issue and log it for some kind of investigation or resolution.

So, I thought I should come up with something that anyone can also play around with on their own, that isn’t tied to some proprietary database or internal web service. I started searching for some kind of public REST API, and I stumbled across the Public APIs web site. There is actually a lot of cool stuff here, but I was looking for something relatively simple, and also something that would seem to have some relation to things that go on inside of ServiceNow. After browsing around a bit, I found the US Street Address API, which looked like something that I could use to validate street addresses in the User Profile. That seemed simple enough and applicable enough to serve my purpose, so that settled that.

There are quite a few parts and pieces to do everything that I want to do, so we will just take them on one at a time. Here is the initial list of the things that I think that I will need to accomplish all that I would like to do:

  • Create an Outbound REST Message using the US Street Address API as the end point,
  • Create a Script Include that will encapsulate all of the functions necessary to make the REST call, evaluate the response, log an Event (if needed), and return the results,
  • Create a Client Script on the sys_user table to call the Script Include if any component of the User’s address changes and display an error message if the address is not valid,
  • Create an Alert Management Rule to produce an Incident whenever the new Event spawns an Alert,
  • Test everything to make sure that it all works under normal circumstances, and then
  • Intentionally mangle the REST end point to produce a failure, thereby testing the creation of the Event, Alert, and Incident.

The first thing to do, then, will be to create the Outbound REST Message, but before we do that, let’s explore the web service just a little bit to understand what we are working with. To do that, there is a handy little API tester here. This will allow us to try a few things out and see what happens. First, let’s just run one of their provided test cases:

https://us-street.api.smartystreets.com/street-address?auth-id=21102174564513388&candidates=10&match=invalid&street=3901%20SW%20154th%20Ave&street2=&city=Davie&state=FL&zipcode=33331

The API is just a simple HTTP GET, and the response is a JSON object:

[
  {
    "input_index": 0,
    "candidate_index": 0,
    "delivery_line_1": "3901 SW 154th Ave",
    "last_line": "Davie FL 33331-2613",
    "delivery_point_barcode": "333312613014",
    "components": {
      "primary_number": "3901",
      "street_predirection": "SW",
      "street_name": "154th",
      "street_suffix": "Ave",
      "city_name": "Davie",
      "default_city_name": "Fort Lauderdale",
      "state_abbreviation": "FL",
      "zipcode": "33331",
      "plus4_code": "2613",
      "delivery_point": "01",
      "delivery_point_check_digit": "4"
    },
    "metadata": {
      "record_type": "S",
      "zip_type": "Standard",
      "county_fips": "12011",
      "county_name": "Broward",
      "carrier_route": "C006",
      "congressional_district": "23",
      "rdi": "Commercial",
      "elot_sequence": "0003",
      "elot_sort": "A",
      "latitude": 26.07009,
      "longitude": -80.35535,
      "precision": "Zip9",
      "time_zone": "Eastern",
      "utc_offset": -5,
      "dst": true
    },
    "analysis": {
      "dpv_match_code": "Y",
      "dpv_footnotes": "AABB",
      "dpv_cmra": "N",
      "dpv_vacant": "N",
      "active": "Y"
    }
  }
]

It looks like the response comes in the form of a JSON Array of JSON Objects, and the JSON Objects contain a number of properties, some of which are JSON Objects themselves. This will be useful information when we attempt to parse out the response in our Script Include. Now we should see what happens if we send over an invalid address, but before we do that, we should take a quick peek at the documentation to better understand what may affect the response. One input parameter in particular, match, controls what happens when you send over a bad address. There are two options;

  • strict  The API will return detailed output only if a valid match is found. Otherwise the API response will be an empty array.
  • invalid  The API will return detailed output for both valid and invalid addresses. To find out if the address is valid, check the dpv_match_code. Values of Y, S, or D indicate a valid address.

The default value in the provided tester is invalid, and that seems to be the appropriate setting for our purposes. Assuming that we will always use that mode, we will need to look for one of the following values in the dpv_match_code property to determine if our address is valid:

Y — Confirmed; entire address is present in the USPS data. To be certain the address is actually deliverable, verify that the dpv_vacant field has a value of N. You may also want to verify that the active field has a value of Y. However, the USPS is often months behind in updating this data point, so use with caution. Some users may prefer not to base any decisions on the active status of an address.
S — Confirmed by ignoring secondary info; the main address is present in the USPS data, but the submitted secondary information (apartment, suite, etc.) was not recognized.
D — Confirmed but missing secondary info; the main address is present in the USPS data, but it is missing secondary information (apartment, suite, etc.).

So, let’s give that a shot and see what happens. Let’s drop the state and zipcode from our original query and give that tester another try.

https://us-street.api.smartystreets.com/street-address?auth-id=21102174564513388&candidates=10&match=invalid&street=3901%20SW%20154th%20Ave&street2=&city=Davie

… which give us this JSON Array in response:

[
  {
    "input_index": 0,
    "candidate_index": 0,
    "delivery_line_1": "3901 SW 154th Ave",
    "last_line": "Davie",
    "components": {
      "primary_number": "3901",
      "street_predirection": "SW",
      "street_name": "154",
      "street_suffix": "Ave",
      "city_name": "Davie"
    },
    "metadata": {
      "precision": "Unknown"
    },
    "analysis": {
      "dpv_footnotes": "A1",
      "active": "Y",
      "footnotes": "C#"
    }
  }
]

This result doesn’t even have a dpv_match_code property, which is actually kind of interesting, but that would still fail a positive test for the values Y, S, or D, so that would make it invalid, which is what we wanted to see.

OK, I think we know enough now about the way things work that we can start building out out list of components. This is probably a good place to wind things up for this episode, as we can start out next time with the construction process, beginning with of our first component, the Outbound REST Message.

Testing ServiceNow Event Utilities

“Testing leads to failure, and failure leads to understanding.”
Burt Rutan

Now that we have put together a basic ServiceNow Event utility and added a few enhancements, it’s time to try it out and see what happens. There are actually two reasons that we would want to do this: 1) to verify that the code performs as intended, and 2) to see what happens to these reported Events once they are generated. We will want to test both the server side process and the client side process, so we will want a simple tool that will allow us to invoke both. One way to do that would be with a basic UI Page that contains a few input fields for Event data and a couple of buttons, one to report the Event via the server side function and another to report the Event using the client side function.

For the sake of simplicity, let’s just collect the description value from the user input and hard code all of the rest of the values. We could provide more options for input fields, but we’re just testing here, so this will be good enough to prove that everything works. We can always add more later. But for now, maybe just something like this:

Simple Event utility tester

The first thing that we will need is some HTML to lay out the page:

<?xml version="1.0" encoding="utf-8" ?>
<j:jelly trim="false" xmlns:j="jelly:core" xmlns:g="glide" xmlns:j2="null" xmlns:g2="null">
<script src="client_event_util.jsdbx"></script>
<div>
 <g:ui_form>
  <h4>Event Tester</h4>
  <label for="description">Enter some text for the Event details:</label>
  <textarea id="description" name="description" class="form-control"></textarea>
  <div style="text-align: center; padding: 10px;">
    <input class="btn" name="submit" type="submit" value="Client Side Test" onclick="clientSideTest();"/>
	 
    <input class="btn" name="submit" type="submit" value="Server Side Test"/>
  </div>
 </g:ui_form>
</div>
</j:jelly>

There’s really nothing too special here; just a single textarea and a couple of submit buttons, one for the client side and one for the server side. On the client side button we add an onclick attribute so that we can run the client side script. On the server side button, we just let the form submit to the server, and then run the server side script when we get to the other side. The client side script is similarly very simple stuff:

function clientSideTest() {
	ClientEventUtil.logEvent('event_tester', 'None', 'Client Event Test', 3, document.getElementById('description').value);
	alert('Event generated via Client Side function');
}

… as is the server side script:

if (submit == "Server Side Test") {
	new ServerEventUtil().logEvent('event_tester', 'None', 'Server Event Test', 3, description);
	gs.addInfoMessage('Event generated via Server Side function');
}

Now all we have to do is hit that Try It button on the UI page, enter some description text, and then click one of the submit buttons to see what happens. On the client side:

Client side Event test

… and on the server side:

Server side Event test

Now that we have generated the Events, we can verify that they were created by going into the Event Management section of the menu and selecting the All Events option. By inspecting the individual Events, you can also see that each Event triggered an Alert, and by setting up Alert Management Rules, these Alerts could drive subsequent actions such as creating an Incident or initiating some automated recovery activity. But now we are getting into the whole Event Management subsystem, which is way outside of the scope of this discussion. My only intent here was to demonstrate that your ServiceNow components can easily leverage the Event Management infrastructure built into the ServiceNow platform, and in fact, do it quite easily once you created a few simple utility modules to handle all of the heavy lifting. Hopefully, that objective has been achieved.

Just in case anyone might be interested in playing around with this code, I bundled the two scripts and the test page together into an Update Set.

Enhanced Event Management for ServiceNow

“Great things are done by a series of small things brought together.”
Vincent Van Gogh

The one property of a ServiceNow Event that we virtually skipped over last time was the additional_info property. This is pretty much a catch-all for any other thing that you might want to record along with the Event itself. The additional_info property is stored in the database as a JSON-formatted string, which you can instantiate in use and then access like any other Javascript object. By leveraging the additional_info property, we can inject standard elements into the Event so that the reporting module does not have to include the code to provide that information. One such bit of info could be details on the currently logged on user. Another might be a Stack Trace containing the details of how we arrived at the point of an Event occurring.

The one thing that we would not want to do, however, would be to overlay any information that the reporting entity has provided, so it will be important to first check for the presence of any data in the additional_info object before we set any values of our own. The first thing that we would have to do would be to check to see if an additional_info value was even provided, and that it was an object to which we could add additional values. Here is one way to approach such a check:

if (additional_info) {
	if (typeof additional_info != 'object') {
		additional_info = {info: additional_info};
	}
} else {
	additional_info = {};
}

This ensures that we have an object, and that we have preserved whatever non-object (string, boolean, number,etc.) values that may have been provided instead of an object. Once we know we have an object to work with, then we can check the object for other properties, and if not already provided, provide a standard value. For example, here is how we could potentially include the various details on the user:

if (!additional_info.user) {
	additional_info.user = {};
	additional_info.user.sys_id = gs.getUserID();
	additional_info.user.id = gs.getUserName();
	additional_info.user.name = gs.getUserDisplayName();
}

Injecting a Stack Trace could be handled in a similar fashion:

if (!additional_info.stackTrace) {
	additional_info.stackTrace = this.getStackTrace();
}

Of course, a server side Stack Trace is of little value if your issue is a client side Event, so you would probably want to snag a client side Stack Trace while you were on the client side, before you sent everything over to the server side to be reported. We can steal some of the code from our server side counterpart to enhance the client side function and turn it into something like this:

logEvent: function(source, resource, metric_name, severity, description, additional_info) {
	if (additional_info) {
		if (typeof additional_info != 'object') {
			additional_info = {info: additional_info};
		}
	} else {
		additional_info = {};
	}
	if (!additional_info.stackTrace) {
		additional_info.stackTrace = this.getStackTrace();
	}
	var ga = new GlideAjax('ServerEventUtil');
	ga.addParam('sysparm_name', 'logClientEvent');
	ga.addParam('sysparm_source', source);
	ga.addParam('sysparm_resource', resource);
	ga.addParam('sysparm_metric_name', metric_name);
	ga.addParam('sysparm_severity', severity);
	ga.addParam('sysparm_description', description);
	ga.addParam('sysparm_additional_info', JSON.stringify(additional_info));
	ga.getXML();
}

By creating a common Event reporting utility function and leveraging the additional_info property for specific selected values, virtually all of the Events reported by ServiceNow components can share a common set of properties. This creates opportunities for common Event processing scripts and generic reporting possibilities that would not exist if everyone were simply following their own unique approach to reporting Events. And once you establish an organizational standard for common values stored in the additional_info property, adding additional items of interest at a future point in time is simply a matter of updating the common routine that everyone calls to report Events.

We still need to put together that testing page that we talked about last time out, but at this point, I think that will have to be a project for another day

Update: There is an even better version here.

Event Management for ServiceNow

“If you add a little to a little, and then do it again, soon that little shall be much.”
Hesiod

The Event Management service built into ServiceNow is primarily designed for collecting and processing events that occur outside of ServiceNow. However, there is no reason that you cannot leverage that very same capability to handle events that occur in your own ServiceNow applications and customizations. To do that easily and consistently, it’s helpful to bundle up all of the code to make that happen into a function that can be called from a variety of potential users. A server-side Script Include can handle that quite nicely:

var EventUtil = Class.create();
EventUtil.prototype = {
	initialize: function() {
	},
	logEvent: function(source, resource, metric_name, severity, description) {
		var event = new GlideRecord('em_event');
		event.initialize();
		event.source = source;
		event.event_class = gs.getProperty('instance_name');
		event.resource = resource;
		event.node = 'ServiceNow';
		event.metric_name = metric_name;
		event.type = 'ServiceNow';
		event.severity = severity;
		event.description = description;
		event.insert();
	},
	type: 'EventUtil'
};

There are a number of properties associated with Events in ServiceNow. Here is the brief explanation of each as explained in the ServiceNow Event Management documentation:

VariableDescription
SourceThe name of the event source type. For example, SCOM or SolarWinds.
Source Instance (event_class)Specific instance of the source. For example, SCOM 2012 on 10.20.30.40
nodeThe node field should contain an identifier for the Host (Server/Switch/Router/etc.) that the event was triggered for. The value of the node field can be one of the following identifiers of the Host:
  • Name
  • FQDN
  • IP
  • Mac Address
If it exists in the CMDB, this value is also used to bind the event to the corresponding ServiceNow CI.
resourceIf the event refers to a device, such as, Disk, CPU, or Network Adapter, or to an application or service running on a Host, the name of the device or application must be populated in this field. For example, Disk C:\ or Nic 001 or Trade web application.
metric_nameUsed Memory or Total CPU utilization.
typeThe type of event. This type might be similar to the metric_name field, but is used for general grouping of event types.
message_keyThis value is used for de-duplication of events. For example, there might be two events for the same CI, where one event has CPU of 50% and the next event has CPU of 99%. Where both events must be mapped to the same ServiceNow alert, they should have the same message key. The field can be left empty, in which case the field value defaults to source+node+type+resource+metric_name. The message_key should be populated only when there is a better identifier than the default.
severitySeverity of the event. ServiceNow values for severity range from 1 – Critical to 5 – Info, with the severity of 0 – Clear. Original severity values should be sent as part of the additional information.
additional_infoThis field is in JSON key/value format, and is meant to contain any information that might be of use to the user. It does not map to a pre-defined ServiceNow event field. Examples include IDs of objects in the event source, event priority (if it is not the same as severity), assignment group information, and so on. Values in the Additional information field of an Event that are not in JSON key/value format are normalized to JSON format when the event is processed.
time_of_eventTime when the event occurred on the event origin. The format is: yyyy-MM-dd HH:mm:ss GMT
resolution_stateOptional – To indicate that an event has been resolved or no longer occurring, some event monitors use ‘clear’ severity, while other event monitors use a ‘close’ value for severity. This field is used for those monitors proffering the latter. Valid values are New and Closing.

Generating an Event in ServiceNow is simply writing a record to the em_event table. To reduce the amount of info that needs to be passed to the utility, our example function assumes a standard value for a number of properties of the Event, such as the event_class, node, and type, and leaves out completely those things that will receive a default value from the system such as message_key, time_of_event, and resolution_state. For our purpose, which is a means to generate internal Events within ServiceNow, we can accept all of those values as standard defaults. The rest will need to be passed in from the process reporting the Event.

For the source value, I like to use the name of the object (Widget, UI Script, Script Include, etc.) reporting the Event. For the resource value, I like to use something that describes the data involved, such as the Incident number or User ID. The source is the tool, and the resource is the specific data that is being processed by that tool. The other three data points that we pass are metric_name, severity, and description, all of which further classify and describe the event.

The example above takes care of the server side, but what about the client side? To support client-side event reporting, we can add an Ajax version of the function to our server-side Script Include, and then create a client-side UI Script that will make the Ajax call. The modified Script Include looks like this:

var ServerEventUtil = Class.create();
ServerEventUtil.prototype = Object.extendsObject(AbstractAjaxProcessor, {

	logClientEvent: function() {
		this.logEvent(
			this.getParameter('sysparm_source'),
			this.getParameter('sysparm_resource'),
			this.getParameter('sysparm_metric_name'),
			this.getParameter('sysparm_severity'),
			this.getParameter('sysparm_description'));
	},

	logEvent: function(source, resource, metric_name, severity, description) {
		var event = new GlideRecord('em_event');
		event.initialize();
		event.source = source;
		event.event_class = gs.getProperty('instance_name');
		event.resource = resource;
		event.node = 'ServiceNow';
		event.metric_name = metric_name;
		event.type = 'ServiceNow';
		event.severity = severity;
		event.description = description;
		event.insert();
	},

	type: 'ServerEventUtil'
});

To access this code from the client side of things, a new UI Script will do the trick:

var ClientEventUtil = {

	logEvent: function(source, resource, metric_name, severity, description, additional_info) {
		var ga = new GlideAjax('ServerEventUtil');
		ga.addParam('sysparm_name', 'logClientEvent');
		ga.addParam('sysparm_source', source);
		ga.addParam('sysparm_resource', resource);
		ga.addParam('sysparm_metric_name', metric_name);
		ga.addParam('sysparm_severity', severity);
		ga.addParam('sysparm_description', description);
		ga.getXML();
	},

	type: 'ClientEventUtil'
};

Now that we have created our utility functions to do all of the heavy lifting, reporting an Event is a simple matter of calling the logEvent function from the appropriate module. On the server side, that would something like this:

var seu = new ServerEventUtil();
seu.logEvent(this.type, gs.getUserID(), 'Unauthorized Access Attemtp', 3, 'User ' + gs.getUserName() + ' attempted to access ' + functionName + ' without the required role.');

On the client side, where we don’t have to instantiate a new object, the code is event simpler:

ClientEventUtil.logEvent('some_page.do', NOW.user.userID, 'Unauthorized Access Attemtp', 3, 'User ' + NOW.user.name + ' attempted to access ' + functionName + ' without the required role.');

To test all of this out, we should be able to build a simple UI Page with a couple of test buttons on it (one for the server-side test and one for the client-side test). This will allow us to both test the utility modules and also see what happens to the Events once they get generated. That sounds like a good project for next time out.