Category: Cool Stuff

Fun with Outbound REST Events, Part X

Posted on | by snhackery

“No one has a problem with the first mile of a journey. Even an infant could do fine for a while. But it isn’t the start that matters. It’s the finish line.”
— Julien Smith

After our last installment in this series, our Events now spawn Incidents that are pretty much just what we would like to see. The only remaining challenge at this point is to create a meaningful Description field value. Although we have set things up to produce this Description in a Script Include function, I should point out right here at the outset that everything that we are about to do in our script could also be accomplished in the Flow Designer itself. In fact, it probably should be done using the Flow Designer if we are to fully embrace the whole no-code future towards which we all seem to be herded. I’m still an old coder at heart, though, so it seems easier to me to scratch out another quick function than it does to build out all of those action steps using input forms. Still, it would probably be a worthwhile exercise to replace this script with a subflow one day; today is just not that day. Today we code!

Although we are passing the Alert to our function as an argument, much of the data we need is actually in the Event that spawned the Alert, so the first thing that we are going to want to do is go out and get that guy. That’s pretty basic GlideRecord stuff.

// get initial Event
var eventGR = new GlideRecord('em_event');
eventGR.addQuery('alert', alertGR);
eventGR.orderBy('sys_created_on');
eventGR.query();
eventGR.next();

Since it is possible that there could be more than one Event associated with our Alert, we include an orderBy directive to ensure that we get the very first Event out of the bunch. Once we have our Event in hand, we will have access to the additional_info JSON string, which we will want to convert to a Javascript object so that we can reference all of the various component parts.

// get addition info from Event
var additionalInfo = {};
try {
	additionalInfo = JSON.parse(eventGR.getValue('additional_info'));
} catch (e) {
	gs.info('Unable to parse additonal_info from Event ' + eventGR.number);
}

We also have access to the Event resource, which in our case is a User’s user_name. We can use that to get the sys_user record for that user, much in the same way that we retrieved the Event record.

// get affected User record
var userGR = new GlideRecord('sys_user');
userGR.get('user_name', alertGR.getValue('resource'));

This assumes, of course, that the only place that we using our address validation capability is on the User Profile page. If we ever expand its use to other places — say on the Building or Location form — then we would need to have some way to know whether the resource was a User or a Location or a Building or some other entity with an address to validate. Based on that information, we might be retrieving a Building record or a Location record instead of a User record. For now, though, we can safely assume that the resource is a User.

Now that we have gathered up all of the data that we need, we can start building out our Description. To begin, let’s start out with something that will be universal to all of our Incidents, regardless of the problem being reported.

// format description
var alertDesc = alertGR.getDisplayValue('description');
var section = '\n========================================\n';
var subsection = '\n----------------------------------------\n';
desc += additionalInfo.user.name;
desc += ' attempted to update the address on the user profile for user ';
desc += userGR.getDisplayValue('name');
desc += ', but was unable to verify the address using the US Address Validation service due to the following error:\n\n';
desc += alertDesc;
desc += '\n\nIncident Details:';
desc += section;

Beyond this point, we are going to want to be a little more specific based on what actually happened to trigger the Event. We can do that by introducing some conditional code based on the known values found in the Alert’s description field.

if (alertDesc.startsWith('The response code')) {
	// bad response code language will go here
} else if (alertDesc.startsWith('The response object')) {
	// bad response object language will go here
} else if (alertDesc.startsWith('The response content')) {
	// bad response content language will go here
} else {
	// we should never get here, but just in case ...
}

Since most of the Events that we have triggered up to this point have been of the bad response code variety, let’s do those first.

if (!additionalInfo.response.code) {
	desc += 'No response was received from the service, which could be an indication that the service is unavailable or unreachable. Check the status or the external service as well as the status of your connection to the Internet.';
} else if (additionalInfo.response.code == 401) {
	desc += 'A Response Code of 401 indicates an authentication error of some kind. Verify that your account credentials are correct and that your account is in good standing with the service provider.';
} else {
	desc += 'The service returned a Response Code of ';
	desc +=  additionalInfo.response.code;
	desc += '. Additional information on the meaning of this response may be found in the Response Body. Also, you can check with the service provider for further clarification on the appropriate handling of this response.';
	desc += '\n\nDetailed information on the ';
	desc += additionalInfo.response.code;
	desc += ' Response Code can be found on the web at https://httpstatuses.com/';
	desc += additionalInfo.response.code;
}

This gives us specialized language for no response code at all, and a response code of 401. Everything else is handled in a more generalized section that covers any other bad response code. As more knowledge of the potential response codes becomes available through experience with the service, more specialized language can be added that can be more specific to other known response codes.

Now let’s take a look at what we can do for bad response objects.

desc += 'The service returned a valid Response Code and a parsable response, but the response did not contain certain expected elements necessary to determine the validity of the address. Review the response received and check with the service provider to see if anything has changed with the API specifications.';

That one is about as simple as you can get; everyone gets the same language. For the bad response content issues, things are a little bit more sophisticated. Everyone still gets the same language, but there is a possibility for an exception with this group, so we include code to handle that as well.

desc += 'The service returned a valid Response Code, but the response was either empty or ill-formatted. Review the response received and check with the service provider to see if the service is experiencing problems, or if anything has changed with the API specifications.';
if (additionalInfo.exception) {
	desc += '\n\nException Details:';
	desc += subsection;
	desc += '   Exception: ';
	desc += additionalInfo.exception;
	desc += '\n   Stack Trace:\n';
	desc += additionalInfo.stackTrace;
}

Once we complete all of the conditional logic, we wrap things up with some more universal code that applies to everyone. This just serves to include the user’s input and the service’s response at the end of the body of the Description field for reference.

desc += '\n\nAddress Details:';
desc += subsection;
desc += '   Street: ';
desc += additionalInfo.input.street;
desc += '\n   City: ';
desc += additionalInfo.input.city;
desc += '\n   State: ';
desc += additionalInfo.input.state;
desc += '\n   Zip Code: ';
desc += additionalInfo.input.zip;
desc += '\n\nResponse Details:';
desc += subsection;
desc += '   Response Code: ';
desc += additionalInfo.response.code;
desc += '\n   Response Body: ';
desc += additionalInfo.response.content;
if (additionalInfo.response.object) {
	desc += '\n   Response Object: ';
	desc += JSON.stringify(additionalInfo.response.object, null, '\t');
}

There is still more helpful information that we could add, such as links to the service provider’s documentation, or in the case of the 401 error, the names of the system properties that contain the credentials, but this is good enough for a sample. Let’s just save what we have and then trigger another Event and see what comes out the other side.

Incident with updated description from the new Script Include function

Well, that’s much, much better than the description that the original Create Incident flow was producing. It’s not perfect, but I think it does provide the person receiving the Incident enough details about both what happened and what might be done about it that they can get to work on the ticket right away without a whole lot of research. Obviously, it can be fine-tuned over time, but this is a good foundation upon which to build for this particular use case.

That pretty much wraps up all that I had hoped to accomplish with this series. It took us 10 installments to get here, but much of that was due to the fact that we had to build out our own address validation infrastructure before we could use it to demonstrate applying Event Management tools and techniques to internal ServiceNow features and functions. For those of you who like to play along at home, I have bundled what I hope are all of the relevant parts and pieces into an Update Set that you are welcome to pull down and import into your own environment.

Fun with Outbound REST Events, Part IX

Posted on | by snhackery

“What we hope ever to do with ease, we must first learn to do with diligence.”
— Samuel Johnson

Last time, we were able to have our Alert produce an Incident, but it wasn’t exactly the Incident that we wanted. Today, we are going to fix that. Since we don’t want to alter the out-of-the-box Create Incident subflow that we are currently using to create our Incidents, we will want to make a copy of the subflow so that we can customize it for our own purposes. To do that, pull up the Create Incident subflow in the Flow Designer, click on the vertical ellipses in the upper right-hand corner, and select Copy subflow to create a new copy of the subflow.

Copy the Create Incident subflow

A pop-up dialog box will appear where you can enter the name of your new subflow, which we will call Create Address Issue Incident.

Enter the name of the new subflow

After entering the name, click on the Copy button to create your new subflow from the original. This should open up your new subflow for editing.

Your new Create Address Issue Incident subflow

Now that we have our own copy, we can make whatever modifications that we would like to make without disturbing the original. All of the changes that we will want to make are in the Create Task step, so let’s open that up and see what we can do to produce Incidents that include the detail that we would like to provide to the technician working the ticket. Let’s get rid of the Description value entirely, as that’s not the description that we want. That very same text is repeated in the Additional Comments field, anyway. The rest of the values that are there seem to be OK for now, but let’s add a few more using the +Add Field Value button at the bottom of the field list.

Let’s set the State to Assigned, the Assignment Group to ITSM Engineering, the Category to Software, and the Subcategory to Internal Application. Some of that may not be exactly right, but this is just an example of the kinds of things that you can do. For the new Description value, which is going to be conditional depending on the nature of the issue that triggered this Incident, let’s use an inline script. That can be done by clicking on the little f(x) button to the right of the field value.

Create Task step expanded

At this point, we don’t necessarily need to build the entire script, but we will want to stub it out enough to keep things functional for testing. Since the script might get a little complex, I like to push all of the logic out to a Script Include and then limit the code in the subflow to just a call to a function in the Script Include. That keeps the clutter out of the subflow itself, and also allows us to refine the output of the process by just editing the Script Include without having to publish a new version of the subflow. We already have a Script Include devoted to the address verification process, so let’s just add a simple function to that existing artifact so that we have something that we can call in the subflow.

formatIncidentDescription: function(alertGR) {
	return 'Test description for ' + alertGR.number;
},

There isn’t much to this at this point, but there is enough here to verify that we are receiving the Alert, which we will want as a reference when we start building out the actual description that we want. Getting back to our subflow, the script to invoke this new function will look like this:

var avu = new AddressValidationUtils();
return avu.formatIncidentDescription(fd_data.subflow_inputs.ah_alertgr);

Figuring out that fd_data.subflow_inputs.ah_alertgr was the correct syntax for referencing the Alert was not all that intuitive. I have worked with the Flow Designer long enough now to know about the fd_data object, but I couldn’t find much in the way of documentation on identifying the names of the various properties of that object. Fortunately, I did come across some documentation on the type-ahead feature, which finally led me to the information for which I had been searching. Typing a single dot after the fd_data brings up a nice pick list of choices, and another one after selecting the choice does the same for that object as well.

fd_data properties pick list

With our script in place for the Description value, all that is left is to Save and Publish the new subflow and we are done with the Forms Designer. To use our new subflow, we will need to go back into our Alert Management Rule, open up the Action section, and replace the Create Incident subflow with our modified copy.

Modifying our rule to use the newly created subflow

At this point, we should be good to test again and see what kind of Incident gets generated now. Just remember to select a different person so that our Event is not assigned to an existing Alert that has already been processed. We want to be sure to create a brand new Alert to activate our modified rule. Once we force a new Event, we can pull it up and take a look at it to see what values have been set in the Event record.

Newly generated Event record

From the Event, we can navigate to the Alert, and from the Alert we can then navigate to the Incident. Let’s check out the Incident.

Incident generated from our new subflow

This is an improvement over our initial effort, as the ticket has now been properly categorized and routed to an Assignment Group for resolution. We still need a much better Description, but the presence of the Alert ID in the current Description value verifies that we are indeed passing the Alert record to our stubbed-out function, which we can now use to produce a more detailed and informative description value. Scripting that out for all of the various possibilities will be a bit of an effort, through, so let’s just make that the focus of our next installment.

Fun with Outbound REST Events, Part VIII

Posted on | by snhackery

“Computer science education cannot make anybody an expert programmer any more than studying brushes and pigment can make somebody an expert painter.”
—Eric S. Raymond

Now that we have completed our address verification feature, added Event logging, and tested the creation of those Events, it’s time to actually do something with the Events when they come out. Well, actually, we won’t be doing anything with the Events themselves; we will be doing something with the Alerts that come out as a result of the Events. To process those Alerts, we will need to create a new Alert Management Rule.

To create a new rule, pull up the list of Alert Management Rules and click on the New button at the top of the list. The form is divided into three sections and the first section is the Alert info section. In that section, you will want to enter the name of the Alert, a description of the Alert, and you will want to set the Multiple alert rules field to Stop search for additional rules. This will prevent additional rules from evaluating or taking action on your Alert, as your rule should handle everything that needs to be done and no further rules should be applied.

Alert Info section of Alert Management Rule form

After completing the Alert info section, use the progress bar at the top of the form to move on to the next section of the form, the Alert Filter section. This is where you specify which Alerts you would like to process with this rule. In our case, we want to handle anything that comes out of our Script Include, which is identified in the Alert in the Source field. That makes our filter quite simple, as we only want to process Alerts where the Source is AddressValidationUtils:

Alert Filter section of Alert Management Rule form

After completing the Alert Filter section, use the progress bar at the top of the form once again to move on to the next section of the form, the Action section. This is where you specify what action should be taken whenever a new Alert is created that meets your filter criteria. There are quite a lot of things that you can do here, but we want to create an Incident. Fortunately for us, there is a built-in, out-of-the-box Subflow already developed that does exactly that. To select this Subflow, called Create Incident, double-click on the Insert new row … line to open up a new row and then double-click on the Subflow column of the newly inserted row to select the Create Incident subflow from the selection list.

Action section of Alert Management Rule form

After completing all three sections of the form, click on the Submit button to save your new rule. Once the rule has been created and saved, it is now active in the system, and the next time any Events are logged by our Script Include, the rule will be triggered. Let’s go ahead and do that now, just to see what happens.

We want to trigger an Event, so we can mangle our credentials again and then update someone’s address, which should do the trick. We will want to select a different person for this test, just to make sure that we trigger a brand new Alert, and not just have our Event associated with an existing Alert from any of our previous testing. Once we submit the address change, we can find our Event, and from there, navigate to the Alert.

Alert resulting from address service Event

Unlike all of our previous Alerts, this new one now has an Incident number in the Task field. This is the Incident that was generated from the execution of our new Alert Management Rule. Click on the info icon to the right of the Task field and then click on the Open Record button in the resulting pop-up window to bring up the Incident.

Incident generated from the Alert Management Rule

This may not be exactly the Incident that we would like to see, but we are taking things one step at a time, and we just produced an Incident from our Alert, which is a huge step in and of itself. Now let’s take a look at this Incident and see where we might make things a little better.

One of the first things that you might notice is that there is no Assignment Group, so it has not been routed to anyone for resolution. We should know to whom this Incident should be assigned, which might be the ServiceNow support team or at a minimum, the Service Desk, so we should populate that field right from the start. If we do that, then we should also set the State to Assigned rather than New.

We should also use a more appropriate Category, but the biggest improvement that we could make would be in the Description. When you generate an Incident via Event Management, you want to do as much as you can to explain both what happened and what can be done about it to the technician who will end up having to work the Incident. The Description that we are generating right now really doesn’t do that at all. We can do much better.

All of these fields are populated in the Create Incident flow that we assigned to our rule. Since that’s an out-of-the-box generic flow, we don’t really want to modify it, but we can make our own copy of it and then make whatever changes we want to make to our copy. That sounds like a bit of a project, though, so let’s make that exercise the subject or our next installment.

Fun with Outbound REST Events, Part VII

Posted on | by snhackery

“It is not the critic who counts; not the man who points out how the strong man stumbles, or where the doer of deeds could have done them better. The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood; who strives valiantly; who errs, who comes short again and again, because there is no effort without error and shortcoming; but who does actually strive to do the deeds; who knows great enthusiasms, the great devotions; who spends himself in a worthy cause; who at the best knows in the end the triumph of high achievement, and who at the worst, if he fails, at least fails while daring greatly, so that his place shall never be with those cold and timid souls who neither know victory nor defeat.”
— Theodore Roosevelt

Now that we have added the code to log all of our potential Events, we need to test that code out to make sure that it actually works. The only way to do that is for something to happen to trigger the logging of the Event. Some errors are easier to produce than others, so we might as well start out with an easy one first.

Probably the easiest of all, particularly since we have already done this in our earlier testing, is to force an invalid HTTP Response Code. We accomplished that when we were testing our Outbound REST Message by having the wrong credentials for the service. That got us a 401 response code instead of the desired 200. Since we are storing our credential values in System Properties, all we need to do in order to force a 401 response code is to change the value of one or both of those properties. Let’s do that now.

Updating the credentials properties

Now all that we need to do is make an address change on some User Profile and see what happens. Since our approach to service failures was to allow the update to proceed without address validation, you won’t really see anything when you update the user’s record. To find out if an Event was actually generated from the issue, we will have to take a peek at the Events table. The easiest way to do that is to select the All Events option from the left-hand navigation. Sure enough, our new Event is now sitting out there. Let’s take a look.

Event generated from address service failure

Everything looks to be in order, and thanks to the Event logging utility that we were able to leverage, there is data populated in the Event that we did not have to pass in ourselves. The JSON data in the Additional Info field is a little hard to read, but we have already gone over a quick fix for that. We should go ahead and do that same thing here.

Additional Info formatted using the JSON View Dictionary Attribute

That’s much better.

One other thing that you may have noticed is that logging this Event generated an Alert. Let’s take a look at the Alert now by clicking on the little info icon on the right side of the Alert field and then clicking on the Open Record button in the pop-up window.

The Alert generated from logging the Event

One of the things that you may have noticed is that ServiceNow generated a Message Key for our Event by combining a number of other Event properties. The generated message key for this Event is:

AddressValidationUtils_ServiceNow_ServiceNow_alene.rabeck_Invalid Response Code

If you do not supply a Message Key of your own, then one will be generated for you by combining the Source, Node, Type, Resource, and Metric Name. ServiceNow collects all Events with the same Message Key under a single Alert. This prevents multiple actions from being initiated for the same issue. For example, if a user attempted to update the profile of the same User multiple times, an Event would be logged for every failed attempt to reach the address validation service. However, all of those Events would be associated with a single Alert, so only one remediation action would be invoked. On the other hand, if an update was attempted for a different User, any Events logged as a result of that activity would be consolidated under a different Alert, as the Resource (the User, in our example) would be different, which would generate a different Message Key.

Another thing that you may have noticed is that there is no Task associated with this Alert. Tasks can be generated from Alerts using Alert Management Rules, but there are currently no rules in place that apply to this Alert, so no further action was taken. Before we are through with this exercise, we will be building a rule to spawn Incidents from our Alerts, but that’s not today’s concern. Today I want to focus on the testing of our Events.

We added code to our Script Include to log 4 different kinds of Events, and so far, we have only tested one of those, the invalid HTTP Response Code. The other three all have something to do with the response content returned from the service, which makes it a little more difficult to test, since we have no control over the response returned from the service. To test these other three, we will we need to add some temporary code to alter the response that comes back from the service to something that will trigger each of our other Events. We can add that code right after we get the actual response from the service and then alter it to force an error for testing purposes. Here is the original line of code that grabs the response content along with our alterations to produce an error condition:

var body = resp.getBody();
// temporary test code (remove after testing)
body = '[';
// end temporary test code

That value should trigger the unparsable response error. Now, all we need to do to test it is to issue an address change and then check the Events table for the resulting Event. To trigger the invalid response content error, you can change the inserted line to this:

body = '[]';

Now the response is parsable, but it is empty, which should take us to our third error condition. To get to the fourth, we can alter it again to this:

body = '[{}]';

Now the response is parsable and the array contains a single element, so that should get us past the earlier two issues. Since the object does not have an analysis property, though, that should drop us into our fourth error condition, which should log yet a different Event.

Once you complete all of your testing, you will want to go back into the code and remove all of the lines we added for testing purposes, and then test one more time, just to make sure that everything is now back working as it should. With that out of the way, we have now completed the testing for all of our recent changes.

Now that we are successfully logging all of these Events, we are going to want to do something with them. That process deserves an installment devoted exclusively to that effort, so we will leave that exercise for our next time out.

Fun with Outbound REST Events, Part VI

Posted on | by snhackery

“Quality is never an accident. It is always the result of intelligent effort.”
— John Ruskin

Now that we have completed our address verification feature, we can finally turn our full and complete attention to the actual purpose of this entire adventure, which which is to explore the use of ServiceNow Event Management practices on the internal workings of ServiceNow itself. When we last left our Script Include, we had identified a number of places in the script where things could potentially go wrong. As a temporary measure, we just put a simple gs.info statement in each one of those places. Now we want to replace those with Event logging so that we can leverage the built-in power of the ServiceNow Event Management infrastructure.

To make that easier, we built a utility a while back to handle much of the heavy lifting of logging an Event. We can take advantage of that utility and minimize the code that we will need to our Script Include. Each gs.info statement will need to be replaced with something like this:

var seu = new ServerEventUtil();
seu.logEvent(source, resource, metric_name, severity, description, additional_info);

Now we just need to figure out what values to send for each of those function arguments. Let’s take them one at a time.

source

This is the source of the Event, which is our case is the Script Include that is logging the Event. Since the name of the Script Include is always stored in an internal property called type, I just like to pass this.type for this argument, which works in all Script Includes without modification.

resource

This is a reference to thing that you were working on when the problem occurred. In our case, this would be a User, but in the current configuration, we do not have a handle on the User record that is being updated. We could use the address here, just to have some kind of unique value, but when we turn this Event into an Incident, it would be good to know which User was being updated. The solution to that would be to have the calling script pass some reference to User record as an additional argument to the function. That’s a little more work, but it will be worth it in the long run.

metric_name

This is basically the problem that occurred, and we will end up with a different value here for different issues such as an unparsable JSON string or a bad HTTP Response Code.

severity

This is just your standard severity values, and for our purposes, I think we will just pass a hard-coded 3 (Moderate) here.

description

As the name implies, this is just a text description of what happened. Ours will be unique to the problem that occurred.

additional_info

This is an open-ended JSON object into which you can stuff basically anything that you might want to know about what happened that isn’t already in a defined property. The Event logging utility automatically adds some standard things to this object such as user information and a stack trace, but we will want to add some additional information as well such as what was sent to the service and what came back. It takes a bit of code to construct the additional info object, so I like to build a function for that so that it can be called from wherever it is needed instead of duplicating the code everywhere. Here is the one that we will add for this exercise:

buildAdditionalInfo: function(input, response, respObject, exception) {
	var additionalInfo = {input: {}, response: {}};

	additionalInfo.input.street = input.street;
	additionalInfo.input.city = input.city;
	additionalInfo.input.state = input.state;
	additionalInfo.input.zip = input.zip;
	additionalInfo.response.code = response.getStatusCode();
	additionalInfo.response.content = response.getBody();
	additionalInfo.response.headers = response.getHeaders();
	if (respObject) {
		additionalInfo.response.object = respObject;
	}
	if (exception) {
		additionalInfo.exception = exception.toString();
		additionalInfo.stackTrace = exception.stack;
	}

	return additionalInfo;
},

Using a function for this not only consolidates the code into a single place, it also ensures some consistency between the various Events, which makes it easier to pull the data back out when you want to use it for things like formatting the description of a resulting Incident.

Now that know how we are going to populate these arguments, let’s go down through the code and replace each of our gs.info statements with Event logging. The first one that we come across is the JSON parsing exception.

try {
	respArray = JSON.parse(body);
} catch (e) {
	seu.logEvent(
		this.type,
		user,
		'Unparsable response',
		3,
		'The response content received from the US Address validation service could not be parsed.',
		this.buildAdditionalInfo(response, resp, null, e));
}

At this point in the process, we do not have a response object, but we do have an exception, so we pass null as the response object to the function that builds out the additional info. All of the others will be very similar, so we don’t have to go through each one individually. Here is the complete function, with all of the gs.info statements replaced and the user identifier added as a function argument:

validateAddress: function(user, street, city, state, zip) {
	var response = {result: 'failure', street: street, city: city, state: state, zip: zip};

	var seu = new ServerEventUtil();
	var rest = new RESTMessage('US Street Address API', 'get');
	rest.setStringParameter('authid', gs.getProperty('us.address.service.auth.id'));
	rest.setStringParameter('authToken', gs.getProperty('us.address.service.auth.token'));
	rest.setStringParameter('street', encodeURIComponent(street));
	rest.setStringParameter('city', encodeURIComponent(city));
	rest.setStringParameter('state', encodeURIComponent(state));
	rest.setStringParameter('zip', encodeURIComponent(zip));
	var resp = rest.execute();
	var body = resp.getBody();
	if (resp.getStatusCode() == 200) {
		var eventLogged = false;
		var respArray = [];
		try {
			respArray = JSON.parse(body);
		} catch (e) {
			seu.logEvent(
				this.type,
				user,
				'Unparsable response',
				3,
				'The response content received from the US Address validation service could not be parsed.',
				this.buildAdditionalInfo(response, resp, null, e));
			eventLogged = true;
		}
		if (respArray && respArray.length > 0) {
			respObj = respArray[0];
			if (typeof respObj.analysis == 'object') {
				var validity = respObj.analysis.dpv_match_code;
				if (validity == 'Y' || validity == 'S' || validity == 'D') {
					response.result = 'valid';
					response.street = respObj.delivery_line_1;
					response.city = respObj.components.city_name;
					response.state = respObj.components.state_abbreviation;
					response.zip = respObj.components.zipcode;
					if (respObj.components.plus4_code) {
						response.zip += '-' + respObj.components.plus4_code;
					}
				} else {
					response.result = 'invalid';
				}
			} else {
				seu.logEvent(
					this.type,
					user,
					'Invalid Response Object',
					3,
					'The response object received from the US Address validation service was not valid.',
					this.buildAdditionalInfo(response, resp, respObj));
			}
		} else {
			if (!eventLogged) {
				seu.logEvent(
					this.type,
					user,
					'Invalid Response Content',
					3,
					'The response content received from the US Address validation service was not valid.',
					this.buildAdditionalInfo(response, resp));
			}
		}
	} else {
		seu.logEvent(
			this.type,
			user,
			'Invalid Response Code',
			3,
			'The response code received from the US Address validation service was not valid.',
			this.buildAdditionalInfo(response, resp));
	}

	return response;
},

The one place where we had to add a little bit of extra logic was the Event that is triggered when the respArray is empty. One possible reason for that field to be empty would be if we failed to successfully parse the JSON string. When that happens, we have already logged an Event, so we would not want to now log a second one for the same issue. To prevent that from happening, we added the eventLogged variable, and then we only log an Event later on if that variable is still set to false. Other than that one special circumstance, all of these are pretty much the same other than the unique values that are specific to the particular problem triggering the Event.

That completes the modifications necessary to support Event logging, but since we added the user identifier to the list of function arguments, we still have a little work to do to carry that change forward through all of the other components. To begin, we will have to collect the user value from the Ajax parameters and pass that on to the primary function. That client callable function now looks like this:

validateAddressViaClient: function() {
	var user = this.getParameter('sysparm_user');
	var street = this.getParameter('sysparm_street');
	var city = this.getParameter('sysparm_city');
	var state = this.getParameter('sysparm_state');
	var zip = this.getParameter('sysparm_zip');
	return JSON.stringify(this.validateAddress(user, street, city, state, zip));
},

Not much change here; we pull one more parameter into a variable and then add that variable to the function call arguments. Of course, none of that will do any good if we don’t send that extra parameter with the Ajax call, so we will need to modify our Client Script as well. Again, there is not much to change here, but we need to make the change. Our code to value the parameters now has one additional line:

ga.addParam('sysparm_name', 'validateAddressViaClient');
ga.addParam('sysparm_user', g_form.getValue('user_name'));
ga.addParam('sysparm_street', street);
ga.addParam('sysparm_city', city);
ga.addParam('sysparm_state', state);
ga.addParam('sysparm_zip', zip);

That completes the changes that we need to make in order to log an Event whenever something unexpected occurs. We still need to test everything to make sure that it all works, but to do that, we are going to have to force some kind of error to occur. That sounds like a project in and of itself, so this seems like a good stopping place for now. We’ll figure out all of that testing stuff in our next installment.

Fun with Outbound REST Events, Part V

Posted on | by snhackery

“The best preparation for good work tomorrow is to do good work today.”
— Elbert Hubbard

At the end of the last installment in this series, I mentioned two possible options for the next item on which to focus our energies. At the time, I wasn’t really sure which direction would be the preferable choice, but now that it is time to fish or cut bait, a decision needs to be made. Given that the entire purpose of this exercise is to demonstrate the use of Event Management practices on internal ServiceNow processes, I believe it would be good to go ahead and wrap up our example Use Case at this point so that we can devote the remainder of our time exclusively on the Event Management aspects. All that is really left to do in order to to complete our address verification scenario is to add the form validation to the User Profile form. That process will leverage the new Script Include and Outbound REST Message that we have already completed.

There are two different ways that we can go about this: we can use an onSubmit Business Rule on the server side, or we can use an onSubmit Client Script on the client side. The Business Rule route is actually a little easier, as you have access to both the current and previous versions of the GlideRecord, and you can call the Script Include directly, without the need for GlideAjax. My problem with that, though, from a User Experience perspective, is that you have to submit the entire form to the server for processing, which then gets reloaded if you have validation issues. My preference is to validate the form right where it sits on the client side, before the form ever gets submitted to the server. For that, we need to build a Client Script.

In fact, for our purpose, we will need two Client Scripts, one an onLoad script and the other an onSubmit script. The reason that we have to have an onLoad script is because you do not have access to the previous field values on the client side like you do with a server-side Business Rule. We will need to snag those values and stuff them somewhere for safekeeping as soon as the form loads. The ServiceNow platform provides a place for just that sort of thing called the g_scratchpad. You can pretty much throw whatever you want in there and it will be available for use until the form is reloaded. The entire onLoad script is just a few short lines of code.

function onLoad() {
	g_scratchpad.originalStreet = g_form.getValue('street');
	g_scratchpad.originalCity = g_form.getValue('city');
	g_scratchpad.originalState = g_form.getValue('state');
	g_scratchpad.originalZip = g_form.getValue('zip');
}

That’s all there is to that. With those initial values safely tucked away, we can then pull them back out later on and refer to them in our onSubmit script to see if anything has changed. Before we start in on our onSubmit script, however, we need to go back to our Script Include and add just a bit of code. When we first built out Script Include, we did not set it up to be client callable, but we are going to need to do that now so that we can access it via GlideAjax. It’s a simple checkbox on the Script Include form, which triggers a slight change in the prototype for the script. That change is handled automatically for any new script, but since we have been working on ours for a while, checking the box may not actually alter any modified code. In that case, you just need to modify the prototype line yourself to look like this:

AddressValidationUtils.prototype = Object.extendsObject(AbstractAjaxProcessor, {

Also, you will need to go down to the very bottom of the script and insert a closing paren in between the final curly brace and the terminating semi-colon. Once that’s done, we can add a new function that we can call from the client side.

validateAddressViaClient: function() {
	var street = this.getParameter('sysparm_street');
	var city = this.getParameter('sysparm_city');
	var state = this.getParameter('sysparm_state');
	var zip = this.getParameter('sysparm_zip');
	return JSON.stringify(this.validateAddress(street, city, state, zip));
},

This function just grabs all of the parameters passed via GlideAjax and passes them to our existing function, and then turns the response object into a string that can be returned in the XML Ajax response. Now we are all set up to receive calls from the client side.

Being a validation script, our onSubmit script will need to prevent the submission of the form in the event that any validation errors are detected. Without an Ajax call back to the server, that’s easily accomplished by returning false from the onSubmit function. However, making an asynchronous Ajax call means you are leaving your function without the answer in hand, so you don’t know whether or not any issues were detected until the response comes back, which will be in a completely different thread. Although you could utilize the getXMLWait() method to simply wait for the answer, that approach is quite frowned upon in Client Scripts for a variety of reasons, so we do not want to go down that road. Instead, we will use a modified version of this technique.

The approach is to cancel submission of the form before making the Ajax call, and then when the response comes in, submit the form a second time if all is well. Of course, submitting the form again will launch the onSubmit script again, so we need to make it smart enough to know that this is the second submit and to not start the whole process all over again. To accomplish that, we use yet another g_scratchpad property to let the script know that validation has already taken place. Here is the main onSubmit script:

function onSubmit() {
	var submitForm = true;

	var actionName = g_form.getActionName();
	if (!g_scratchpad.isFormValid) {
		submitForm = checkAddress();
	}

	return submitForm;
}

On the first submit, we default the submitForm variable to true, capture the name of the button that was pushed to submit the form so that we can use it when we submit again later, and then check to see if validation has already taken place and passed by testing the isFormValid scratchpad property. In the case of the first submit, isFormValid has not been set to true, so we check the address to see if it needs validated. If it does, then the submitForm variable will be set to false, and the form will not be submitted.

Assuming that it does need to be validated, the checkAddress function will make the Ajax call and cancel the original form submission by returning false. When the Ajax response comes in, if the address is valid, the response function will then submit the form a second time using the saved actionName after setting the isFormValid scratchpad property to true. When the onSubmit function then starts again due to the second submit, it will bypass the address check and simply allow the form to submit due to the isFormValid scratchpad property being set to true.

It’s all a little convoluted, but it works. Here is the checkAddress function:

function checkAddress() {
	var submitForm = true;

	var street = g_form.getValue('street');
	var city = g_form.getValue('city');
	var state = g_form.getValue('state');
	var zip = g_form.getValue('zip');
	if (street || city || state || zip) {
		if (street != g_scratchpad.originalStreet || city != g_scratchpad.originalCity || state != g_scratchpad.originalState || zip != g_scratchpad.originalZip) {
			GlideUI.get().clearOutputMessages();
			var ga = new GlideAjax("AddressValidationUtils");
			ga.addParam('sysparm_name', 'validateAddressViaClient');
			ga.addParam('sysparm_street', street);
			ga.addParam('sysparm_city', city);
			ga.addParam('sysparm_state', state);
			ga.addParam('sysparm_zip', zip);
			ga.getXMLAnswer(processXMLAnswer);
			submitForm = false;
		}
	}

	return submitForm;
}

Just like in the main onSubmit function, we first default the submitForm variable to true, and then we grab all of values for the four address components off of the g_form object. The first thing that we check is if there is even any data in any of the four fields. If they are all empty, then there is nothing to validate. If there is some data there, then the next thing that we check is if it is any different than the values that we squirreled away in our onLoad script. If there are no changes, then again there is no need for validation. But if there is data there and it has changed in any way, now we are going to be making that GlideAjax call. Most of that is pretty standard GlideAjax stuff, but we also clear out any error messages on the screen from previous attempts and set the submitForm variable to false to kill the original form submission.

To process the Ajax response, we have yet another function, processXMLAnswer:

function processXMLAnswer(answer) {
	var response = JSON.parse(answer);
	if (response.result == 'invalid') {
		g_form.addErrorMessage('Unable to verify address entered using the US Address validation service');
		g_form.showFieldMsg('street', 'Unverifiable address', 'error');
		g_form.showFieldMsg('city', 'Unverifiable address', 'error');
		g_form.showFieldMsg('state', 'Unverifiable address', 'error');
		g_form.showFieldMsg('zip', 'Unverifiable address', 'error');
	} else {
		if (response.result == 'valid') {
			g_form.setValue('street', response.street);
			g_form.setValue('city', response.city);
			g_form.setValue('state', response.state);
			g_form.setValue('zip', response.zip);
		}
		g_scratchpad.isFormValid = true;
		g_form.submit(actionName);
	}
}

The response comes back in the form of a string, so the first thing that we have to do is convert it back to an object. If the result property of that object is ‘invalid’, then we leave the form unsubmitted and throw a few error messages up on the screen; otherwise, we are going to submit the form a second time. But before we do that, we check to see if the result property is ‘valid’, in which case we overlay the user’s input with the clean address values returned by the service. After that , we set our isFormValid scratchpad property to true and resubmit the form using the saved actionName.

It’s all a little complicated, having to pass through the onSubmit script twice due to the double submit, but it all makes sense if you really think about it. Of course, we won’t really know if it works until we try it, so let’s pull up a user’s profile and give it a shot.

For our first test, let’s see if we can get a validation error. We can use the test address that we have been using up to this point, but let’s change the state from FL to TX and let’s drop the zip code entirely. Also, let’s put everything in lower case, just for fun. OK, let’s see what happens.

Test using invalid address

Having both form-level and field-level error messages is probably a little overkill, but everything seems to have worked. Now, let’s change the state back to FL and see if that is enough to consider it a valid address.

User profile after address validation

Not only did it pass validation, it also corrected our capitalization and added the missing zip code. That’s actually pretty slick. This wasn’t really the point of this series, but I like this address validation feature. I was originally just looking for something that might have a failure to showcase the Event Management stuff, but this turned out to be a pretty handy little addition that could be useful in a number of other places, such as the Building and Location forms.

This now completes the example working feature that has the potential for failure. From this point on, we can focus on what we came here for, which is to log and process Events that originate in ServiceNow. Next time out, we will start in on the logging.

Fun with Outbound REST Events, Part IV

Posted on | by snhackery

“Life is trying things to see if they work.”
— Ray Bradbury

We got a little side-tracked last time, but we are finally ready to tackle the Script Include that will utilize our new Outbound REST Message. I like to tackle things a little bit at a time and test, so we will start out with just the code to invoke the service and format a response to the caller, and then deal with all of the error handling later on in the process. We will also want to call this from a Client Script, but again, let’s not worry about that part just yet and focus on successfully launching the service and interpreting the results.

To start with, we pull up the list of Script Includes and hit the New button to create a new script. We will call our new script AddressValidationUtils and create a function called validateAddress that takes street, city, state, and zip as arguments. The first thing that we will want to do in our new function will be to establish the object that will be returned to the caller.

var response = {result: 'failure', street: street, city: city, state: state, zip: zip};

Our result property will have three possible values: valid, invalid, or failure. We initially default it to failure, mainly because there will be a lot places where we might have to set it to failure, and then alter it to one of the other two values when appropriate. The remaining properties are defaulted to their original value, and if the address is deemed valid, will be updated with the values returned by the service to clean up any inconsistencies. Now let’s add the code to invoke the service.

var rest = new RESTMessage('US Street Address API', 'get');
rest.setStringParameter('authid', gs.getProperty('us.address.service.auth.id'));
rest.setStringParameter('authToken', gs.getProperty('us.address.service.auth.token'));
rest.setStringParameter('street', encodeURIComponent(street));
rest.setStringParameter('city', encodeURIComponent(city));
rest.setStringParameter('state', encodeURIComponent(state));
rest.setStringParameter('zip', encodeURIComponent(zip));
var resp = rest.execute();

This is all pretty straightforward: we establish the rest object by creating a new RESTMessage using the name of the message and the name of the method. Once the object has been established, we use the setStringParameter method to set the value for each of the required variables. Once again, the system does not encode any of the values, so we need to do that for any user input. We don’t need to do that for any of the credentials, as we already know the value of those and we know that there are no characters in either of those that would cause an issue with the URL. Once all of the variable value have been set, we then make the call to the service using the execute method.

Now we need to add code to interpret the response.

var body = resp.getBody();
if (resp.getStatusCode() == 200) {
	var respArray = [];
	try {
		respArray = JSON.parse(body);
	} catch (e) {
		gs.info('Exception: ' + e);
	}
	if (respArray && respArray.length > 0) {
		respObj = respArray[0];
		if (typeof respObj.analysis == 'object') {
			var validity = respObj.analysis.dpv_match_code;
			if (validity == 'Y' || validity == 'S' || validity == 'D') {
				response.result = 'valid';
				response.street = respObj.delivery_line_1;
				response.city = respObj.components.city_name;
				response.state = respObj.components.state_abbreviation;
				response.zip = respObj.components.zipcode;
				if (respObj.components.plus4_code) {
					response.zip += '-' + respObj.components.plus4_code;
				}
			} else {
				response.result = 'invalid';
			}
		} else {
			gs.info('Invalid Response Object: ' + JSON.stringify(respObj));
		}
	} else {
		gs.info('Invalid Response Body: ' + body);
	}
} else {
	gs.info('Invalid HTTP Response Code: ' + resp.getStatusCode());
}

There is a little more code here, but most of it is defensive, just to make sure that we don’t choke on something unexpected. All of the gs.info statements are just placeholders at this point; eventually we will replace those with Event logging, but I’m not ready to deal with that just yet, so we’ll put that off for now and just write to the system log. Assuming all goes well, we will have the response object in hand, and then all we have to do is check to see if it validated or not. If it did, we pass back their version of four address components, and if it didn’t, then we just indicate that is invalid.

Once get through all of that, all that is left is to return the response to the caller. Here’s the whole thing, all put together.

var AddressValidationUtils = Class.create();
AddressValidationUtils.prototype = {

	initialize: function() {
	},

	validateAddress: function(street, city, state, zip) {
		var response = {result: 'failure', street: street, city: city, state: state, zip: zip};

		var rest = new RESTMessage('US Street Address API', 'get');
		rest.setStringParameter('authid', gs.getProperty('us.address.service.auth.id'));
		rest.setStringParameter('authToken', gs.getProperty('us.address.service.auth.token'));
		rest.setStringParameter('street', encodeURIComponent(street));
		rest.setStringParameter('city', encodeURIComponent(city));
		rest.setStringParameter('state', encodeURIComponent(state));
		rest.setStringParameter('zip', encodeURIComponent(zip));
		var resp = rest.execute();
		var body = resp.getBody();
		if (resp.getStatusCode() == 200) {
			var respArray = [];
			try {
				respArray = JSON.parse(body);
			} catch (e) {
				gs.info('Exception: ' + e);
			}
			if (respArray && respArray.length > 0) {
				respObj = respArray[0];
				if (typeof respObj.analysis == 'object') {
					var validity = respObj.analysis.dpv_match_code;
					if (validity == 'Y' || validity == 'S' || validity == 'D') {
						response.result = 'valid';
						response.street = respObj.delivery_line_1;
						response.city = respObj.components.city_name;
						response.state = respObj.components.state_abbreviation;
						response.zip = respObj.components.zipcode;
						if (respObj.components.plus4_code) {
							response.zip += '-' + respObj.components.plus4_code;
						}
					} else {
						response.result = 'invalid';
					}
				} else {
					gs.info('Invalid Response Object: ' + JSON.stringify(respObj));
				}
			} else {
				gs.info('Invalid Response Body: ' + body);
			}
		} else {
			gs.info('Invalid HTTP Response Code: ' + resp.getStatusCode());
		}

		return response;
	},

	type: 'AddressValidationUtils'
};

Now all we need to do is test it out and make sure that it all works. Once again, we can use that same test case that was provided in their testing tool; we just have to pass the values as arguments to the function call. We can write a quick little stand-alone script for that that looks like this:

var avu = new AddressValidationUtils();
gs.info(JSON.stringify(avu.validateAddress('3901 SW 154th Ave', 'Davie', 'FL', '33331'), null, '\t'));

To actually run that, we can bring up the background script runner, paste it in, and click on the Run Script button.

Test script in the background script runner

… which gets us the following result:

{
	"result": "valid",
	"street": "3901 SW 154th Ave",
	"city": "Davie",
	"state": "FL",
	"zip": "33331-2613"
}

Not only did we validate the address, we also picked up the last four digits of the zip code, which was not a part of the original user input. And of course, we just proved that our new Script Include does, in fact, do what we intended for it to do. Chalk up another baby step along the path that we first laid out when we began this journey. So now what?

At this point, we actually have two different ways to go as far as our next step. Given that we have a working script to call when everything is working as it should, we could move on to the Client Script and see if we can now integrate this with the User Profile form, or we could turn our attention to those gs.info placeholders and start replacing those with calls to the function that logs Events. It all has to be done at some point, so it really doesn’t matter other than the fact that we have to choose. But, I guess we don’t have to choose right now. We can figure that out when it’s time to put together the next installment in the series.

Fun with Outbound REST Events, Part III

Posted on | by snhackery

“If it’s a good idea, go ahead and do it. It is much easier to apologize than it is to get permission.”
— Rear Admiral Grace Murray Hopper (“Amazing Grace”)

In the last installment in this series, we set up an Outbound REST Message and announced the intention of devoting this installment to the creation of a Script Include that would utilize the new REST message. In testing out that new Script Include, however, I ended up making a few tweaks to that REST message, so we should probably go over those first, just to bring everyone up to speed on what the REST message looks like today.

The first thing that I ended up changing was the name of the address variable. The web service uses the word street for that parameter, but for some reason I was thinking that ServiceNow called the property on the sys_user table address, so I wanted to adopt their name and not the one used by the web service. As it turns out, however, ServiceNow uses the word street as well, so I went into the REST message definition and changed it back to street in the end point URL and in the variable test value list.

The other thing that I ended up doing was adding a completely new variable to both the URL and the variable list called authToken. When we tested the GET method, we were using the auth-id from the web site’s testing tool, and we expected to get the 401 HTTP Response that we received. However, when testing the new Script Include with my own auth-id, I got the same thing. After rooting around in the documentation on the service’s web site, I discovered that server-side calls to the service require both an auth-id and an auth-token, both of which you receive when you sign up for their services. Once I figured that out, I added an auth-token parameter to the URL, added an authToken variable, and added a new System Property to store the value of the auth-token called us.address.service.auth.token.

After doing all of that, before getting back to the Script Include, I went ahead and tested the modified GET method using the Test link on the method form. That actually got me valid results, but they were a little hard to read.

Web Service Test Results

Fortunately, there is a handy little trick that you can use in ServiceNow to clean up that big, long string quite nicely. If you right click on the Response field label and select Configure Dictionary, you will be taken to the Dictionary Entry form for the Response field. Down at the bottom of the form, you will find the Related Lists. Open up the Attributes tab and then click on the New button at the top of the list. This will take you to the Dictionary Attribute form.

Dictionary Attribute form

Select JSON view from the drop-down list or pop-up selection and then type the word true in the Value field. Save the form, which will take you back to the Dictionary Attribute form, and from there you can use the back arrow on the form to return to the test results. Now you should see a new little icon next to the field label, and if you click on it, a new pop-up screen will appear with the contents of the Response field formatted for much easier reading.

Formatted JSON response

That’s a much better way to look at that data. The JSON View attribute is just one of many field attributes available on the platform. When you have some time, it’s a worthwhile exercise to go back into that selection list and take a look at all of the various choices. It’s very easy to try one or two out, just to see what they do. Some of them, like this one, can be quite useful.

Well, we never got around to actually working on the Script Include, but at least we are all caught up on the changes that I made to get to this point. Since the Script Include discussion will undoubtedly consume an entire post all on its own, I think this will be a good place to stop for now. We will tackle that Script Include in the next installment in this series.

Fun with Outbound REST Events, Part II

Posted on | by snhackery

“The Hacker Way is an approach to building that involves continuous improvement and iteration. Hackers believe that something can always be better, and that nothing is ever complete.”
— Mark Zuckerberg

In the first installment of this series, I just laid out my intentions, and didn’t really produce anything of value. With that out of the way, it’s now time to roll up our sleeves and actually get to work. The first component on our itemized list of artifacts to construct is the Outbound REST Message, so we might as well start there. To begin, pull up the list of Outbound REST Messages and then click on the New button to bring up the form. There are only a couple of required fields, the name and end point, but we will also add the optional description and expand the availability to all scopes on the platform.

New Outbound REST Message

Submitting the form will also generate a default GET HTTP method, which is really all that we will need for our purpose. There are other ways to invoke the address service, including an HTTP POST, but the GET will work, so that is all that we will really need to define. Click on the method to pull up the form so that we can add all of our details.

The HTTP GET method for our new Outbound REST Message

I changed the name of the method to simply get, mainly because we have to call the method by name, and I don’t like to type any more than I have to. The only other thing that we need is the end point. There are a couple of different ways that you can do this, including leaving out the end point all together and inheriting the end point from the main REST Message record, but then you have to define all of the URL parameters individually under the HTTP Request tab. It seems easier to me to just cut and paste the entire URL, query parameters and all, right in the end point field and leave it at that. But, the other way works just as well; your mileage may vary.

I use REST Message Variables for all of the query parameter values, which we can then substitute at execution time. I only do that for the ones that change; last episode we decided that we would always set the match parameter to invalid, so there is no need for a variable for that, as it will always be the same. But for the rest, we will want to use variables for the values, so here is the URL that I ended up with:

https://us-street.api.smartystreets.com/street-address?auth-id=${authid}&street=${address}&city=${city}&state=${state}&zipcode=${zip}&match=invalid

Once we save everything, we can test it out right here on the form, but before we do that, we need to provide a test value to all of the variables that we put in the URL. HTTP Method variables are a Related List, which you will find down at the bottom of the form. We can use the New button on that list to add values for all of our variables. For testing purposes, we can just use the same values that we used when we were using the test tool provided by the provider.

Test values for our HTTP GET Method

If you are paying close attention, you will have noticed that the auth-id value that I used is the same as the one provided in the provider’s test tool. That only works for requests that come from that tool. If you try that from anywhere else, you will get 401 HTTP Response Code. To actually use the service, you have to register to obtain your own ID. There is no cost for up to 250 requests per month, but you still have to register. However, even a 401 response is a valid indication that we have reached the service and the service responded, so let’s click that Test button and see what happens.

First test run (failure)

Well, that didn’t turn out so good. Instead of getting the expected 401, we received an HTTP status value of 0. That basically means that it didn’t even try, so let’s take a closer look at that error message:

Error executing REST request: Invalid uri 'https://us-street.api.smartystreets.com/street-address?auth-id=21102174564513388&street=3901 SW 154th Ave&city=Davie&state=FL&zipcode=33331&match=invalid': I

The complaint is Invalid URL, which is undoubtedly related to the embedded spaces in the street parameter, which are not allowed. Apparently, the test tool does not encode the test values for the variables. This, in my opinion, is a shortcoming of the tool, but it can be easily remedied; I will just replace all of the embedded spaces with %20 and give it another go.

Second test run (success)

There’s that 401 that we were looking for! OK, we have now created our Outbound REST Message, configured our HTTP GET Method, set up values for all of our Method Variables, and successfully tested everything that we have built. We can run another test with our real authentication credentials, just to get a real, valid response, but we have done enough to demonstrate that this configuration actually does reach out and interact with our intended target. That’s good enough for now.

Speaking of your real authentication ID, that’s basically your password to the service, and not really something that you want to have floating around in test scripts or any other components in the system for that matter. The best way to stuff that into a corner somewhere is to create a System Property that can contain the value. This will keep the value out of everything else except for the System Properties table. The easiest way to do that is to go over to the Filter navigator and type sys_properties.do (or you can do what the documentation suggests and type sys_properties.list and then click on the New button). This will bring up the System Property form, where you can define your new property. We’ll call ours us.address.service.auth.id.

System property to hold the auth-id for the service

With that out of the way, we can now use a built-in GlideSystem function to obtain the auth-id in our scripts without having to have the actual value embedded in the script itself. Here is a simple example:

var authId = gs.getProperty('us.address.service.auth.id');

This should give us everything that we need to start in on our Script Include, but that’s a fairly large undertaking, so this seems like a good place to wind things up for now. We’ll start right off with the Script Include next time out.

Fun with Outbound REST Events

Posted on | by snhackery

“A good programmer is someone who always looks both ways before crossing a one-way street.”
— Doug Linder

A while back I mentioned that ServiceNow Event Management can be used within ServiceNow itself. I explained how all of that could work, but I never really came up with a real-world Use Case that would demonstrate the value of starting to wander down that road. I have code to generate Events in a lot of places that never get executed, but it is still there, just in case. One place where unwanted things do tend to happen, though, is when interacting with outside resources such as JDBC imports or external REST calls. Here, you are dependent on some outside database or system being up and available, and that’s not always the case, so you need to build in processes that can gracefully handle that. This is an excellent place for Event Management to step in and capture the details of the issue and log it for some kind of investigation or resolution.

So, I thought I should come up with something that anyone can also play around with on their own, that isn’t tied to some proprietary database or internal web service. I started searching for some kind of public REST API, and I stumbled across the Public APIs web site. There is actually a lot of cool stuff here, but I was looking for something relatively simple, and also something that would seem to have some relation to things that go on inside of ServiceNow. After browsing around a bit, I found the US Street Address API, which looked like something that I could use to validate street addresses in the User Profile. That seemed simple enough and applicable enough to serve my purpose, so that settled that.

There are quite a few parts and pieces to do everything that I want to do, so we will just take them on one at a time. Here is the initial list of the things that I think that I will need to accomplish all that I would like to do:

  • Create an Outbound REST Message using the US Street Address API as the end point,
  • Create a Script Include that will encapsulate all of the functions necessary to make the REST call, evaluate the response, log an Event (if needed), and return the results,
  • Create a Client Script on the sys_user table to call the Script Include if any component of the User’s address changes and display an error message if the address is not valid,
  • Create an Alert Management Rule to produce an Incident whenever the new Event spawns an Alert,
  • Test everything to make sure that it all works under normal circumstances, and then
  • Intentionally mangle the REST end point to produce a failure, thereby testing the creation of the Event, Alert, and Incident.

The first thing to do, then, will be to create the Outbound REST Message, but before we do that, let’s explore the web service just a little bit to understand what we are working with. To do that, there is a handy little API tester here. This will allow us to try a few things out and see what happens. First, let’s just run one of their provided test cases:

https://us-street.api.smartystreets.com/street-address?auth-id=21102174564513388&candidates=10&match=invalid&street=3901%20SW%20154th%20Ave&street2=&city=Davie&state=FL&zipcode=33331

The API is just a simple HTTP GET, and the response is a JSON object:

[
  {
    "input_index": 0,
    "candidate_index": 0,
    "delivery_line_1": "3901 SW 154th Ave",
    "last_line": "Davie FL 33331-2613",
    "delivery_point_barcode": "333312613014",
    "components": {
      "primary_number": "3901",
      "street_predirection": "SW",
      "street_name": "154th",
      "street_suffix": "Ave",
      "city_name": "Davie",
      "default_city_name": "Fort Lauderdale",
      "state_abbreviation": "FL",
      "zipcode": "33331",
      "plus4_code": "2613",
      "delivery_point": "01",
      "delivery_point_check_digit": "4"
    },
    "metadata": {
      "record_type": "S",
      "zip_type": "Standard",
      "county_fips": "12011",
      "county_name": "Broward",
      "carrier_route": "C006",
      "congressional_district": "23",
      "rdi": "Commercial",
      "elot_sequence": "0003",
      "elot_sort": "A",
      "latitude": 26.07009,
      "longitude": -80.35535,
      "precision": "Zip9",
      "time_zone": "Eastern",
      "utc_offset": -5,
      "dst": true
    },
    "analysis": {
      "dpv_match_code": "Y",
      "dpv_footnotes": "AABB",
      "dpv_cmra": "N",
      "dpv_vacant": "N",
      "active": "Y"
    }
  }
]

It looks like the response comes in the form of a JSON Array of JSON Objects, and the JSON Objects contain a number of properties, some of which are JSON Objects themselves. This will be useful information when we attempt to parse out the response in our Script Include. Now we should see what happens if we send over an invalid address, but before we do that, we should take a quick peek at the documentation to better understand what may affect the response. One input parameter in particular, match, controls what happens when you send over a bad address. There are two options;

  • strict  The API will return detailed output only if a valid match is found. Otherwise the API response will be an empty array.
  • invalid  The API will return detailed output for both valid and invalid addresses. To find out if the address is valid, check the dpv_match_code. Values of Y, S, or D indicate a valid address.

The default value in the provided tester is invalid, and that seems to be the appropriate setting for our purposes. Assuming that we will always use that mode, we will need to look for one of the following values in the dpv_match_code property to determine if our address is valid:

Y — Confirmed; entire address is present in the USPS data. To be certain the address is actually deliverable, verify that the dpv_vacant field has a value of N. You may also want to verify that the active field has a value of Y. However, the USPS is often months behind in updating this data point, so use with caution. Some users may prefer not to base any decisions on the active status of an address.
S — Confirmed by ignoring secondary info; the main address is present in the USPS data, but the submitted secondary information (apartment, suite, etc.) was not recognized.
D — Confirmed but missing secondary info; the main address is present in the USPS data, but it is missing secondary information (apartment, suite, etc.).

So, let’s give that a shot and see what happens. Let’s drop the state and zipcode from our original query and give that tester another try.

https://us-street.api.smartystreets.com/street-address?auth-id=21102174564513388&candidates=10&match=invalid&street=3901%20SW%20154th%20Ave&street2=&city=Davie

… which give us this JSON Array in response:

[
  {
    "input_index": 0,
    "candidate_index": 0,
    "delivery_line_1": "3901 SW 154th Ave",
    "last_line": "Davie",
    "components": {
      "primary_number": "3901",
      "street_predirection": "SW",
      "street_name": "154",
      "street_suffix": "Ave",
      "city_name": "Davie"
    },
    "metadata": {
      "precision": "Unknown"
    },
    "analysis": {
      "dpv_footnotes": "A1",
      "active": "Y",
      "footnotes": "C#"
    }
  }
]

This result doesn’t even have a dpv_match_code property, which is actually kind of interesting, but that would still fail a positive test for the values Y, S, or D, so that would make it invalid, which is what we wanted to see.

OK, I think we know enough now about the way things work that we can start building out out list of components. This is probably a good place to wind things up for this episode, as we can start out next time with the construction process, beginning with of our first component, the Outbound REST Message.