Fun with Webhooks, Part V

“Change is easy. Improvement is far more difficult.”
Ferdinand Porsche

Now that we have a Business Rule to launch our Subflow, it’s time to get busy and actually create the Subflow. To create a new Subflow, open up the Flow Designer, click on the New button and select Subflow from drop-down menu. I initially called mine Webhook Poster, but the Business Rule couldn’t find it to launch it for some reason. I finally got it to work when I replaced the space with an underscore, so now the name is Webhook_Poster. I’m not really sure why I had to do that, but it works, so we will leave it at that.

With the Subflow named, the next thing to do is to define all of the inputs. In my mind, there were only two, the current object and the previous object. Unfortunately, the Flow Designer will not recognize any of the properties of those objects unless you explicitly define them, so I had to specify every element of both objects. That seemed like a rather tedious waste of time, but eventually I got through it and now it’s done.

Subflow inputs defined

That takes care of the set-up, so now it’s on to the flow itself. The first thing that we are going to want to do is to gather up all of the registration records that would be triggered by this Incident. Since we gave our users several options for selecting records, we will have to test them all. Essentially, we will need a query filter for each possible Type, which we can then OR together to create one huge query filter. Here is how that looks in practice:

Webhook Registry selection criteria

As a second step, I threw in an IF statement to see if the query returned any records. We are basically done at this point if there are no records, but if we do have records to process, then we will need to build the payload that we will be posting to all of the target URLs. For that job, I needed to create a new Flow Designer Action, so I saved my Subflow and closed the Subflow tab for now.

The payload is basically a generic JSON object containing the relevant information about the the things that have changed about the record since the last time that it was saved. Pulling that all together will basically be a lot of tedious scripting, so that sounds like a job for yet another function in our Script Include. Here is what I came up with:

buildPayload: function(current, previous) {
	var payload = {};

	payload.id = current.number;
	payload.short_description = current.short_description;
	payload.message = '';
	var separator = '';
	if (current.state != previous.state) {
		payload.state = current.state;
		if (previous.state) {
			payload.message += separator + 'State changed from ' + previous.state + ' to ' + current.state + ' on ' + payload.id + '.';
		} else {
			payload.message += separator + 'State set to ' + current.state + ' on ' + payload.id + '.';
		}
		separator = '\n\n';
	}
	if (current.assignment_group != previous.assignment_group) {
		payload.assignment_group = current.assignment_group;
		if (previous.assignment_group) {
			payload.message += separator + 'Assignment Group changed from ' + previous.assignment_group + ' to ' + current.assignment_group + ' on ' + payload.id + '.';
		} else {
			payload.message += separator + 'Assignment Group set to ' + current.assignment_group + ' on ' + payload.id + '.';
		}
		separator = '\n\n';
	}
	if (current.assigned_to != previous.assigned_to) {
		payload.assigned_to = current.assigned_to;
		if (previous.assigned_to) {
			payload.message += separator + 'Assigned To changed from ' + previous.assigned_to + ' to ' + current.assigned_to + ' on ' + payload.id + '.';
		} else {
			payload.message += separator + 'Assigned To set to ' + current.assigned_to + ' on ' + payload.id + '.';
		}
		separator = '\n\n';
	}
	if (current.comments) {
		payload.comments = current.comments;
		payload.message += separator + current.operator + ' has added a new comment to ' + payload.id + ':\n' + current.comments;
		separator = '\n\n';
	}
	if (current.work_notes) {
		payload.work_notes = current.work_notes;
		payload.message += separator + current.operator + ' has added a new work note to ' + payload.id + ':\n' + current.work_notes;
		separator = '\n\n';
	}

	return JSON.stringify(payload, null, '\t');
},

We still need a Flow Designer Action to call this function, but the function does almost all of the heavy lifting and our Action should now be quite simple to put together. Let’s open up the Flow Designer again, click on the New button, and select Action from the drop-down selection list. We’ll call our new Action Build Webhook Payload since that’s what it does, and we will define two inputs, the current and previous objects. Since we won’t be referencing any of the properties of those objects directly, this time we won’t have to invest the time in specifying all of the elements of the objects.

All we will need for our Action is to add a Script step with the same two inputs and the payload as an output. The script itself is just a call to the function that we just added to our Script Include.

(function execute(inputs, outputs) {
	var wru = new WebhookRegistryUtils();
	outputs.payload = wru.buildPayload(inputs.current, inputs.previous);
})(inputs, outputs);

We also need to define the payload as an output of Action itself, and map the Action inputs to the Script step inputs and the Script step output to the Action output. That should complete our new Action.

Build Webhook Payload Action

Now we can go back into our Subflow and select this Action as a third step under our second step conditional that checks to see if there are any records from the query in the first step. For our fourth step, we will add a flow logic step that loops through all of the records from step 1, and inside of that loop, our fifth step will make the POST of the payload. I looked for an out-of-the-box simple HTTP POST Action already included on the platform, but I couldn’t find anything, so that’s another Action that we are going to have to produce ourselves. We already built much of the script for that when we built our testURL function; if we rework that code just a little bit, we can probably find a way to make it work just fine for both purposes.

testURL: function(whrGR) {
	var jsonObj = {message: 'This is a test posting.'};
	return this.postWebhook(whrGR, JSON.stringify(jsonObj, null, 4));
},

postWebhook: function(whrGR, payload) {
	var result = {};

	var request = new sn_ws.RESTMessageV2();
	request.setEndpoint(whrGR.getValue('url'));
	request.setHttpMethod('POST');
	request.setRequestHeader('Content-Type', 'application/json');
	request.setRequestHeader('Accept', 'application/json');
	request.setRequestBody(payload);
	var response = request.execute();
	result.status = response.getStatusCode();
	result.body = response.getBody();
	if (result.body) {
		try {
			result.obj = JSON.parse(result.body);
		} catch (e) {
			result.parse_error = e.toString();
		}
	}
	result.error = response.haveError();
	result.error_code = response.getErrorCode();
	result.error_message = response.getErrorMessage();

	return result;
},

Now the testURL function is just a call to our new postWebhook function that contains most of the code previously contained in the testURL function. The testURL function will pass in our simple test payload, and when we create out new Action, we can call the very same postWebhook function, passing in a real payload. The script for our new Action will then simply be this:

(function execute(inputs, outputs) {
	var wru = new WebhookRegistryUtils();
	var result = wru.postWebhook(inputs.webhook_registry, inputs.payload);
	for (var key in result) {
		outputs[key] = result[key];
	}
})(inputs, outputs);

Stopping right here should give us enough with which to test out the process so far. This isn’t all that we want to do here, because we still want to record both the attempt and the response in our log table. However, since we haven enough built out to just test the POSTing of payloads, I think we should stop and do that. Afterwards, we can circle back and build out the activity logging and any Event logging that we might want to do for any kind of bad responses. But right now, we have enough built out that we can create a few Webhook Registrations and then go update some qualifying Incidents to see what actually happens.

Setting all of that up and verifying the results seems worthy of its own episode, so let’s wrap things up for today and pick that up next time out.

Event Management for ServiceNow, Revisited

“True prevention is not waiting for bad things to happen, it’s preventing things from happening in the first place.”
Don McPherson

Some time ago we built some utility functions to support reporting Events within the ServiceNow Platform. That was before the Flow Designer, though, so that effort did not include any support for that environment. We already have the script to do all of the heavy lifting from our earlier work, so it wouldn’t take much to create a Flow Designer Action that called that script to report an Event that occurred during Flow processing. We can call our new Action Log Event, and set up Action Inputs for all of the usual suspects.

Log Event Action Inputs

For our script step, we will basically set up the same inputs and then source them directly from the primary Action Inputs.

Script step inputs mapped to Action Inputs

Those of you who are paying attention will notice that we defined the additional_input field as a String even though it needs to be an Object when we make the call to our existing script. The assumption here is that the caller will provide a valid JSON String, and then we can turn it into an Object in our script before we make the call. Here is the script to convert the String and then make the call.

(function execute(inputs, outputs) {
	if (inputs.additional_info) {
		try {
			inputs.additional_info = JSON.parse(inputs.additional_info);
		} catch(e) {
			//
		}
	}
	var seu = new ServerEventUtil();
	seu.logEvent(inputs.source, inputs.resource, inputs.metric_name, inputs.severity, inputs.description, inputs.additional_info);
})(inputs, outputs);

There are no outputs from this process, so this is the entire Action. Once we Save and Publish it, it will be available from the Action selection list, and then we can add Log Event steps anywhere in our Flows and Subflows where we want to report an Event. That was fairly quick, easy, and relatively painless. For those of you would like to try it out on your own, here is an Update Set.

Fun with Webhooks, Part IV

“Try not to become a man of success, but rather try to become a man of value.”
Albert Einstein

Last time, we came to a fork in the road, not knowing whether it would be better to jump into the My Webhooks Service Portal widget, or to start working out the details of the process that actually POSTs the Webhooks. At this point, I am thinking that the My Webhooks widget will end up being a fairly simple clone of my old My Delegates widget, so that may not be all that interesting. POSTing the Webhooks, on the other hand, does sound like it might be rather challenging, so let’s start there.

When I first considered attempting this, my main question was whether it would be better to handle this operation in a Business Rule or by building something out in the Flow Designer. After doing a little experimenting with each, I later came to realize that the best alternative involved a little bit of both. In a Business Rule, you have access to both the current and previous values of every field in the record; that information is not available in the Flow Designer. In fact, it’s not available in a Business Rule, either, if you attempt to run it async. But if you don’t run it async, then you are holding everything up on the browser while you wait for everything to get posted. That’s not very good, either. What I ultimately decided to do was to start out with a Business Rule running before the update, gather up all of the needed current and previous values, and then pass them to a Subflow, which runs async in the background.

My first attempt was to just pass the current and previous objects straight into my Subflow, but that failed miserably. Apparently, when you pass a GlideRecord into a Subflow, you are just passing a reference, not the entire record, and then when the Subflow starts up, it uses that reference to fetch the data from the database. That’s not the data that I wanted, though. I want the data as it was before the database was updated. I had to take a different route with that part of it, but the basic rule remained the same.

Business Rule to POST webhooks

The rule is attached to the Incident table, runs on both Insert and Update, and is triggered whenever there is a change to any of the following fields: State, Assignment Group, Assigned to, Work notes, or Additional comments.

To get around my GlideRecord issue, I wound up creating two objects, one for current and one for previous, which I populated with data that I pulled out of the original GlideRecords. When I passed those two objects to my Subflow, everything was there so that I could do what I wanted to do. Populating the objects turned out to be quite a bit of code, so rather than put that all in my Business Rule, I created a function in my Script Include called buildSubflowInputs and I passed it current and previous as arguments. That simplified the Business Rule script quite a bit.

(function executeRule(current, previous) {
	var wru = new WebhookRegistryUtils();
	sn_fd.FlowAPI.startSubflow('Webhook_Poster', wru.buildSubflowInputs(current, previous));
})(current, previous);

In the Script Include, things got a lot more complicated. Since I was going to be turning both the current and previous GlideRecords into objects, I thought it would be helpful to create a function that did that, which I could call once for each. That function would pull out the values for the Sys ID, Number, State, Caller, Assignment group, and Assigned to fields and return an object populated with those values.

getIncidentValues: function(incidentGR) {
	var values = {};

	values.sys_id = incidentGR.getValue('sys_id');
	values.number = incidentGR.getDisplayValue('number');
	if (!incidentGR.short_description.nil()) {
		values.short_description = incidentGR.getDisplayValue('short_description');
	}
	if (!incidentGR.state.nil()) {
		values.state = incidentGR.getDisplayValue('state');
	}
	if (!incidentGR.caller_id.nil()) {
		values.caller = incidentGR.getDisplayValue('caller_id');
		values.caller_id = incidentGR.getValue('caller_id');
	}
	if (!incidentGR.assignment_group.nil()) {
		values.assignment_group = incidentGR.getDisplayValue('assignment_group');
		values.assignment_group_id = incidentGR.getValue('assignment_group');
	}
	if (!incidentGR.assigned_to.nil()) {
		values.assigned_to = incidentGR.getDisplayValue('assigned_to');
		values.assigned_to_id = incidentGR.getValue('assigned_to');
	}

	return values;
},

Since my Business Rule could be fired by either an Update or an Insert, I had to allow for the possibility that there was no previous GlideRecord. I could arbitrarily call the above function for current, but I needed to check to make sure that there actually was a previous before making that call. I also wanted to add some additional data to the current object, including the name of the person making the changes. The field sys_updated_by contains the username of that person, so to get the actual name I had to use that in a query of the sys_user table to access that data.

buildSubflowInputs: function(currentGR, previousGR) {
	var inputs = {};

	inputs.current = this.getIncidentValues(currentGR);
	if (currentGR.isNewRecord()) {
		inputs.previous = {};
	} else {
		inputs.previous = this.getIncidentValues(previousGR);
	}
	inputs.current.operator = currentGR.getDisplayValue('sys_updated_by');
	var userGR = new GlideRecord('sys_user');
	if (userGR.get('user_name', inputs.current.operator)) {
		inputs.current.operator = userGR.getDisplayValue('name');
	}

	return inputs;
},

One other thing that I wanted to track was any new comments or work_notes, and that turned out to be the most the most complicated code of all. If you try the normal getValue or getDisplayValue methods on any of these Journal Fields, you end up with all of the comments ever made. I just wanted the last one. I had to do a little searching around to find it, but there is a function called getJournalEntry that you can use on these fields, and if you pass it a 1 as an argument, it will return the most recent entry. However, it is not just the entry; it is also a collection of metadata about the entry, which I did not want either. To get rid of that, I had to find the first line feed and then pick off all of the text that followed that. Putting all of that together, you end up with the following code:

if (!currentGR.comments.nil()) {
	var c = currentGR.comments.getJournalEntry(1);
	inputs.current.comments = c.substring(c.indexOf('\n') + 1);
}
if (!currentGR.work_notes.nil()) {
	var w = currentGR.work_notes.getJournalEntry(1);
	inputs.current.work_notes = w.substring(w.indexOf('\n') + 1);
}

That’s pretty much all that there is to the new Business Rule and the needed additions to the Script Include. Now we need to start tackling the Subflow that will actually take these current and previous objects and do something with them. That’s a bit much to jump into at this point, so we’ll wrap things up here for now and save all of that for our next time out.

Fun with Webhooks

“Good ideas are common – what’s uncommon are people who’ll work hard enough to bring them about.”
Ashleigh Brilliant

There is quite a bit of Webhook stuff in various IntegrationHub spokes, but it all seems to be oriented towards consuming incoming events from different external event publishers. I want to actually be the publisher, and send out information based on some preferences selected by the consumer. That may be hidden somewhere in the Now Platform already, but I can’t seem to find it, so I have decided that I would try to develop a Scoped Application to do just that. This may very well be recreating something that already exists in the platform today, but it sounds like a fun exercise, so I am going to give it the old college try.

As always, I will attempt to start out with the most basic of offerings, and then incrementally expand to add more and better features. My approach is to treat this feature as somewhat analogous to a Watch List, in that you sign up to follow certain events, but instead of sending a notification to a User when the event occurs, the result will be that the information is posted to a specified URL. This can apply to any number of things, but to start off, I am going to focus on some very specific changes to one particular table (Incident), and then later expand from there.

To make this work, there will need to be some kind of Webhook Registry where a consumer would sign up to receive these posts. When registering your webhook, you would enter the URL to which you want the data posted along with the specifics of what type or types of events you would like to have included. I’m thinking about linking them directly to an owner, and having some kind of My Webhooks Portal Page where you could manage your existing registrations and add new ones. When adding a new one, you should be able to enter and test your URL, and for our first iteration, that may be the only choice that you get. Later on, we will want to add the ability to choose what you want to follow, which specific updates should trigger a new post, and even what you would like to have included in the payload. But we will also want to start out as simple as possible, so the initial registry may turn out to be quite barren as far as input fields go.

Once registered, there will need to be some process to actually send out the posts as requested in the registration. This could be a Business Rule on the source table, or maybe something created in the Flow Designer. Either way, the process should scan the registry for any condition matches and then send out a post for each match. Each post and response should be logged in some kind of Webhook Activity Log, and any bad HTTP Response Codes should be reported to Event Management. A robust service would attempt to repost any failures up to a certain limit before giving up completely, but all of that can be delegated to some Alert Management Rule at some later time. Again, we will want to start out simple, so our initial focus will just be on making that initial post attempt. Everything else can be pushed off until later on in the process.

Those would seem to be the two major functions: registering the webhook and sending out the posts. We may want some other things at some point, such as the ability to review the logs or to manually repost or to clone an existing registration, but for now, just those two things should get the ball rolling. We may also want to set up a sample receiver for testing purposes, but in practice, the receivers would be other products and outside the scope of this development exercise. There is actually an existing service out on the Internet called Webhook.site that might turn out to be just what I need in order to do a little testing. We should check that out when we get to that point.

For our parts list, then, I can see the need for the following artifacts:

  • A table to hold the webhook registrations,
  • A my_webhooks portal widget to list all webhooks owned by the user,
  • A webhook portal widget for editing a single webhook registration,
  • A Business Rule or Flow to send out the posts,
  • A log table to record the posts and response, and possibly
  • A Script Include to contain some common functions.

Of course, before we create any of that, we will have to create the Scoped Application itself, so that should be where we start next time when we initiate the actual construction phase of this effort.

Flow Variables vs. Scratchpad Variables

“An essential part of being a humble programmer is realizing that whenever there’s a problem with the code you’ve written, it’s always your fault.”
Jeff Atwood

Now that you can get yourself a copy of the new Quebec version of ServiceNow, you can start playing around with all of the new features that will be coming out with this release. One of those new features is Flow Variables, which look like they might serve to replace the Scratchpad that I had built earlier for pretty much the exact same purpose. Since I have used that custom feature in a variety of different places after I first put it together, the trick will be to see if I can just do a one-for-one swap without having to re-engineer things using an entirely different philosophy.

I like building parts, but I also like to get rid my home-made parts whenever I find that there is a ServiceNow component that can serve the same purpose. If I can figure out how to make a Flow Variable do all of the things that I have been able to do with a Scratchpad Variable, then I will be the first in line to throw my home-made scratchpad into the dust bin. But before we get too far ahead of ourselves, let’s try a few things and see what we can actually do.

According to the ServiceNow Developer Blog, you can set the value of a Flow Variable inside of a loop and use it to control the number of times that you go through the loop.

Subtracting 1 from an Integer Flow Variable

Let’s create a sample flow, define our first Integer Flow Variable and then set up a loop that we can escape based on the value of the variable. Open up the Flow Designer, click on the New button, select Flow from the drop-down, and then name the new Flow Sample Flow. From the vertical ellipses menu in the upper right-hand corner, we can select Flow Variables and define a new variable called index of type Integer. For our first step, we can set the value of our new variable to 0, and then we can add a loop that will run until the variable value is greater than 5. After that, all we will have to add is another step inside the loop to increment our index variable.

Simple variable test flow

All of that went relatively smoothly for me until I tried to add the +1 to the Set Flow Variables step inside of the loop. Once I dragged the index pill into the value box, it would not accept any more input. I tried putting the +1 in first, and then dragging the pill in front of it second, but that just replaced the +1. I could never get the pill+1 value like the sample image in the developers blog entry. Obviously, I was doing something wrong, but I did not know exactly what. Nothing that I tried seemed to work, though, so eventually I decided to give that up and just script it.

Scripted variable increment

It was a fairly simple script that just added one to the existing value.

return fd_data.flow_var.index + 1;

Unfortunately, that resulted in a text concatenation rather than an increment of the value. 0 + 1 became 01 and 01 + 1 became 011 and so forth. Even though I had defined my variable as an Integer, it was getting treated as if it were text. No problem, though … I will just convert it to an Integer before I attempt to increment the value.

return parseInt(fd_data.flow_var.index) + 1;

That’s better. Now 0 + 1 becomes 1 and 1 + 1 becomes 2 just the way that I thought that it should. But … things are still not working the way that they should. In my test runs, the new value was not always present when I reentered the loop. 0 + 1 became 1, but then the next time through, the variable value was still 0. That’s not right! I kept erroring out by looping through the code more than the maximum 10,000 times until I finally changed my bail-out condition to be any value greater than zero. Even then, it ended up looping 456 times before it dropped out.

Completed test run results

Clearly, something is not quite right here. It is possible that I just got a hold of an early release that still has a few kinks left to be worked out, but if you’re a true believer in The First Rule of Programming, then you know that I’ve just done something wrong somewhere along the line and need to figure out what that is. Obviously, you shouldn’t have to run through the loop over 400 times just bump the count above zero.

Well, back to the drawing board, as they say. Maybe I will have to resort to reading the instructions. Nah … I’ll just keep trying stuff until I get something to work. In the meantime, it looks like I won’t be flushing the old home-grown scratchpad down the drain just yet. At least not until I figure out how to make this thing work. Too bad, though. I had such high hopes when I saw that this was coming out.

Note: Eventually, I was able to figure out what I was doing wrong.

Fun with Outbound REST Events, Part X

“No one has a problem with the first mile of a journey. Even an infant could do fine for a while. But it isn’t the start that matters. It’s the finish line.”
Julien Smith

After our last installment in this series, our Events now spawn Incidents that are pretty much just what we would like to see. The only remaining challenge at this point is to create a meaningful Description field value. Although we have set things up to produce this Description in a Script Include function, I should point out right here at the outset that everything that we are about to do in our script could also be accomplished in the Flow Designer itself. In fact, it probably should be done using the Flow Designer if we are to fully embrace the whole no-code future towards which we all seem to be herded. I’m still an old coder at heart, though, so it seems easier to me to scratch out another quick function than it does to build out all of those action steps using input forms. Still, it would probably be a worthwhile exercise to replace this script with a subflow one day; today is just not that day. Today we code!

Although we are passing the Alert to our function as an argument, much of the data we need is actually in the Event that spawned the Alert, so the first thing that we are going to want to do is go out and get that guy. That’s pretty basic GlideRecord stuff.

// get initial Event
var eventGR = new GlideRecord('em_event');
eventGR.addQuery('alert', alertGR);
eventGR.orderBy('sys_created_on');
eventGR.query();
eventGR.next();

Since it is possible that there could be more than one Event associated with our Alert, we include an orderBy directive to ensure that we get the very first Event out of the bunch. Once we have our Event in hand, we will have access to the additional_info JSON string, which we will want to convert to a Javascript object so that we can reference all of the various component parts.

// get addition info from Event
var additionalInfo = {};
try {
	additionalInfo = JSON.parse(eventGR.getValue('additional_info'));
} catch (e) {
	gs.info('Unable to parse additonal_info from Event ' + eventGR.number);
}

We also have access to the Event resource, which in our case is a User’s user_name. We can use that to get the sys_user record for that user, much in the same way that we retrieved the Event record.

// get affected User record
var userGR = new GlideRecord('sys_user');
userGR.get('user_name', alertGR.getValue('resource'));

This assumes, of course, that the only place that we using our address validation capability is on the User Profile page. If we ever expand its use to other places — say on the Building or Location form — then we would need to have some way to know whether the resource was a User or a Location or a Building or some other entity with an address to validate. Based on that information, we might be retrieving a Building record or a Location record instead of a User record. For now, though, we can safely assume that the resource is a User.

Now that we have gathered up all of the data that we need, we can start building out our Description. To begin, let’s start out with something that will be universal to all of our Incidents, regardless of the problem being reported.

// format description
var alertDesc = alertGR.getDisplayValue('description');
var section = '\n========================================\n';
var subsection = '\n----------------------------------------\n';
desc += additionalInfo.user.name;
desc += ' attempted to update the address on the user profile for user ';
desc += userGR.getDisplayValue('name');
desc += ', but was unable to verify the address using the US Address Validation service due to the following error:\n\n';
desc += alertDesc;
desc += '\n\nIncident Details:';
desc += section;

Beyond this point, we are going to want to be a little more specific based on what actually happened to trigger the Event. We can do that by introducing some conditional code based on the known values found in the Alert’s description field.

if (alertDesc.startsWith('The response code')) {
	// bad response code language will go here
} else if (alertDesc.startsWith('The response object')) {
	// bad response object language will go here
} else if (alertDesc.startsWith('The response content')) {
	// bad response content language will go here
} else {
	// we should never get here, but just in case ...
}

Since most of the Events that we have triggered up to this point have been of the bad response code variety, let’s do those first.

if (!additionalInfo.response.code) {
	desc += 'No response was received from the service, which could be an indication that the service is unavailable or unreachable. Check the status or the external service as well as the status of your connection to the Internet.';
} else if (additionalInfo.response.code == 401) {
	desc += 'A Response Code of 401 indicates an authentication error of some kind. Verify that your account credentials are correct and that your account is in good standing with the service provider.';
} else {
	desc += 'The service returned a Response Code of ';
	desc +=  additionalInfo.response.code;
	desc += '. Additional information on the meaning of this response may be found in the Response Body. Also, you can check with the service provider for further clarification on the appropriate handling of this response.';
	desc += '\n\nDetailed information on the ';
	desc += additionalInfo.response.code;
	desc += ' Response Code can be found on the web at https://httpstatuses.com/';
	desc += additionalInfo.response.code;
}

This gives us specialized language for no response code at all, and a response code of 401. Everything else is handled in a more generalized section that covers any other bad response code. As more knowledge of the potential response codes becomes available through experience with the service, more specialized language can be added that can be more specific to other known response codes.

Now let’s take a look at what we can do for bad response objects.

desc += 'The service returned a valid Response Code and a parsable response, but the response did not contain certain expected elements necessary to determine the validity of the address. Review the response received and check with the service provider to see if anything has changed with the API specifications.';

That one is about as simple as you can get; everyone gets the same language. For the bad response content issues, things are a little bit more sophisticated. Everyone still gets the same language, but there is a possibility for an exception with this group, so we include code to handle that as well.

desc += 'The service returned a valid Response Code, but the response was either empty or ill-formatted. Review the response received and check with the service provider to see if the service is experiencing problems, or if anything has changed with the API specifications.';
if (additionalInfo.exception) {
	desc += '\n\nException Details:';
	desc += subsection;
	desc += '   Exception: ';
	desc += additionalInfo.exception;
	desc += '\n   Stack Trace:\n';
	desc += additionalInfo.stackTrace;
}

Once we complete all of the conditional logic, we wrap things up with some more universal code that applies to everyone. This just serves to include the user’s input and the service’s response at the end of the body of the Description field for reference.

desc += '\n\nAddress Details:';
desc += subsection;
desc += '   Street: ';
desc += additionalInfo.input.street;
desc += '\n   City: ';
desc += additionalInfo.input.city;
desc += '\n   State: ';
desc += additionalInfo.input.state;
desc += '\n   Zip Code: ';
desc += additionalInfo.input.zip;
desc += '\n\nResponse Details:';
desc += subsection;
desc += '   Response Code: ';
desc += additionalInfo.response.code;
desc += '\n   Response Body: ';
desc += additionalInfo.response.content;
if (additionalInfo.response.object) {
	desc += '\n   Response Object: ';
	desc += JSON.stringify(additionalInfo.response.object, null, '\t');
}

There is still more helpful information that we could add, such as links to the service provider’s documentation, or in the case of the 401 error, the names of the system properties that contain the credentials, but this is good enough for a sample. Let’s just save what we have and then trigger another Event and see what comes out the other side.

Incident with updated description from the new Script Include function

Well, that’s much, much better than the description that the original Create Incident flow was producing. It’s not perfect, but I think it does provide the person receiving the Incident enough details about both what happened and what might be done about it that they can get to work on the ticket right away without a whole lot of research. Obviously, it can be fine-tuned over time, but this is a good foundation upon which to build for this particular use case.

That pretty much wraps up all that I had hoped to accomplish with this series. It took us 10 installments to get here, but much of that was due to the fact that we had to build out our own address validation infrastructure before we could use it to demonstrate applying Event Management tools and techniques to internal ServiceNow features and functions. For those of you who like to play along at home, I have bundled what I hope are all of the relevant parts and pieces into an Update Set that you are welcome to pull down and import into your own environment.

Fun with Outbound REST Events, Part IX

“What we hope ever to do with ease, we must first learn to do with diligence.”
Samuel Johnson

Last time, we were able to have our Alert produce an Incident, but it wasn’t exactly the Incident that we wanted. Today, we are going to fix that. Since we don’t want to alter the out-of-the-box Create Incident subflow that we are currently using to create our Incidents, we will want to make a copy of the subflow so that we can customize it for our own purposes. To do that, pull up the Create Incident subflow in the Flow Designer, click on the vertical ellipses in the upper right-hand corner, and select Copy subflow to create a new copy of the subflow.

Copy the Create Incident subflow

A pop-up dialog box will appear where you can enter the name of your new subflow, which we will call Create Address Issue Incident.

Enter the name of the new subflow

After entering the name, click on the Copy button to create your new subflow from the original. This should open up your new subflow for editing.

Your new Create Address Issue Incident subflow

Now that we have our own copy, we can make whatever modifications that we would like to make without disturbing the original. All of the changes that we will want to make are in the Create Task step, so let’s open that up and see what we can do to produce Incidents that include the detail that we would like to provide to the technician working the ticket. Let’s get rid of the Description value entirely, as that’s not the description that we want. That very same text is repeated in the Additional Comments field, anyway. The rest of the values that are there seem to be OK for now, but let’s add a few more using the +Add Field Value button at the bottom of the field list.

Let’s set the State to Assigned, the Assignment Group to ITSM Engineering, the Category to Software, and the Subcategory to Internal Application. Some of that may not be exactly right, but this is just an example of the kinds of things that you can do. For the new Description value, which is going to be conditional depending on the nature of the issue that triggered this Incident, let’s use an inline script. That can be done by clicking on the little f(x) button to the right of the field value.

Create Task step expanded

At this point, we don’t necessarily need to build the entire script, but we will want to stub it out enough to keep things functional for testing. Since the script might get a little complex, I like to push all of the logic out to a Script Include and then limit the code in the subflow to just a call to a function in the Script Include. That keeps the clutter out of the subflow itself, and also allows us to refine the output of the process by just editing the Script Include without having to publish a new version of the subflow. We already have a Script Include devoted to the address verification process, so let’s just add a simple function to that existing artifact so that we have something that we can call in the subflow.

formatIncidentDescription: function(alertGR) {
	return 'Test description for ' + alertGR.number;
},

There isn’t much to this at this point, but there is enough here to verify that we are receiving the Alert, which we will want as a reference when we start building out the actual description that we want. Getting back to our subflow, the script to invoke this new function will look like this:

var avu = new AddressValidationUtils();
return avu.formatIncidentDescription(fd_data.subflow_inputs.ah_alertgr);

Figuring out that fd_data.subflow_inputs.ah_alertgr was the correct syntax for referencing the Alert was not all that intuitive. I have worked with the Flow Designer long enough now to know about the fd_data object, but I couldn’t find much in the way of documentation on identifying the names of the various properties of that object. Fortunately, I did come across some documentation on the type-ahead feature, which finally led me to the information for which I had been searching. Typing a single dot after the fd_data brings up a nice pick list of choices, and another one after selecting the choice does the same for that object as well.

fd_data properties pick list

With our script in place for the Description value, all that is left is to Save and Publish the new subflow and we are done with the Forms Designer. To use our new subflow, we will need to go back into our Alert Management Rule, open up the Action section, and replace the Create Incident subflow with our modified copy.

Modifying our rule to use the newly created subflow

At this point, we should be good to test again and see what kind of Incident gets generated now. Just remember to select a different person so that our Event is not assigned to an existing Alert that has already been processed. We want to be sure to create a brand new Alert to activate our modified rule. Once we force a new Event, we can pull it up and take a look at it to see what values have been set in the Event record.

Newly generated Event record

From the Event, we can navigate to the Alert, and from the Alert we can then navigate to the Incident. Let’s check out the Incident.

Incident generated from our new subflow

This is an improvement over our initial effort, as the ticket has now been properly categorized and routed to an Assignment Group for resolution. We still need a much better Description, but the presence of the Alert ID in the current Description value verifies that we are indeed passing the Alert record to our stubbed-out function, which we can now use to produce a more detailed and informative description value. Scripting that out for all of the various possibilities will be a bit of an effort, through, so let’s just make that the focus of our next installment.

Fun with Outbound REST Events, Part VIII

“Computer science education cannot make anybody an expert programmer any more than studying brushes and pigment can make somebody an expert painter.”
Eric S. Raymond

Now that we have completed our address verification feature, added Event logging, and tested the creation of those Events, it’s time to actually do something with the Events when they come out. Well, actually, we won’t be doing anything with the Events themselves; we will be doing something with the Alerts that come out as a result of the Events. To process those Alerts, we will need to create a new Alert Management Rule.

To create a new rule, pull up the list of Alert Management Rules and click on the New button at the top of the list. The form is divided into three sections and the first section is the Alert info section. In that section, you will want to enter the name of the Alert, a description of the Alert, and you will want to set the Multiple alert rules field to Stop search for additional rules. This will prevent additional rules from evaluating or taking action on your Alert, as your rule should handle everything that needs to be done and no further rules should be applied.

Alert Info section of Alert Management Rule form

After completing the Alert info section, use the progress bar at the top of the form to move on to the next section of the form, the Alert Filter section. This is where you specify which Alerts you would like to process with this rule. In our case, we want to handle anything that comes out of our Script Include, which is identified in the Alert in the Source field. That makes our filter quite simple, as we only want to process Alerts where the Source is AddressValidationUtils:

Alert Filter section of Alert Management Rule form

After completing the Alert Filter section, use the progress bar at the top of the form once again to move on to the next section of the form, the Action section. This is where you specify what action should be taken whenever a new Alert is created that meets your filter criteria. There are quite a lot of things that you can do here, but we want to create an Incident. Fortunately for us, there is a built-in, out-of-the-box Subflow already developed that does exactly that. To select this Subflow, called Create Incident, double-click on the Insert new row … line to open up a new row and then double-click on the Subflow column of the newly inserted row to select the Create Incident subflow from the selection list.

Action section of Alert Management Rule form

After completing all three sections of the form, click on the Submit button to save your new rule. Once the rule has been created and saved, it is now active in the system, and the next time any Events are logged by our Script Include, the rule will be triggered. Let’s go ahead and do that now, just to see what happens.

We want to trigger an Event, so we can mangle our credentials again and then update someone’s address, which should do the trick. We will want to select a different person for this test, just to make sure that we trigger a brand new Alert, and not just have our Event associated with an existing Alert from any of our previous testing. Once we submit the address change, we can find our Event, and from there, navigate to the Alert.

Alert resulting from address service Event

Unlike all of our previous Alerts, this new one now has an Incident number in the Task field. This is the Incident that was generated from the execution of our new Alert Management Rule. Click on the info icon to the right of the Task field and then click on the Open Record button in the resulting pop-up window to bring up the Incident.

Incident generated from the Alert Management Rule

This may not be exactly the Incident that we would like to see, but we are taking things one step at a time, and we just produced an Incident from our Alert, which is a huge step in and of itself. Now let’s take a look at this Incident and see where we might make things a little better.

One of the first things that you might notice is that there is no Assignment Group, so it has not been routed to anyone for resolution. We should know to whom this Incident should be assigned, which might be the ServiceNow support team or at a minimum, the Service Desk, so we should populate that field right from the start. If we do that, then we should also set the State to Assigned rather than New.

We should also use a more appropriate Category, but the biggest improvement that we could make would be in the Description. When you generate an Incident via Event Management, you want to do as much as you can to explain both what happened and what can be done about it to the technician who will end up having to work the Incident. The Description that we are generating right now really doesn’t do that at all. We can do much better.

All of these fields are populated in the Create Incident flow that we assigned to our rule. Since that’s an out-of-the-box generic flow, we don’t really want to modify it, but we can make our own copy of it and then make whatever changes we want to make to our copy. That sounds like a bit of a project, though, so let’s make that exercise the subject or our next installment.

Flow Designer Counter

“If you have built castles in the air, your work need not be lost; that is where they should be. Now put the foundations under them.”
Henry David Thoreau

Since I first threw together my Flow Designer Array Iterator, I have had a number of occasions to put it to good use, but recently I had a need for an iterating index, but I did not have a String Array to use as the basis for the iteration. I thought about just creating one, just to have something to pass in to the Action, but then it occurred to me that it might be useful to have some kind of simple index action that wasn’t dependent on the presence of a String Array. Basically, it could be very similar to my Array Iterator, but without the array.

I thought about making a new Script Include just for these new Actions, but I decided to just add them to my existing SNHStringArrayUtils Script Include, mainly because I wanted to copy the existing functions from there, anyway, and so it was easier to just drop the copy right there into the same script. To start with, I copied the createIterator function and then hacked it up to create a new createCounter function.

createCounter: function(scratchpadId, counterId) {
	var response = {};

	var snhspu = new SNHScratchpadUtils();
	response = snhspu.setScratchpadProperty(scratchpadId, counterId, 0);
	if (response.success) {
		response.message = 'Counter ' + counterId + ' successfully created';
	}

	return response;
}

For the most part, that was just a matter of removing all of the code related to the String Array, which we are not using here, and then setting the value of our new counter to zero. Once that was done, I copied the existing iteratorNext function and hacked that up to create the counterNext function.

counterNext: function(scratchpadId, counterId) {
	var response = {};

	var snhspu = new SNHScratchpadUtils();
	response = snhspu.getScratchpadProperty(scratchpadId, counterId);
	if (response.success) {
		var counter = parseInt(response.property_value);
		var previous = counter;
		counter++;
		var current = counter;
		counter++;
		var next = counter;
		response = snhspu.setScratchpadProperty(scratchpadId, counterId, current);
		if (response.success) {
			response.previous_value = previous;
			response.current_value = current;
			response.next_value = next;
			response.message = 'The current value of the counter is ' + current;
		}
	}

	return response;
}

That was pretty much it for the scripting changes. With that all put to bed, I popped over to the Flow Designer and basically did the same thing with my array Actions that I did with the array functions, copy them and then modify them for my new purpose. I used my Create Array Iterator Action as the starting point for my new Create Counter Action, and then used my Array Iterator Next Action as the basis of my new Increment Counter action. Once again, I spent more time removing things than I did adding things, and it all went relatively quickly. The only thing to do now was to do test it all out.

As with the array Actions, you have to first have a Scratchpad on which to store the values, so I ran a quick test on my Create Scratchpad Action to get a fresh Scratchpad ID. With that in hand, I pulled up my new Create Counter Action and hit the Test button, entered my Scratchpad ID and Counter ID and ran the test.

Testing the Create Counter Action

That all seemed to work out OK, so I pulled up my new Increment Counter Action and gave that one a whirl as well. In fact, I ran the test a few times, just to run up the number.

Increment Counter Action test results

Well, everything seems to work. Obviously, there is a lot more testing to do in order to check out all of the error handling built into the processes, but the main purpose of the exercise seems to be satisfied, so that’s always a good thing. If you want to play around with all of the various parts and pieces, here’s an Update Set that should contain everything that you would need.

Note: With the introduction of Flow Variables, this component is no longer necessary.

Flow Designer Array Iterator

“It is by logic that we prove, but by intuition that we discover.”
Henri Poincaré

One of the reasons that I built my little Flow Designer Scratchpad was to keep track of an index while I looped through items in an Array. After doing a few of those, I decided it would be even nicer if I had some kind of iterator Action that would do the work of incrementing the index and returning the Array element at the current index. I already had the scratchpad to store all of the data necessary to support an iterator, so it seemed as if I could write some kind of Action that would take a Scratchpad ID, an Iterator ID, and an Array as input and use that information to set up the ability to iterate through the Array provided. As usual, I decided to put the bulk of the code in a Script Include to keep the actual code in the Action down to the absolute minimum. Here is the SNHStringArrayUtils that I came up with:

var SNHStringArrayUtils = Class.create();
SNHStringArrayUtils.prototype = {
	initialize: function() {
	},

	createIterator: function(scratchpadId, interatorId, stringArray) {
		var response = {};

		var snhspu = new SNHScratchpadUtils();
		response = snhspu.setScratchpadProperty(scratchpadId, interatorId, interatorId);
		if (response.success) {
			if (Array.isArray(stringArray)) {
				var iterator = {};
				iterator.index = 0;
				iterator.array = stringArray;
				response = snhspu.setScratchpadProperty(scratchpadId, interatorId, JSON.stringify(iterator));
				if (response.success) {
					response.message = 'Array Interator ' + interatorId + ' successfully created';
				}
			} else {
				response.success = false;
				response.message = 'String Array parameter is missing or invalid';
			}
		}

		return response;
	},

	iteratorNext: function(scratchpadId, interatorId) {
		var response = {};

		var snhspu = new SNHScratchpadUtils();
		response = snhspu.getScratchpadProperty(scratchpadId, interatorId);
		if (response.success) {
			var iterator = {};
			try {
				iterator = JSON.parse(response.property_value);
			} catch (e) {
				response.success = false;
				response.message = 'Unable to parse JSON string containing iterator details';
			}
			if (response.success) {
				if (iterator.index >= 0 && iterator.index < iterator.array.length) {
					response.current_value = iterator.array[iterator.index];
					response.current_index = iterator.index;
					iterator.index++;
					response.has_next = (iterator.index < iterator.array.length);
					response.message = 'The current value at index ' + response.current_index + ' is ' + response.current_value;
					snhspu.setScratchpadProperty(scratchpadId, interatorId, JSON.stringify(iterator));
				} else {
					response.success = false;
					response.message = 'Current index value out of range';
				}
			}
		}

		return response;
	},

	type: 'SNHStringArrayUtils'
};

Basically, there are two methods, one for each of the two Flow Designer Actions that I intend to build. The first one is createIterator, which is used to initialize a new iterator, and the second is iteratorNext, which will support the Action that you will invoke inside of your loop to get the next item in the Array. Both utilize an existing scratchpad, so you will need to create that prior to invoking these Actions, and both require an Iterator ID, which is just a unique key to be used in storing the iterator data in the scratchpad. The createIterator action would be called once outside of the loop, and then the iteratorNext function would be called inside of the loop, usually right at the top to pull out the next value in the array.

The iterator itself is just a two-property object containing the array and the current value of the index. This is converted to a JSON string and stuffed into the scratchpad using the passed Iterator ID as the key. When creating the iterator, we set the index value to zero, and in the next Action, after using the index to get the current element, we increment it and update the scratchpad. Now that we have the basic code to support the two Actions, we need to go into the Flow Designer and create the Actions.

The Create Array Iterator Action seems like the logical place to start. That will will need three Inputs defined.

The Create Array Iterator Action Inputs

… and it will need two Outputs defined, a success indicator and an optional error message detailing any failure to perform its intended function.

The Create Array Iterator Action Outputs

In between the Inputs and Outputs will be a simple Script step, where we will produce the Outputs by passing the Inputs to our associated Script Include function.

var snhsau = new SNHStringArrayUtils();
var result = snhsau.createIterator(inputs.scratchpad_id, inputs.iterator_id, inputs.string_array);
for (var key in result) {
	outputs[key] = result[key];
}

That’s pretty much all there is to that. We can test it using the Test button up at the top of the Action Editor, but first we will need a Scratchpad. We can take care of that real quick by hitting the Test button on the Create Scratchpad Action and then grabbing the Scratchpad ID from the Outputs. With our Scratchpad ID in hand, we can now test our Create Array Iterator Action.

Testing the Create Array Iterator Action

So far so good. now we just need to do the same thing for the Array Iterator Next Action, and we’ll be all set.

The Array Iterator Next Action

This is pretty much a rinse and repeat kind of thing, with fewer Inputs, but more Outputs. When it comes time to test, we can use the Scratchpad ID and Iterator ID from our last test, and then run it through a few times to see the different results at different stages of the process. Rather that go through all of that here, I will just bundle everything up into an Update Set, and you can pull it down and play with it on your own.

Note: With the introduction of Flow Variables, this component is no longer necessary.