REST API – Page 4

Fun with Outbound REST Events, Part II

Posted on December 9, 2020 | by snhackery

“The Hacker Way is an approach to building that involves continuous improvement and iteration. Hackers believe that something can always be better, and that nothing is ever complete.”
— Mark Zuckerberg

In the first installment of this series, I just laid out my intentions, and didn’t really produce anything of value. With that out of the way, it’s now time to roll up our sleeves and actually get to work. The first component on our itemized list of artifacts to construct is the Outbound REST Message, so we might as well start there. To begin, pull up the list of Outbound REST Messages and then click on the New button to bring up the form. There are only a couple of required fields, the name and end point, but we will also add the optional description and expand the availability to all scopes on the platform.

Submitting the form will also generate a default GET HTTP method, which is really all that we will need for our purpose. There are other ways to invoke the address service, including an HTTP POST, but the GET will work, so that is all that we will really need to define. Click on the method to pull up the form so that we can add all of our details.

*The HTTP GET method for our new Outbound REST Message*

I changed the name of the method to simply get, mainly because we have to call the method by name, and I don’t like to type any more than I have to. The only other thing that we need is the end point. There are a couple of different ways that you can do this, including leaving out the end point all together and inheriting the end point from the main REST Message record, but then you have to define all of the URL parameters individually under the HTTP Request tab. It seems easier to me to just cut and paste the entire URL, query parameters and all, right in the end point field and leave it at that. But, the other way works just as well; your mileage may vary.

I use REST Message Variables for all of the query parameter values, which we can then substitute at execution time. I only do that for the ones that change; last episode we decided that we would always set the match parameter to invalid, so there is no need for a variable for that, as it will always be the same. But for the rest, we will want to use variables for the values, so here is the URL that I ended up with:

https://us-street.api.smartystreets.com/street-address?auth-id=${authid}&street=${address}&city=${city}&state=${state}&zipcode=${zip}&match=invalid

Once we save everything, we can test it out right here on the form, but before we do that, we need to provide a test value to all of the variables that we put in the URL. HTTP Method variables are a Related List, which you will find down at the bottom of the form. We can use the New button on that list to add values for all of our variables. For testing purposes, we can just use the same values that we used when we were using the test tool provided by the provider.

If you are paying close attention, you will have noticed that the auth-id value that I used is the same as the one provided in the provider’s test tool. That only works for requests that come from that tool. If you try that from anywhere else, you will get 401 HTTP Response Code. To actually use the service, you have to register to obtain your own ID. There is no cost for up to 250 requests per month, but you still have to register. However, even a 401 response is a valid indication that we have reached the service and the service responded, so let’s click that Test button and see what happens.

Well, that didn’t turn out so good. Instead of getting the expected 401, we received an HTTP status value of 0. That basically means that it didn’t even try, so let’s take a closer look at that error message:

Error executing REST request: Invalid uri 'https://us-street.api.smartystreets.com/street-address?auth-id=21102174564513388&street=3901 SW 154th Ave&city=Davie&state=FL&zipcode=33331&match=invalid': I

The complaint is Invalid URL, which is undoubtedly related to the embedded spaces in the street parameter, which are not allowed. Apparently, the test tool does not encode the test values for the variables. This, in my opinion, is a shortcoming of the tool, but it can be easily remedied; I will just replace all of the embedded spaces with %20 and give it another go.

There’s that 401 that we were looking for! OK, we have now created our Outbound REST Message, configured our HTTP GET Method, set up values for all of our Method Variables, and successfully tested everything that we have built. We can run another test with our real authentication credentials, just to get a real, valid response, but we have done enough to demonstrate that this configuration actually does reach out and interact with our intended target. That’s good enough for now.

Speaking of your real authentication ID, that’s basically your password to the service, and not really something that you want to have floating around in test scripts or any other components in the system for that matter. The best way to stuff that into a corner somewhere is to create a System Property that can contain the value. This will keep the value out of everything else except for the System Properties table. The easiest way to do that is to go over to the Filter navigator and type sys_properties.do (or you can do what the documentation suggests and type sys_properties.list and then click on the New button). This will bring up the System Property form, where you can define your new property. We’ll call ours us.address.service.auth.id.

*System property to hold the auth-id for the service*

With that out of the way, we can now use a built-in GlideSystem function to obtain the auth-id in our scripts without having to have the actual value embedded in the script itself. Here is a simple example:

var authId = gs.getProperty('us.address.service.auth.id');

This should give us everything that we need to start in on our Script Include, but that’s a fairly large undertaking, so this seems like a good place to wind things up for now. We’ll start right off with the Script Include next time out.

Fun with Outbound REST Events

Posted on December 8, 2020 | by snhackery

“A good programmer is someone who always looks both ways before crossing a one-way street.”
— Doug Linder

A while back I mentioned that ServiceNow Event Management can be used within ServiceNow itself. I explained how all of that could work, but I never really came up with a real-world Use Case that would demonstrate the value of starting to wander down that road. I have code to generate Events in a lot of places that never get executed, but it is still there, just in case. One place where unwanted things do tend to happen, though, is when interacting with outside resources such as JDBC imports or external REST calls. Here, you are dependent on some outside database or system being up and available, and that’s not always the case, so you need to build in processes that can gracefully handle that. This is an excellent place for Event Management to step in and capture the details of the issue and log it for some kind of investigation or resolution.

So, I thought I should come up with something that anyone can also play around with on their own, that isn’t tied to some proprietary database or internal web service. I started searching for some kind of public REST API, and I stumbled across the Public APIs web site. There is actually a lot of cool stuff here, but I was looking for something relatively simple, and also something that would seem to have some relation to things that go on inside of ServiceNow. After browsing around a bit, I found the US Street Address API, which looked like something that I could use to validate street addresses in the User Profile. That seemed simple enough and applicable enough to serve my purpose, so that settled that.

There are quite a few parts and pieces to do everything that I want to do, so we will just take them on one at a time. Here is the initial list of the things that I think that I will need to accomplish all that I would like to do:

Create an Outbound REST Message using the US Street Address API as the end point,
Create a Script Include that will encapsulate all of the functions necessary to make the REST call, evaluate the response, log an Event (if needed), and return the results,
Create a Client Script on the sys_user table to call the Script Include if any component of the User’s address changes and display an error message if the address is not valid,
Create an Alert Management Rule to produce an Incident whenever the new Event spawns an Alert,
Test everything to make sure that it all works under normal circumstances, and then
Intentionally mangle the REST end point to produce a failure, thereby testing the creation of the Event, Alert, and Incident.

The first thing to do, then, will be to create the Outbound REST Message, but before we do that, let’s explore the web service just a little bit to understand what we are working with. To do that, there is a handy little API tester here. This will allow us to try a few things out and see what happens. First, let’s just run one of their provided test cases:

https://us-street.api.smartystreets.com/street-address?auth-id=21102174564513388&candidates=10&match=invalid&street=3901%20SW%20154th%20Ave&street2=&city=Davie&state=FL&zipcode=33331

The API is just a simple HTTP GET, and the response is a JSON object:

[
  {
    "input_index": 0,
    "candidate_index": 0,
    "delivery_line_1": "3901 SW 154th Ave",
    "last_line": "Davie FL 33331-2613",
    "delivery_point_barcode": "333312613014",
    "components": {
      "primary_number": "3901",
      "street_predirection": "SW",
      "street_name": "154th",
      "street_suffix": "Ave",
      "city_name": "Davie",
      "default_city_name": "Fort Lauderdale",
      "state_abbreviation": "FL",
      "zipcode": "33331",
      "plus4_code": "2613",
      "delivery_point": "01",
      "delivery_point_check_digit": "4"
    },
    "metadata": {
      "record_type": "S",
      "zip_type": "Standard",
      "county_fips": "12011",
      "county_name": "Broward",
      "carrier_route": "C006",
      "congressional_district": "23",
      "rdi": "Commercial",
      "elot_sequence": "0003",
      "elot_sort": "A",
      "latitude": 26.07009,
      "longitude": -80.35535,
      "precision": "Zip9",
      "time_zone": "Eastern",
      "utc_offset": -5,
      "dst": true
    },
    "analysis": {
      "dpv_match_code": "Y",
      "dpv_footnotes": "AABB",
      "dpv_cmra": "N",
      "dpv_vacant": "N",
      "active": "Y"
    }
  }
]

It looks like the response comes in the form of a JSON Array of JSON Objects, and the JSON Objects contain a number of properties, some of which are JSON Objects themselves. This will be useful information when we attempt to parse out the response in our Script Include. Now we should see what happens if we send over an invalid address, but before we do that, we should take a quick peek at the documentation to better understand what may affect the response. One input parameter in particular, match, controls what happens when you send over a bad address. There are two options;

strict The API will return detailed output only if a valid match is found. Otherwise the API response will be an empty array.
invalid The API will return detailed output for both valid and invalid addresses. To find out if the address is valid, check the dpv_match_code. Values of Y, S, or D indicate a valid address.

The default value in the provided tester is invalid, and that seems to be the appropriate setting for our purposes. Assuming that we will always use that mode, we will need to look for one of the following values in the dpv_match_code property to determine if our address is valid:

Y — Confirmed; entire address is present in the USPS data. To be certain the address is actually deliverable, verify that the dpv_vacant field has a value of N. You may also want to verify that the active field has a value of Y. However, the USPS is often months behind in updating this data point, so use with caution. Some users may prefer not to base any decisions on the active status of an address.
S — Confirmed by ignoring secondary info; the main address is present in the USPS data, but the submitted secondary information (apartment, suite, etc.) was not recognized.
D — Confirmed but missing secondary info; the main address is present in the USPS data, but it is missing secondary information (apartment, suite, etc.).

So, let’s give that a shot and see what happens. Let’s drop the state and zipcode from our original query and give that tester another try.

https://us-street.api.smartystreets.com/street-address?auth-id=21102174564513388&candidates=10&match=invalid&street=3901%20SW%20154th%20Ave&street2=&city=Davie

… which give us this JSON Array in response:

[
  {
    "input_index": 0,
    "candidate_index": 0,
    "delivery_line_1": "3901 SW 154th Ave",
    "last_line": "Davie",
    "components": {
      "primary_number": "3901",
      "street_predirection": "SW",
      "street_name": "154",
      "street_suffix": "Ave",
      "city_name": "Davie"
    },
    "metadata": {
      "precision": "Unknown"
    },
    "analysis": {
      "dpv_footnotes": "A1",
      "active": "Y",
      "footnotes": "C#"
    }
  }
]

This result doesn’t even have a dpv_match_code property, which is actually kind of interesting, but that would still fail a positive test for the values Y, S, or D, so that would make it invalid, which is what we wanted to see.

OK, I think we know enough now about the way things work that we can start building out out list of components. This is probably a good place to wind things up for this episode, as we can start out next time with the construction process, beginning with of our first component, the Outbound REST Message.

Using the GlideExcelParser on a Fetched Remote File

Posted on June 30, 2020 | by snhackery

“Do not go where the path may lead, go instead where there is no path and leave a trail.”
— Ralph Waldo Emerson

When we hacked the REST Message API to fetch a remote file, all we did with the file was to attach it to an existing Incident. But there are times when what you are really after is the contents of the file, not the file itself. When that file happens to be an Excel spreadsheet, the ServiceNow platform comes shipped with a handy tool called the GlideExcelParser that will allow you to dig into that file and pull out the data. By combining this capability with our earlier work on fetching remote files, you can reach out to a URL, pull in a spreadsheet from another site, and then parse the spreadsheet to extract the data that you need.

One example of where this might be useful would be in importing data records where one or more fields on the record is a URL of an associated spreadsheet containing relevant information such a list of items to be ordered or a list of people for which you would like to place an order. Another example might be a mail-in ticket where the details are not included in the body of the email or attached to the email as a file, but accessible via a URL link found in the message body. Basically, this applies to any scenario where you have the URL of a file, and just want the data contained in the file and not the actual file.

To demo this capability, we just need to find a spreadsheet hosted on the Internet that we can use for an example. Something like this:

https://www.servicenow.com/content/dam/servicenow-assets/public/en-us/doc-type/other-document/csc/servicenow-platform-support-calculator-template.xlsx

This document has three sheets, but for our purposes we can just focus on one of the areas of one of the sheets, just to show how things work. Here is the second sheet, labeled Input:

*Example spreadsheet to use for demonstration purposes*

For our little demo, we will focus on the grey block on rows 20 through 31 so that we can pull out some data and then use it for some other purpose such as creating an Incident. Once we pull out the data, we will no longer need the actual file, so we can delete the attachment that we created to tidy things up when we are done. Since this is just a demo, we can do the whole thing in a Fix Script that we can discard later once our little proof of concept has been completed.

The first thing that we will need to do is to go get our file. For that, we can leverage our earlier Script Include, but we will need to attach it to something while we work with it. For the purposes of this demonstration, let’s just attach it to the Fix Script itself.

var url = 'https://snhackery.com/update_sets/servicenow-platform-support-calculator-template.xlsx';
var table = 'sys_script_fix';
var sys_id = '0bbdf83e2f21d0104425fcecf699b660';
var fileFetcher = new FileFetchUtils();
var attachment = fileFetcher.fetchFileFromUrlAndAttach(table, sys_id, url);

That was simple enough. Now let’s parse the file.

var stream = new GlideSysAttachment().getContentStream(attachment);
var parser = new sn_impex.GlideExcelParser();
parser.setSheetName('Input');
var row = [];
if (parser.parse(stream)) {
	while (parser.next()) {
		row.push(parser.getRow());
	}
} else {
	gs.info("Error: Unable to parse uploaded file");
}

OK, this one is a little more complicated. First, we create a content stream from the file attachment. Then we create a new parser. Then we tell the parser that we want it to parse the second sheet, the one labeled Input. Then we create an empty array to store the rows, and loop through the parsed stream adding rows to our array one at a time until we run out of rows. Now we have an array of row that that we can use for whatever purpose that we want.

At this point, we can delete the attachment, since we have no further use for it, but let’s save that for a little later and first see what we can do with our row data. To start with, let’s just make sure that we have some data, and if we do, let’s call a function in which we can encapsulate all of the code to utilize the data.

if (row.length) {
	gs.info('Incident created: ' + createIncident(row));
} else {
	gs.info("Error: No data rows in parsed uploaded file");
}

Now it’s time to deal with the data. The parsed rows are values keyed by the column heading. Where there is no column heading, then the key is the Excel column letter. In our little example, we are interested in the data in the second and third columns, and in the first row, there is a value in the second column (Data Input Form), but not in the third. This means that second column values will be obtained with thisRow[‘Data Input Form’] and third column values will be obtained with thisRow[‘C’] (since C is Excel’s name for the third column).

So to rebuild that little grey block of data, we will need the second column value for row #19, the second and third column values for rows 21 through 26, and then the second and third column values for row 28. Now, that’s assuming that no one adds or removes a line or a column from the spreadsheet when filling out the template, which could be dealt with using a lot more defensive coding strategy, but that’s a little beyond the scope of this little exercise here. For our purposes, we are just going to assume that things are going to be right where they are supposed to be. So here is our function.

function createIncident(row) {
	var full_description = 'The following data was extracted from the fetched spreadsheet:\n\n';
	full_description += row[19]['Data Input Form'] + '\n\n';
	for (var i=21; i<27; i++) {
		full_description += '\t' + row[i]['Data Input Form'] + ':\t' + row[i]['C'] + '\n';
	}
	full_description += '\n' + row[28]['Data Input Form'] + '\t' + row[28]['C'];
	var incidentGR = new GlideRecord('incident');
	incidentGR.caller_id = gs.getUserID();
	incidentGR.short_description = 'Excel parsing example';
	incidentGR.description = full_description;
	incidentGR.assignment_group.setDisplayValue('Service Desk');
	incidentGR.insert();
	return incidentGR.getDisplayValue('number');
}

Now that the Incident has been created, we just need to delete the attachment record to clean things up. That’s just a few more lines of code.

var attachmentGR = new GlideRecord('sys_attachment');
attachmentGR.get(attachment);
attachmentGR.deleteRecord();

That’s it. All that is left is to save the script and then click on that little Run Fix Script button to see what happens.

*** Script: Incident created: INC0010019

Let’s go check out our new Incident and see how things came out.

*Incident generated from data extracted from Excel spreadsheet*

Well, it looks pretty close, but we missed a line of data. The System Administrators line didn’t make it over. Looks like our start index should have been 20 and not 21. Oh well, you get the idea. I would go back and fix it, but this is just a little demo to show how things could work together, and I think we have accomplished that. If you want to play around with this, you will be using your own spreadsheet anyway, so you’ll be doing something completely different. But if you want to fix your own copy of this one, please go right ahead!

Hacking the REST Message API to Fetch a Remote File

Posted on June 25, 2020 | by snhackery

“The way to get started is to quit talking and begin doing.”
— Walt Disney

The other day I needed to fetch a file that was posted on another web site via its URL and process it. I have processed ServiceNow file attachments before, but I have never attempted to go out to the Internet and pull in a file using an HTTP GET of the URL. There are several ways to do that in Javascript such as XMLHttpRequest or fetch, but none of those seemed to work in server-side code in ServiceNow. But you can open up a URL using the ServiceNow RESTMessageV2 API, so I thought that maybe I would give that a shot. How hard could it be?

I decided to encapsulate everything into a Script Include, mainly so that if I ever needed to do this again, I could call the same function in some other context. My thought was to pass in the URL of the file that I wanted to fetch along with the table name and sys_id of the record to which I wanted the file to be attached, and then have the function fetch the file, attach it, and send back the attachment. Something like this:

var FileFetchUtils = Class.create();
FileFetchUtils.prototype = {
	initialize: function() {
	},

	fetchFileFromUrlAndAttach: function(table, sys_id, url) {
		...
	},

	type: 'FileFetchUtils'
};

That was idea, anyway. Let’s say that I wanted to attach this file, available on the ServiceNow web site, to some Incident:

https://www.servicenow.com/content/dam/servicenow-assets/public/en-us/doc-type/success/playbook/implementation.pdf

The code to do that would look something like this:

var fileFetcher = new FileFetchUtils();
fileFetcher.fetchFileFromUrlAndAttach('incident', incSysId, 'https://www.servicenow.com/content/dam/servicenow-assets/public/en-us/doc-type/success/playbook/implementation.pdf');

All we need to do now is come up with the code needed to do all of the work of fetching the file, attaching it to the specified record, and returning the attachment. To begin, we will need to extract the name of the file from the URL. Assuming that the file name is the last component of the path on the URL, we can do that by splitting the path into its component parts and grabbing the last part.

var parts = url.split('/');
var fileName = parts[parts.length-1];

Next, we will need to create and configure the request object.

var request  = new sn_ws.RESTMessageV2();
request.setHttpMethod('get');
request.setEndpoint(url);

The next thing to do would be to execute the request, but before we do that, we can take advantage of a nice built-in feature that will really simplify this whole operation. There is an available function of the RESTMessageV2 API that allows you to declare your intent to turn the retrieved file into an attachment, which will then handle all of the details of doing that on your behalf when the request is executed. You just need to invoke the function before you execute the request.

request.saveResponseBodyAsAttachment(table, sys_id, fileName);        
response = request.execute();

Although that really makes things super simple, it’s still a good idea to check the HTTP Response Code, just to make sure all went well. If not, it’s a good practice to relay that to the user.

if (response.getStatusCode() == '200') {
	...
} else {
	returnValue = 'Error: Invalid HTTP Response: ' + response.getStatusCode();
}

Now, assuming that things actually did go well and our new attachments was created, we still want to send that back to the calling script as a response to this function. The RESTMessageV2 API saveResponseBodyAsAttachment function does not return the attachment that is created, so we will have to use the table and sys_id to hunt it down. And if we cannot find it for any reason, we will want to report that as well.

var attachmentGR = new GlideRecord('sys_attachment');
attachmentGR.addQuery('table_name', table);
attachmentGR.addQuery('table_sys_id', sys_id);
attachmentGR.orderByDesc('sys_created_on');
attachmentGR.query();
if (attachmentGR.next()) {
	returnValue = attachmentGR.getUniqueValue();
} else {
	returnValue = 'Error: Unable to fetch attachment';
}

That should now be everything that we need to fetch the file, attach it, and send back the attachment. Putting it all together, the entire Script Include looks like this:

var FileFetchUtils = Class.create();
FileFetchUtils.prototype = {
	initialize: function() {
		this.REST_MESSAGE = '19bb0cde2fedd4101a75ad2ef699b6da';
	},

	fetchFileFromUrlAndAttach: function(table, sys_id, url) {
		var returnVlaue = '';
		var parts = url.split('/');
		var fileName = parts[parts.length-1];
		var request  = new sn_ws.RESTMessageV2();
		request.setHttpMethod('get');
		request.setEndpoint(url);
		request.saveResponseBodyAsAttachment(table, sys_id, fileName);        
		response = request.execute();
		if (response.getStatusCode() == '200') {
			var attachmentGR = new GlideRecord('sys_attachment');
			attachmentGR.addQuery('table_name', table);
			attachmentGR.addQuery('table_sys_id', sys_id);
			attachmentGR.orderByDesc('sys_created_on');
			attachmentGR.query();
			if (attachmentGR.next()) {
				returnValue = attachmentGR.getUniqueValue();
			} else {
				returnValue = 'Error: Unable to fetch attachment';
			}
		} else {
			returnValue = 'Error: Invalid HTTP Response: ' + response.getStatusCode();
		}
		return returnValue;
	},

	type: 'FileFetchUtils'
};

Now all we need to do is find an Incident to use for testing and use the background scripts feature to give it a try. First, we’ll need to pull up an Incident and then use the Copy sys_id option of the hamburger drop-down menu to snag the sys_id of the Incident.

Now we can pop over to the background script processor and enter our code, using the sys_id that we pulled from the selected Incident.

After running the script, we can return to the Incident to verify that the file from the specified URL is now attached to the Incident.

*The selected test Incident with the remote file attached*

Just to make sure that all went well, you can click on the download link to pull down the attachment and look it over, verifying that it came across complete and intact. That pretty much demonstrates that it all works as we had intended. If you would like a copy to play around with on your own, you can pick it up here.

Hacking the Scripted REST API, Part II

Posted on June 9, 2020 | by snhackery | 2 Comments

“If you find a path with no obstacles, it probably doesn’t lead anywhere.”
— Frank A. Clark

Last time, we built a Scripted REST API with a single Scripted REST Resource. To finish things up, we just need to add the actual script to the resource. To process our legacy support form, create an Incident, and respond to the user, the script will need to handle the following operations:

Obtain the input form field values from the request,
Format the data for use in creating the Incident,
Locate the user based on the email address, if one exists,
Create the Incident, and
Return the response page.

Obtain the input form field values from the request

This turned out to be much simpler than I first made it out to be. I knew that the out-of-the-box Scripted REST API was set up to handle JSON, both coming in and going out, and could also support XML, but I never saw anything regarding URL encoded form fields, so I assumed I would have to parse and unencode the request body myself. The problem that I was having, though, was getting the actual request body. I tried request.body, request.body.data, request.body.dataString, and even request.body.dataStream — nothing produced anything but null or errors. Then I read somewhere that that the Scripted REST API treats form fields as if they were URL query string parameters, and lumps them all together in a single map: request.queryParams. Once I learned that little tidbit of useful information, the rest was easy.

// get form data from POST
var formData = request.queryParams;
var name = formData.name[0];
var email = formData.email[0];
var short_description = formData.title[0];
var description = formData.description[0];

It did take me a bit to figure out that the values returned from the map are arrays and not strings, but once that became clear in my testing, I just added an index to the form field name and everything worked beautifully.

Format the data for use in creating the Incident

This was just a matter of formatting a few of the incoming data values with labels so that anything that did not have a field of its own on the Incident could be included in the Incident description.

// format the data
var full_description = 'The following issue was reported via the Legacy Support Form:\n\n';
full_description += 'Name: ' + name + '\n';
full_description += 'Email: ' + email + '\n\n';
full_description += 'Details:\n' + description;

Locate the user based on the email address

This doesn’t really have much to do with hacking the Scripted REST API, but I threw it in just as an example of the kind of thing that you can do once you have some data with which to work. In this case, we are simply using the email address that was entered on the form to search the User table to see if we have a user with that email. If we do, then we can use as the Caller on the Incident.

// see if we have a user on file with that email address
var contact = null;
var userGR = new GlideRecord('sys_user');
if (userGR.get('email', email)) {
	contact = userGR.getUniqueValue();
}

Create the Incident

This part is pretty vanilla as well.

// create incident
var incidentGR = new GlideRecord('incident');
if (contact) {
	incidentGR.caller_id = contact;
} else {
	incidentGR.caller_id.setDisplayValue('Guest');
}
incidentGR.contact_type = 'self-service';
incidentGR.short_description = short_description;
incidentGR.description = full_description;
incidentGR.assignment_group.setDisplayValue('Service Desk');
incidentGR.insert();
var incidentId = incidentGR.getDisplayValue('number');

The last line grabs the Incident number from the inserted Incident so that we can send that back to the user on the response page.

Return the response page

Now that we have complete all of the work, the last thing left to do is to respond to the user. Again, since we are not using the native JSON or XML formats, we are going to have to do some of the work a little differently than the standard Scripted REST API. Here is the working code:

// send response page
response.setContentType('text/html');
response.setStatus(200);
var writer = response.getStreamWriter();
writer.writeString(getResponsePageHTML(incidentId));

The first thing that you have to know is that you must set the content type and status before you get the stream writer from the response. If you don’t do that first, then things will not work. And even though you clicked the override checkbox and specified text/html as the format in the Scripted REST API definition, you still have to set it here as well. But once you do all of that, and do it in the right sequence, it all works great.

The response string itself is just the text of a standard HTML page. I encapsulated my sample page into a function so that it could be easily replaced with something else without disturbing any of the rest of the code. The sample version that I put together looks like this:

function getResponsePageHTML(incidentId) {
	var html = '';

	html += '<html>';
	html += ' <head>';
	html += '  <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.5.0/css/bootstrap.min.css" integrity="sha384-9aIt2nRpC12Uk9gS9baDl411NQApFmC26EwAOH8WgZl5MYYxFfc+NcPb1dKGj7Sk" crossorigin="anonymous">';
	html += '  <script src="https://stackpath.bootstrapcdn.com/bootstrap/4.5.0/js/bootstrap.min.js" integrity="sha384-OgVRvuATP1z7JjHLkuOU7Xw704+h835Lr+6QL9UvYjZE3Ipu6Tp75j7Bh/kR0JKI" crossorigin="anonymous"></script>';
	html += ' </head>';
	html += ' <body>';
	html += '  <div style="padding: 25px;">';
	html += '   <h2>Thank you for your input</h2>';
	html += '   <p>We appreciate your business and value your feedback. Your satisfaction is of the utmost concern to us.</p>';
	html += '   <p>Your issue has been documented and one of our marginally competent technicians will get back to you within a few months to explain to you what you have been doing wrong.</p>';
	html += '   <p>Your incident number is ' + incidentId + '.</p>';
	html += '  </div>';
	html += ' </body>';
	html += '</html>';

	return html;
}

That’s it for the coding in ServiceNow. To see how it all works, we just need to point our old form to our new form processor after which we can pull up the modified form in a browser and give it a try. To repoint the legacy form, pull it up in a text editor, find the form tag, and enter the path to our Scripted REST API in the action attribute:

<form action="https://<instance>.service-now.com/api/<scope>/legacy/support" method="post">

With that little bit of business handled, we can pull up the form, fill it out, click on the submit button, and if all goes well, be rewarded for our efforts with our HTML response, including the number of the newly created Incident:

*HTML response from the Scripted REST API form processor*

Just to be sure, let’s pop over to our instance and check out the Incident that we should have created.

*Incident created from submitting the legacy form*

Well, that’s all there is to that. We built a Scripted REST API that accepted standard form fields as input and responded with a standard HTML web page, and then pointed an old form at it to produce an Incident and a thank you page. All in all, not a bad little perversion of the REST API feature of the tool!

Hacking the Scripted REST API to process a form

Posted on June 8, 2020 | by snhackery

“Try something new each day. After all, we’re given life to find it out. It doesn’t last forever.”
— Ruth Gordon

One of our older systems includes a form through which you could report issues, and when you filled out the form and submitted it, it would send an e-mail to the support team. Pretty cool stuff back in the day, but the procedure bypasses the Service Desk and hides the activity from our support statistics because no one ever opens up an Incident. They just quietly resolve the issue and move on without a trace. The people who like to have visibility into those kinds of activities are not really too keen on these little side deals that allow certain groups to fly below the radar. So the question arose as to whether or not we could keep the form, with which everyone was comfortable and familiar, but have it create an Incident rather than send an e-mail.

Well, the first thing that came to mind was to just send the existing e-mail to the production instance and then set up an inbound mail processor to turn the e-mail into an Incident. The problem with that approach, though, was the the Incident was created off-line, and by that time, you had no way to inform the user that the Incident was successfully created or to give them the ID or some kind of handle to pull it up and check on the progress. What would really be nice would be to be able to simply POST the form to ServiceNow and have it respond back with an HTML thank you page. ServiceNow is not really set up to be a third-party site forms processor, though, so that really didn’t seem to be a feasible concept.

But, then again …

ServiceNow does have the Scripted REST API, but that is built for Web Services, not user interaction. Still, with a little creative tweaking maybe we could actually fool it into taking a form POST and responding with an HTML page. That would actually be interesting. And as it turns out, it wasn’t all that hard to do.

To make our example relatively simple and easy to follow, let’s just build a nice clean HTML page that contains nothing but our example legacy form:

*Simple stand-alone input form for demonstration purposes*

This clears out all of the window dressing, headers, footers, and other clutter and just gets down to the form itself. None of that other stuff has any relevance to what we are trying to do here, so we just want a simple clean slate without all of the distractions. Here is the entire HTML code for the page:

<html>
 <head>
  <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.5.0/css/bootstrap.min.css" integrity="sha384-9aIt2nRpC12Uk9gS9baDl411NQApFmC26EwAOH8WgZl5MYYxFfc+NcPb1dKGj7Sk" crossorigin="anonymous">
  <script src="https://stackpath.bootstrapcdn.com/bootstrap/4.5.0/js/bootstrap.min.js" integrity="sha384-OgVRvuATP1z7JjHLkuOU7Xw704+h835Lr+6QL9UvYjZE3Ipu6Tp75j7Bh/kR0JKI" crossorigin="anonymous"></script>
 </head>
 <body>
  <form action="/some/legacy/form/handler" method="post">
   <div style="padding: 25px;">
    <h2>Legacy Support Form</h2>
    <div style="margin-top: 10px;">
     <label for="name">Please enter your name:</label>
    </div>
    <div>
     <input name="name" size="64"/>
    <div style="margin-top: 10px;">
     <label for="email">Please enter your email address:</label>
    </div>
    <div>
     <input name="email" size="64"/>
    </div>
    <div style="margin-top: 10px;">
     <label for="title">Please enter a brief statement describing the issue:</label>
    </div>
    <div>
     <input name="title" size="64"/>
    </div>
    <div style="margin-top: 10px;">
     <label for="description">Please describe the problem in detail:</label>
    </div>
    <div>
     <textarea name="description" cols="62" rows="5"></textarea>
    </div>
    <div style="margin-top: 20px;">
     <input type="submit" value="Submit Problem Report"/>
    </div>
   </div>
  <form>
 </body>
</html>

The idea here is to now take the existing form handler, as specified in the action attribute of the form tag, and replace it with a URL for a ServiceNow “web service” that we will create using the Scripted REST API tools. That is the only change that we want to make on this existing page. Everything else should look and behave exactly as it did before; we are just pointing the form to a new target for processing the user input. So let’s build that target.

To begin, select the Scripted REST APIs option on the left-hand sidebar menu, which will bring up the list of all of the existing Scripted REST APIs. From there, click on the New button, which will take you to a blank form on which you can start entering the details about your new Scripted REST API.

*Initial Scripted REST API data entry form*

Enter the name of your new Scripted REST API along with the API ID, which will become a component in the eventual URL that will take you to your new web service. Once you have saved these initial values, you will be returned to the list, and your newly created API will appear on the list. Select it, and you will be taken to an expanded form where you can now enter the rest of the information.

*Full Scripted REST API data entry form*

Here is one of the secrets to our little hack: you need to check both of the override boxes so that you can change the MIME type for the data that will be flowing in both directions. For the data that is coming in, you will want to enter x-www-form-urlencoded. These are are the form fields coming in from the input form. For the data going out, you will want to enter text/html. This is the response page that will go back to the browser and be rendered by the browser for display to the user. Form fields come into our new service and HTML comes back out.

Once you have saved your overrides, you can create your Resource. A Scripted REST API can have one or more Resources, and they appear on the form down at the bottom as a Related List. Open up the Resources tab and click on the New button to create your Resource. This brings up the Resource form.

On this form, you want to enter the name of your Resource, the HTTP method (POST), and the relative path, which is another component of the URL for this service. Once you save that, all that is left is my favorite part, the coding. In fact, this is probably a good place to stop, as we are done with all of the forms and form fields. Next time out, we can focus exclusively on the code.

Useful ServiceNow links:
Main web site:	https://servicenow.com
Developer site:	https://developer.servicenow.com
Annual Conference:	https://knowledge.servicenow.com