Collaboration Store, Part XII

“The slogan ‘press on’ has solved and always will solve the problems of the human race.”
Calvin Coolidge

In the previous installment in this series, we created a new Scripted REST API Resource and referenced another nonexistent function in our Script Include. Now it is time to create that function, which will perform some of the work required to register a new client instance and then hand off the remaining tasks to an asynchronous Subflow so that the function can return the results without waiting for all of the other instances to be notified of the new client instance. The only thing to be done in the function will be to insert the new client instance into the database and kick off the Subflow. But before we do that, we need to first check to make sure that the Client has not already registered with the Host.

var result = {body: {error: {}, status: 'failure'}};

var mbrGR = new GlideRecord('x_11556_col_store_member_organization');
if (mbrGR.get('instance', data.instance)) {
	result.status = 400;
	result.body.error.message = 'Duplicate registration error';
	result.body.error.detail = 'This instance has already been registered with this store.';
} else {
	...

As we did before, we construct our result object with the expectation of failure, since there are more error conditions than the one successful path through the logic. In the case of an instance that has already been registered, we respond with a 400 Bad Request HTTP Response Code and accompanying error details. If the instance is not already in the database, then we attempt to insert it.

mbrGR.initialize();
mbrGR.name = data.name;
mbrGR.instance = data.instance;
mbrGR.email = data.email;
mbrGR.description = data.description;
mbrGR.token = data.sys_id;
mbrGR.active = true;
mbrGR.host = false;
mbrGR.accepted = new GlideDateTime();
if (mbrGR.insert()) {
	result.status = 202;
	delete result.body.error;
	result.body.info = {};
	result.body.info.message = 'Registration complete';
	result.body.info.detail = 'This instance has been successfully registered with this store.';
	result.body.status = 'success';
	...

If the new record was inserted successfully, then we response with a 202 Accepted HTTP Response Code, indicating that the registration was accepted, but the complete registration process (notifying all of the other instances) is not yet complete. At this point, all we have left to do is to initiate the Subflow to handle the rest of the process. We haven’t built the Subflow just yet, but for the purposes of this exercise, we can just assume that it is out there and then we can build it out later. There a couple of different ways to launch an asynchronous Subflow in the background, the original way, and the newer, preferred method. Both methods use the Scripted Flow API. Here is the way that we used to do this:

sn_fd.FlowAPI.startSubflow('New_Collaboration_Store_Instance', {new_instance: data.instance});

… and here is way that ServiceNow would like you to do it now:

sn_fd.FlowAPI.getRunner()
	.subflow('New_Collaboration_Store_Instance')
	.inBackground()
	.withInputs({new_instance: data.instance})
	.run();

Right now, both methods will work, and I’m still using the older, simpler way, but one day I’m going to need to switch over.

There should never be a problem inserting the new record, but just in case, we make that a conditional, and if for some reason it fails, we respond with a 500 Internal Server Error HTTP Response Code.

result.status = 500;
result.body.error.message = 'Internal server error';
result.body.error.detail = 'There was a problem processing this registration request.';

That’s it for all of the little parts and pieces. Here is the entire function, all put together.

processRegistrationRequest: function(data) {
	var result = {body: {error: {}, status: 'failure'}};

	var mbrGR = new GlideRecord('x_11556_col_store_member_organization');
	if (mbrGR.get('instance', data.instance)) {
		result.status = 400;
		result.body.error.message = 'Duplicate registration error';
		result.body.error.detail = 'This instance has already been registered with this store.';
	} else {
		mbrGR.initialize();
		mbrGR.name = data.name;
		mbrGR.instance = data.instance;
		mbrGR.email = data.email;
		mbrGR.description = data.description;
		mbrGR.token = data.sys_id;
		mbrGR.active = true;
		mbrGR.host = false;
		mbrGR.accepted = new GlideDateTime();
		if (mbrGR.insert()) {
			result.status = 202;
			delete result.body.error;
			result.body.info = {};
			result.body.info.message = 'Registration complete';
			result.body.info.detail = 'This instance has been successfully registered with this store.';
			result.body.status = 'success';
			sn_fd.FlowAPI.startSubflow('New_Collaboration_Store_Instance', {new_instance: data.instance});
		} else {
			result.status = 500;
			result.body.error.message = 'Internal server error';
			result.body.error.detail = 'There was a problem processing this registration request.';
		}
	}

	return result;
}

Now we have completed the nonexistent function that was referenced in the REST API Resource, but we have also now referenced a new nonexistent Subflow that we will need to build out before things are complete. That sounds like a good subject for our next installment.

Collaboration Store, Part X

“It’s better to wait for a productive programmer to become available than it is to wait for the first available programmer to become productive.”
Steve McConnell

With the completion of the set-up widget, we can now turn our attention to the missing Script Include functions and the initial registration process that one of those functions will be calling on the Host instance. Since I always like to start with the easy/familiar stuff first (that checks more things off of the “to do” list faster than the other way around!), let’s jump into the createUpdateWorker() function that inserts or updates the Service Account in the sys_user table. But before we do that, we will need to create a special Role for these worker accounts that has limited privileges.

To create a new Role, navigate to System Security -> Roles and click on the New button to bring up the input form. The only field that we need to fill out is the Suffix, which we will set to worker.

Once we have saved our Role, we will want to bring up the form again and grab the sys_id using the Copy sys_id option on the hamburger menu. We will use that to set the value of one of the constants that we will define at the top of our Script Include.

WORKER_ROOT: 'csworker1.',
WORKER_ROLE: 'f1421a6c2fe430104425fcecf699b6a9',

The other constant is the prefix for the user ID for the Service Account, to which we will append the name of the instance. Now that we have defined our Role and set up our constants, we can build the code that will create the account.

var user_name = this.WORKER_ROOT + gs.getProperty('instance_name');
var userGR = new GlideRecord('sys_user');
if (!userGR.get('user_name', user_name)) {
	userGR.initialize();
	userGR.user_name = user_name;
	userGR.insert();
}
userGR.first_name = 'CS';
userGR.last_name = 'Worker';
userGR.title = 'Collaboration Store Worker';
userGR.active = true;
userGR.locked_out = false;
userGR.web_service_access_only  = true;
userGR.user_password.setDisplayValue(passwd);
userGR.update();

Since it is possible that an earlier failed attempt to set up the app already created the account, we first check for that, and if it isn’t already present in the sys_user table, then we create it. Then we set the appropriate fields to their current values and update the record. One thing to note is the way in which we update the user_password field, which is different than all of the others. Because the value of that field is one-way encrypted, we have to set the Display Value instead of the Value. It took me a bit of research to figure that out; it is not very well documented anywhere that I could find.

Once we create the account, we then have to assign it to the Role that we created earlier. Once again, this may have already been done in an earlier failed attempt, so we have to check for that before proceeding.

var userRoleGR = new GlideRecord('sys_user_has_role');
userRoleGR.addEncodedQuery('user=' + userGR.getUniqueValue() + '^role=' + this.WORKER_ROLE);
userRoleGR.query();
if (!userRoleGR.next()) {
	userRoleGR.initialize();
	userRoleGR.user = userGR.getUniqueValue();
	userRoleGR.role = this.WORKER_ROLE;
	userRoleGR.insert();
}

That takes care of creating/updating the account and applying the Role. Putting it all together, the entire function looks like this:

createUpdateWorker: function(passwd) {
	var user_name = this.WORKER_ROOT + gs.getProperty('instance_name');
	var userGR = new GlideRecord('sys_user');
	if (!userGR.get('user_name', user_name)) {
		userGR.initialize();
		userGR.user_name = user_name;
		userGR.insert();
	}
	userGR.first_name = 'CS';
	userGR.last_name = 'Worker';
	userGR.title = 'Collaboration Store Worker';
	userGR.active = true;
	userGR.locked_out - false;
	userGR.web_service_access_only  = true;
	userGR.user_password.setDisplayValue(passwd);
	userGR.update();

	var userRoleGR = new GlideRecord('sys_user_has_role');
	userRoleGR.addEncodedQuery('user=' + userGR.getUniqueValue() + '^role=' + this.WORKER_ROLE);
	userRoleGR.query();
	if (!userRoleGR.next()) {
		userRoleGR.initialize();
		userRoleGR.user = userGR.getUniqueValue();
		userRoleGR.role = this.WORKER_ROLE;
		userRoleGR.insert();
	}
},

Well, that takes care of the easy part. Next time, we will start digging into the more complex elements of the remaining work needed to complete the set-up process.

Collaboration Store, Part IX

“You can’t go back and make a new start, but you can start right now and make a brand new ending.”
James R. Sherman

Now that we have dealt with the two Script Include functions that were referenced in the Save process, we can return to our widget and address the Set-up process. A number of things have to happen in the Set-up process, including saving the user’s input in the database, creating a service account in the User table for authenticated REST API activities, and in the case of a client instance, we need to register the client with the specified Host instance. We will need to build out the other side of that registration process as well, which will include sharing the new instance information with all of the existing instances as well as sharing all of the existing instances with the new client. That’s quite a bit of work, but we’ll take it all on one piece at a time.

One of the first things that we will want to do, regardless of which type of instance we are setting up, is to save the user’s input. That’s pretty basic GlideRecord stuff, but let’s lay it all out here just the same.

mbrGR.initialize();
mbrGR.active = true;
mbrGR.instance = thisInstance;
mbrGR.name = data.instance_name;
mbrGR.email = data.email;
mbrGR.description = data.description;
mbrGR.name = data.instance_name;
if (data.instance_type == 'host') {
	mbrGR.host = true;
	mbrGR.accepted = new GlideDateTime();
	gs.setProperty('x_11556_col_store.host_instance', thisInstance);
} else {
	mbrGR.host = false;
	gs.setProperty('x_11556_col_store.host_instance', data.host_instance_id);
}
mbrGR.insert();

Most data fields are the same for both Host and Client instances, but a Host instance gets the host field set to true and the accepted date valued, while the Client instance gets the host field set to false and the accepted date is not valued until the registration process is successful.

Now that we have entered the user’s input into the database, we will want to perform additional actions depending on the type of instance. For a Client instance, we will want to register the Client with the Host, and for the purpose of this widget, we can just assume for now that there is a function in our Script Include that handles all of the heavy lifting for that operation.

var resp = csu.registerWithHost(mbrGR);

That simple function call represents a lot of work, but it’s code that we really don’t want cluttering up our widget, so we will stuff it all into the Script Include and worry about building it all out later. We will want to check on how it all came out though, because if it was successful, we will want to update the accepted date for the instance and add the Host instance to our database table as well. Again, this is all pretty standard GlideRecord stuff, so nothing really exciting to see here.

mbrGR.accepted = new GlideDateTime();
mbrGR.update();
mbrGR.initialize();
mbrGR.instance = input.store_info.instance;
mbrGR.accepted = input.store_info.accepted;
mbrGR.description = input.store_info.description;
mbrGR.name = input.store_info.name;
mbrGR.email = input.store_info.email;
mbrGR.active = true;
mbrGR.host = true;
mbrGR.insert();

If the registration process was not successful though, we will want delete the record that we just created, inform the user of the error, and cycle the phase back to 1 to start all over again.

mbrGR.deleteRecord();
var errMsg = resp.error;
if (resp.body && resp.body.result && resp.body.result.error) {
	errMsg = resp.body.result.error.message + ': ' + resp.body.result.error.detail;
}
gs.addErrorMessage(errMsg);
data.validationError = true;

One of the things that will happen during the registration process in the Script Include will be to create the Service Account to be used for authenticated REST API calls to the instance. Since the registration process is only called for Client instances, we will need to handle that directly when setting up a Host instance. Once again, we can assume that there is a Script Include function that handles that process, which greatly simplifies the code in the widget.

csu.createUpdateWorker(mbrGR.getUniqueValue());

Putting it all together, the entire server side Javascript code for the set-up process now looks like this:

function setupProcess() {
	gs.setProperty('x_11556_col_store.store_name', data.store_name);
	mbrGR.initialize();
	mbrGR.active = true;
	mbrGR.instance = thisInstance;
	mbrGR.name = data.instance_name;
	mbrGR.email = data.email;
	mbrGR.description = data.description;
	mbrGR.name = data.instance_name;
	if (data.instance_type == 'host') {
		mbrGR.host = true;
		mbrGR.accepted = new GlideDateTime();
		gs.setProperty('x_11556_col_store.host_instance', thisInstance);
	} else {
		mbrGR.host = false;
		gs.setProperty('x_11556_col_store.host_instance', data.host_instance_id);
	}
	mbrGR.insert();
	if (data.instance_type == 'host') {
		csu.createUpdateWorker(mbrGR.getUniqueValue());
	} else {
		var resp = csu.registerWithHost(mbrGR);
		if (resp.responseCode == '202') {
			mbrGR.accepted = new GlideDateTime();
			mbrGR.update();
			mbrGR.initialize();
			mbrGR.instance = input.store_info.instance;
			mbrGR.accepted = input.store_info.accepted;
			mbrGR.description = input.store_info.description;
			mbrGR.name = input.store_info.name;
			mbrGR.email = input.store_info.email;
			mbrGR.active = true;
			mbrGR.host = true;
			mbrGR.insert();
		} else {
			mbrGR.deleteRecord();
			var errMsg = resp.error;
			if (resp.body && resp.body.result && resp.body.result.error) {
				errMsg = resp.body.result.error.message + ': ' + resp.body.result.error.detail;
			}
			gs.addErrorMessage(errMsg);
			data.validationError = true;
		}
	}		
}

That’s it for the work on the widget itself. Of course, we still have a lot of work to do to both complete the referenced Script Include functions that do not yet exist, and to build out the registration process that one of those functions will be calling. There is a lot to choose from there for the subject of our next installment, but when the time comes, we will pick something and dive in.

Adding Detail to the Upgrade History Task

“Ask not that the journey be easy; ask instead that it be worth it.”
John F. Kennedy

With the addition of the Paris Upgrade Center, a new type of Task was introduced, the Upgrade History Task. Whenever you upgrade your instance, issues that require your attention will now produce an Upgrade History Task, which you can assign to the appropriate resources for resolution. This is a nice feature, and fairly well implemented, but the form layout is not quite organized in the way that I would like to see it, and there are some tidbits of data from various other, related tables that I would really like to see all in one place. Another thing that annoys me is that the Upgrade Details Related Link opens up a new window and breaks out of the iFrame, losing the header and sidebar menu. That page has a link back to the Upgrade History Task, and if you click back and forth a few times, suddenly you have all kinds of windows open, and none of them have navigation anymore. I don’t like that.

So, I thought about making a bunch of UI Formatters and rearranging the stock form layout to include all of the information that I like to see when I am working an upgrade issue, but the more upgrades that I work on, the less I like to tinker with stock components. Ultimately, I decided to just add a single UI Action that would pop up a modal dialog box that contained the information that I was looking for. Here are the things that I wanted to see:

  • A standard link to the Skip record that didn’t open up a new window,
  • The name of the affected table,
  • A link to the affected record,
  • When the affected record was last updated,
  • Who last updated the affected record,
  • Any indication that we have dealt with this issue before, and if so,
  • The details of what was done with this the last time that it came up in an upgrade.

None of that is present on the form right now, but I didn’t see any reason that we couldn’t pull it all together from the various sources, so I went to work. It seemed like there would be a bit of database querying to get all of this information, and I didn’t really want all of that on the page itself, so I started out by making myself a little Script Include to house all of major code. I called it UpgradeTaskUtils and created a single function called getAdditionalInfo to pull together all of the data.

var UpgradeTaskUtils = Class.create();
UpgradeTaskUtils.prototype = {
    initialize: function() {
    },

    getAdditionalInfo: function(sysId) {
		var details = {};

		// all of the hard work goes here!

		return details;
	},

	type: 'UpgradeTaskUtils'
};

The first thing that we will need to do, of course, is to use the incoming sysId argument to go out and get the Upgrade History Record, and once we do that, we can then pull out the associated Skip record.

details.taskGR = new GlideRecord('upgrade_history_task');
if (details.taskGR.get(sysId)) {
	details.logGR = details.taskGR.upgrade_detail.getRefRecord();
	if (details.logGR && details.logGR.isValidRecord()) {
		// more code to follow ...
	}
}

Once we know that we have a valid Skip record, the next thing that we will want to do is go get the actual record that has the issue. That’s a little more complicated and uses a table called sys_metadata.

var metaGR = new GlideRecord('sys_metadata');
if (metaGR.get('sys_update_name', details.logGR.file_name.toString())) {
	details.recordGR = new GlideRecord(metaGR.sys_class_name);
	if (details.recordGR.get(metaGR.sys_id)) {
		details.lastRecordUpdate = details.recordGR.getDisplayValue('sys_updated_on');
		details.lastRecordUpdateBy = details.recordGR.getDisplayValue('sys_updated_by');
		// more code to follow ...
	}
}

Since the sys_updated_by fields is just a user_name string and not an actual reference to a User record, if we want to have the details on the User who last updated the record, we will need to go out and fetch that separately.

var userGR = new GlideRecord('sys_user');
if (userGR.get('user_name', details.lastRecordUpdateBy)) {
	details.userName = userGR.getDisplayValue('name');
	details.userSysId = userGR.getUniqueValue();
	details.userLink = '<a href="/sys_user.do?sys_id=' + details.userSysId + '">' + details.userName + '</a>';
}

That takes care of all of the interrelated records involved with this Task, but there is still more work to do if we want any historical data for this same artifact in previous upgrades. Basically, we want to find all of the records in the Upgrade History Task table that reference this same component, except for the one that we already have. We can just do a quick count to start off with, just to see if there is any point in looking any further.

details.previousIssues = 0;
details.prevIssueQuery = 'upgrade_detail.file_name=' + details.taskGR.upgrade_detail.file_name + '^number!=' + details.taskGR.number;
var taskGA = new GlideAggregate('upgrade_history_task');
taskGA.addAggregate('COUNT');
taskGA.addEncodedQuery(details.prevIssueQuery);
taskGA.query();
if (taskGA.next()) {
	details.previousIssues = taskGA.getAggregate('COUNT');
}

Now that we have a count of what’s out there, we can gather up all of the details if the count is greater than zero.

details.prevIssueLink = details.previousIssues;
if (details.previousIssues > 0) {
	details.prevIssueLink = '<a href="/upgrade_history_task_list.do?sysparm_query=' + details.prevIssueQuery + '">' + details.previousIssues + '</a>';
	var taskGR = new GlideRecord('upgrade_history_task');
	taskGR.addEncodedQuery(details.prevIssueQuery);
	taskGR.orderByDesc('sys_created_on');
	taskGR.query();
	if (taskGR.next()) {
		details.previousUpgrade = taskGR.getDisplayValue('upgrade_detail.upgrade_history.to_version');
		details.previousComments = taskGR.getDisplayValue('upgrade_detail.comments');
	}
}

That’s everything that I am looking for right at the moment. I may end up going back one day and tossing in a few more items, but for now, this should do the trick. All together, the new Script Include looks like this:

var UpgradeTaskUtils = Class.create();
UpgradeTaskUtils.prototype = {
    initialize: function() {
    },

    getAdditionalInfo: function(sysId) {
		var details = {};

		details.taskGR = new GlideRecord('upgrade_history_task');
		if (details.taskGR.get(sysId)) {
			details.logGR = details.taskGR.upgrade_detail.getRefRecord();
			if (details.logGR && details.logGR.isValidRecord()) {
				var metaGR = new GlideRecord('sys_metadata');
				if (metaGR.get('sys_update_name', details.logGR.file_name.toString())) {
					details.recordGR = new GlideRecord(metaGR.sys_class_name);
					if (details.recordGR.get(metaGR.sys_id)) {
						details.lastRecordUpdate = details.recordGR.getDisplayValue('sys_updated_on');
						details.lastRecordUpdateBy = details.recordGR.getDisplayValue('sys_updated_by');
						var userGR = new GlideRecord('sys_user');
						if (userGR.get('user_name', details.lastRecordUpdateBy)) {
							details.userName = userGR.getDisplayValue('name');
							details.userSysId = userGR.getUniqueValue();
							details.userLink = '<a href="/sys_user.do?sys_id=' + details.userSysId + '">' + details.userName + '</a>';
						}
					}
				}
			}
			details.previousIssues = 0;
			details.prevIssueQuery = 'upgrade_detail.file_name=' + details.taskGR.upgrade_detail.file_name + '^number!=' + details.taskGR.number;
			var taskGA = new GlideAggregate('upgrade_history_task');
			taskGA.addAggregate('COUNT');
			taskGA.addEncodedQuery(details.prevIssueQuery);
			taskGA.query();
			if (taskGA.next()) {
				details.previousIssues = taskGA.getAggregate('COUNT');
			}
			details.prevIssueLink = details.previousIssues;
			if (details.previousIssues > 0) {
				details.prevIssueLink = '<a href="/upgrade_history_task_list.do?sysparm_query=' + details.prevIssueQuery + '">' + details.previousIssues + '</a>';
				var taskGR = new GlideRecord('upgrade_history_task');
				taskGR.addEncodedQuery(details.prevIssueQuery);
				taskGR.orderByDesc('sys_created_on');
				taskGR.query();
				if (taskGR.next()) {
					details.previousUpgrade = taskGR.getDisplayValue('upgrade_detail.upgrade_history.to_version');
					details.previousComments = taskGR.getDisplayValue('upgrade_detail.comments');
				}
			}
		}

		return details;
	},

	type: 'UpgradeTaskUtils'
};

Now we need put all of this data on page that we can use for our modal pop-up. I created a new UI Page called upgrade_history_task_info, and started it out by calling our new Script Include to obtain the data.

<?xml version="1.0" encoding="utf-8" ?>
<j:jelly trim="false" xmlns:j="jelly:core" xmlns:g="glide" xmlns:j2="null" xmlns:g2="null">
	<g:evaluate var="jvar_not_used">
var utu = new UpgradeTaskUtils();
var obj = utu.getAdditionalInfo(RP.getWindowProperties().get('task_sys_id'));
obj.lastRecordUpdate;		
	</g:evaluate>
	<div style="padding: 20px;">
		<!-- display grid goes here -->
	</div>
</j:jelly>

To format the data, I just used a basic HTML table, with one column for the labels and another for the data.

<table>
	<tr>
		<th style="padding: 5px;">Issue Details:</th>
		<td style="padding: 5px;"><a href="/${obj.logGR.getTableName()}.do?sys_id=${obj.logGR.getUniqueValue()}">${obj.logGR.getDisplayValue()}</a></td>
	</tr>
	<tr>
		<th style="padding: 5px;">Affected Table:</th>
		<td style="padding: 5px;">${obj.recordGR.getLabel()} (${obj.recordGR.getTableName()})</td>
	</tr>
	<tr>
		<th style="padding: 5px;">Affected Record:</th>
		<td style="padding: 5px;"><a href="/${obj.recordGR.getTableName()}.do?sys_id=${obj.recordGR.getUniqueValue()}">${(obj.recordGR.getDisplayValue() > ''?obj.recordGR.getDisplayValue():obj.recordGR.getUniqueValue())}</a></td>
	</tr>
	<tr>
		<th style="padding: 5px;">Record Last Updated:</th>
		<td style="padding: 5px;">${obj.recordGR.getDisplayValue('sys_updated_on')}</td>
	</tr>
	<tr>
		<th style="padding: 5px;">Record Last Updated By:</th>
		<td style="padding: 5px;"><g:no_escape>${(obj.userSysId > ''?obj.userLink:obj.recordGR.getDisplayValue('sys_updated_by'))}</g:no_escape></td>
	</tr>
	<tr>
		<th style="padding: 5px;">Previous Upgrade Issues:</th>
		<td style="padding: 5px;"><g:no_escape>${obj.prevIssueLink}</g:no_escape></td>
	</tr>
	<j:if test="${obj.previousIssues > 0}">
		<tr>
			<th style="padding: 5px;">Last Upgrade w/Issue:</th>
			<td style="padding: 5px;">${obj.previousUpgrade}</td>
		</tr>
		<tr>
			<th style="padding: 5px;">Last Upgrade Comments:</th>
			<td style="padding: 5px;">${obj.previousComments}</td>
		</tr>
	</j:if>
</table>

Now we have our data and we have it laid out nicely on a page, all that’s left to do is to pop it up on the screen. For that, we will need to build a UI Action. I called mine Additional Info, linked it to the Upgrade History Task table, and gave it the following onClick script:

function openAdditionalInfo() {
	var dialog = new GlideDialogWindow('upgrade_history_task_info');
	dialog.setSize(600, 600);
	dialog.setTitle('Additional Info');
	dialog.setPreference('task_sys_id', NOW.sysId);
	dialog.render();
}

That’s pretty much all there is to that. We still need to pull it up and click on it and see what happens, but assuming that all goes well, this exercise should have produced a nice little tool to make plodding through the skipped records in an upgrade just a little bit easier.

Additional Info modal pop-up on the Upgrade History Task form

Nice! Now all we need to do is gather up all of the parts and stuff them into an Update Set.

Hacking the Scripted REST API, Part II

“If you find a path with no obstacles, it probably doesn’t lead anywhere.”
Frank A. Clark

Last time, we built a Scripted REST API with a single Scripted REST Resource. To finish things up, we just need to add the actual script to the resource. To process our legacy support form, create an Incident, and respond to the user, the script will need to handle the following operations:

  • Obtain the input form field values from the request,
  • Format the data for use in creating the Incident,
  • Locate the user based on the email address, if one exists,
  • Create the Incident, and
  • Return the response page.

Obtain the input form field values from the request

This turned out to be much simpler than I first made it out to be. I knew that the out-of-the-box Scripted REST API was set up to handle JSON, both coming in and going out, and could also support XML, but I never saw anything regarding URL encoded form fields, so I assumed I would have to parse and unencode the request body myself. The problem that I was having, though, was getting the actual request body. I tried request.body, request.body.data, request.body.dataString, and even request.body.dataStream — nothing produced anything but null or errors. Then I read somewhere that that the Scripted REST API treats form fields as if they were URL query string parameters, and lumps them all together in a single map: request.queryParams. Once I learned that little tidbit of useful information, the rest was easy.

// get form data from POST
var formData = request.queryParams;
var name = formData.name[0];
var email = formData.email[0];
var short_description = formData.title[0];
var description = formData.description[0];

It did take me a bit to figure out that the values returned from the map are arrays and not strings, but once that became clear in my testing, I just added an index to the form field name and everything worked beautifully.

Format the data for use in creating the Incident

This was just a matter of formatting a few of the incoming data values with labels so that anything that did not have a field of its own on the Incident could be included in the Incident description.

// format the data
var full_description = 'The following issue was reported via the Legacy Support Form:\n\n';
full_description += 'Name: ' + name + '\n';
full_description += 'Email: ' + email + '\n\n';
full_description += 'Details:\n' + description;

Locate the user based on the email address

This doesn’t really have much to do with hacking the Scripted REST API, but I threw it in just as an example of the kind of thing that you can do once you have some data with which to work. In this case, we are simply using the email address that was entered on the form to search the User table to see if we have a user with that email. If we do, then we can use as the Caller on the Incident.

// see if we have a user on file with that email address
var contact = null;
var userGR = new GlideRecord('sys_user');
if (userGR.get('email', email)) {
	contact = userGR.getUniqueValue();
}

Create the Incident

This part is pretty vanilla as well.

// create incident
var incidentGR = new GlideRecord('incident');
if (contact) {
	incidentGR.caller_id = contact;
} else {
	incidentGR.caller_id.setDisplayValue('Guest');
}
incidentGR.contact_type = 'self-service';
incidentGR.short_description = short_description;
incidentGR.description = full_description;
incidentGR.assignment_group.setDisplayValue('Service Desk');
incidentGR.insert();
var incidentId = incidentGR.getDisplayValue('number');

The last line grabs the Incident number from the inserted Incident so that we can send that back to the user on the response page.

Return the response page

Now that we have complete all of the work, the last thing left to do is to respond to the user. Again, since we are not using the native JSON or XML formats, we are going to have to do some of the work a little differently than the standard Scripted REST API. Here is the working code:

// send response page
response.setContentType('text/html');
response.setStatus(200);
var writer = response.getStreamWriter();
writer.writeString(getResponsePageHTML(incidentId));

The first thing that you have to know is that you must set the content type and status before you get the stream writer from the response. If you don’t do that first, then things will not work. And even though you clicked the override checkbox and specified text/html as the format in the Scripted REST API definition, you still have to set it here as well. But once you do all of that, and do it in the right sequence, it all works great.

The response string itself is just the text of a standard HTML page. I encapsulated my sample page into a function so that it could be easily replaced with something else without disturbing any of the rest of the code. The sample version that I put together looks like this:

function getResponsePageHTML(incidentId) {
	var html = '';

	html += '<html>';
	html += ' <head>';
	html += '  <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.5.0/css/bootstrap.min.css" integrity="sha384-9aIt2nRpC12Uk9gS9baDl411NQApFmC26EwAOH8WgZl5MYYxFfc+NcPb1dKGj7Sk" crossorigin="anonymous">';
	html += '  <script src="https://stackpath.bootstrapcdn.com/bootstrap/4.5.0/js/bootstrap.min.js" integrity="sha384-OgVRvuATP1z7JjHLkuOU7Xw704+h835Lr+6QL9UvYjZE3Ipu6Tp75j7Bh/kR0JKI" crossorigin="anonymous"></script>';
	html += ' </head>';
	html += ' <body>';
	html += '  <div style="padding: 25px;">';
	html += '   <h2>Thank you for your input</h2>';
	html += '   <p>We appreciate your business and value your feedback. Your satisfaction is of the utmost concern to us.</p>';
	html += '   <p>Your issue has been documented and one of our marginally competent technicians will get back to you within a few months to explain to you what you have been doing wrong.</p>';
	html += '   <p>Your incident number is ' + incidentId + '.</p>';
	html += '  </div>';
	html += ' </body>';
	html += '</html>';

	return html;
}

That’s it for the coding in ServiceNow. To see how it all works, we just need to point our old form to our new form processor after which we can pull up the modified form in a browser and give it a try. To repoint the legacy form, pull it up in a text editor, find the form tag, and enter the path to our Scripted REST API in the action attribute:

<form action="https://<instance>.service-now.com/api/<scope>/legacy/support" method="post">

With that little bit of business handled, we can pull up the form, fill it out, click on the submit button, and if all goes well, be rewarded for our efforts with our HTML response, including the number of the newly created Incident:

HTML response from the Scripted REST API form processor

Just to be sure, let’s pop over to our instance and check out the Incident that we should have created.

Incident created from submitting the legacy form

Well, that’s all there is to that. We built a Scripted REST API that accepted standard form fields as input and responded with a standard HTML web page, and then pointed an old form at it to produce an Incident and a thank you page. All in all, not a bad little perversion of the REST API feature of the tool!

My Delegates Widget, Enhanced

“No matter how good you get you can always get better, and that’s the exciting part.”
Tiger Woods

The other day I was out on the ServiceNow Developers site and noticed that this project was highlighted on the home page of the Share site as an Editor’s Choice.

ServicePortal delegate widget

This project is quite interesting to me because it is basically an alternative approach to something that I had attempted to accomplish with the My Delegates Widget a while back. I’m always curious to see how other folks address things, so I downloaded it and checked it out. As you would assume from the 5 star rating, it’s pretty nice. In fact, it has a feature that I had not even considered back when I had put mine together: in addition to being able to manage your own delegates, you can also see who has delegated authority to you. I like that. In fact, I liked it so much that I decided to see if I could add that feature to my own slant on building this widget.

The first thing I did was to dust off my old widget, bring it up, and refresh my memory as to how it was that I put it together. This one has been out there for a while, so in addition to adding this new feature, I’m also going to have to clean it up a bit. For one thing, this was built before I read in one our HealthScans that using the name gr for variables that represent a GlideRecord is considered bad form. I think I originally copied that technique from some internal ServiceNow code, so I thought it was a little disingenuous for them to be complaining about that at the time. Still, I’ve stopped doing that ever since, and I’ve been fixing it whenever I come across it in my older stuff.

All that aside, it was a pretty simple addition. I copied the block of code that generated the list of delegates (once I cleaned it up a bit) to make a second one to build a list of the delegations of others. The modification to the query was basically to switch from searching for delegates where user is the current user to searching for users where the delegate is the current user. Also, since this data is not editable (delegation is controlled by the person doing the delegating), I decided to format it as plain English rather than put it in a data table. The final product turned out like this:

function fetchList2() {
	var list = [];
	var today = new Date();
	var delegationGR = new GlideRecord('sys_user_delegate');
	delegationGR.addQuery('delegate', data.userID);
	delegationGR.orderBy('user.name');
	delegationGR.query();
	while (delegationGR.next()) {
		var stillActive = true;
		var endDate = '';
		if (delegationGR.getValue('ends')) {
			endDate = new GlideDate();
			endDate.setValue(delegationGR.getValue('ends'));
			endDate = endDate.getByFormat('M/d/yyyy');
			if (today.after(new Date(endDate))) {
				stillActive = false;
			} else {
				if (new Date(endDate).getFullYear() == 2100) {
					endDate = '';
				}
			}
		}
		if (stillActive) {
			var thisDelegation = {};
			var delegations = [];
			if (delegationGR.getValue('approvals') == 1) {
				delegations.push('Approvals');
			}
			if (delegationGR.getValue('assignments') == 1) {
				delegations.push('Assignments');
			}
			if (delegationGR.getValue('notifications') == 1) {
				delegations.push('CC on Notifications');
			}
			if (delegationGR.getValue('invitations') == 1) {
				delegations.push('Meeting Invitations');
			}
			if (delegations.length > 0) {
				thisDelegation.sys_id = delegationGR.getValue('sys_id');
				thisDelegation.id = delegationGR.getValue('user');
				thisDelegation.user = delegationGR.getDisplayValue('user');
				thisDelegation.ends = endDate;
				thisDelegation.delegations = '';
				var separator = '';
				for (var i=0; i<delegations.length; i++) {
					thisDelegation.delegations += separator;
					thisDelegation.delegations += delegations[i];
					if (delegations.length > 2) {
						separator = ', ';
						if (i == (delegations.length - 2)) {
							separator = ', and ';
						}
					} else {
						separator = ' and ';
					}
				}
				list.push(thisDelegation);
			}
		}
	}
	return list;
}

I also built this widget before I created my Dynamic Service Portal Breadcrumbs widget, so at the time I was always hesitant to leave the page without having a way back. I added that widget to the top of my test page, and then I converted all of the user names on the page to links out to the User Profile page for that person, which adds another nice little feature. The final HTML that formatted the delegations turned out like this:

  <div style="width: 100%; padding: 5px 50px;" ng-show="data.list2Items.length>0">
    <b>I am a Delegate for:</b>
    <div ng-repeat="item in c.data.list2Items track by item.id | orderBy: 'user'" style="padding: 5px;">
      <sn-avatar class="avatar-small-medium" primary="item.id" show-presence="true"/>
       
      <a href="?id=user_profile&table=sys_user&sys_id={{item.id}}" title="{{item.user}}">{{item.user}}</a>
      for {{item.delegations}}
      <span ng-show="item.ends"> until {{item.ends}}</span>
    </div>

All in all, putting this together was a fairly straightforward operation that added a nice new feature to the widget, and gave me a chance to do a little tidying up of some older code. Here’s what it ended up looking like in practice:

My Delegates widget with the addition of delegations of others

Much thanks to Fredrik Larsson for the work he did on his version of a Service Portal delegate widget, and for giving me the idea to make my own a little better. For those of you who like to play along at home, here is an Update Set with all of the latest parts and pieces.

I know it’s in here somewhere …

“Three can keep a secret, if two of them are dead.”
Benjamin Franklin

Every once in a while I am chasing some issue or tying to remember how I did something and I know the answer is in the system somewhere, but I’m just not sure where. Since I mainly deal with Javascript code, what I would really like is a way to search all of the places in which script might be stored looking for some term or phrase. And I mean all of the places, including places that might come up later in future versions. What I really want to do is dynamically look at all of the tables and find all of the columns that might store script and then search all of those columns in all of those tables. Fortunately, because of the way in which the Now Platform is constructed, you can easily do exactly that.

The sys_dictionary table holds all of the information on all of the columns in all of the tables, including the column type. There are several ServiceNow column types that might contain script, but fortunately for us, they all contain the word script in their names. That makes it relatively easy to search the dictionary for all of the script columns in all of the tables:

var table = {};
var columnGR = new GlideRecord('sys_dictionary');
columnGR.addEncodedQuery('internal_typeCONTAINSscript^active=true');
columnGR.query();
while (columnGR.next()) {
	var tableName = columnGR.getDisplayValue('name');
	var fieldName = columnGR.getDisplayValue('element');
	if (tableName && fieldName && !tableName.startsWith('var__m_')) {
		if (!table[tableName]) {
			table[tableName] = [];
		}
		table[tableName].push(fieldName);
	}
}

This script builds a map keyed by table name that contains an array of script columns for each table. Before I add a column to the map, I make sure that there is both a field name and a table name, and I also filter out all of the variable mapping tables, as those don’t contain any scripts and some of them actually cause problems when I attempt to use them. Once we have established our target map, it is simply a matter of stepping through it and querying each table for the presence of your search argument in any of those columns:

var found = [];
for (tableName in table) {
	var query = '';
	var separator = '';
	for (var i=0; i<table[tableName].length; i++) {
		query += separator;
		query += table[tableName][i];
		query += 'CONTAINS';
		query += string;
		separator = 'OR';
	}
	var scriptGR = new GlideRecord(tableName);
	scriptGR.addEncodedQuery(query);
	scriptGR.query();
	while (scriptGR.next()) {
		found.push({table: tableName,  tableName: scriptGR.getLabel(), sys_id: scriptGR.getUniqueValue(), name: scriptGR.getDisplayValue('name') || scriptGR.getDisplayValue() || scriptGR.getUniqueValue()});
	}
}
return found;

This portion of the script builds an array of objects containing the table, table name, name, and sys_id of any records found during the search. I bundled the whole thing into a Script Include that I called ScriptUtils, and you can see the entire thing here:

var ScriptUtils = Class.create();
ScriptUtils.prototype = {
	initialize: function() {
	},

	findInScript: function(string) {
		var table = {};
		var found = [];
		var columnGR = new GlideRecord('sys_dictionary');
		columnGR.addEncodedQuery('internal_typeCONTAINSscript^active=true');
		columnGR.query();
		while (columnGR.next()) {
			var tableName = columnGR.getDisplayValue('name');
			var fieldName = columnGR.getDisplayValue('element');
			if (tableName && fieldName && !tableName.startsWith('var__m_')) {
				if (!table[tableName]) {
					table[tableName] = [];
				}
				table[tableName].push(fieldName);
			}
		}
		for (tableName in table) {
			var query = '';
			var separator = '';
			for (var i=0; i<table[tableName].length; i++) {
				query += separator;
				query += table[tableName][i];
				query += 'CONTAINS';
				query += string;
				separator = 'OR';
			}
			var scriptGR = new GlideRecord(tableName);
			scriptGR.addEncodedQuery(query);
			scriptGR.query();
			while (scriptGR.next()) {
				found.push({table: tableName,  tableName: scriptGR.getLabel(), sys_id: scriptGR.getUniqueValue(), name: scriptGR.getDisplayValue('name') || scriptGR.getDisplayValue() || scriptGR.getUniqueValue()});
			}
		}
		return found;
	},

    type: 'ScriptUtils'
};

To test is out, we can can select Scripts – Background from the sidebar menu and enter something like this:

gs.info(JSON.stringify(new ScriptUtils().findInScript('xxx'), null, 4));

After running the above script, I received the following output:

[
    {
        "table": "discovery_proc_handler",
        "tableName": "Process Handler",
        "sys_id": "1f2473269733200010cb1bd74b297576",
        "name": "Java parameters"
    },
    {
        "table": "sys_script",
        "tableName": "Business Rule",
        "sys_id": "4532f571bf320100710071a7bf073929",
        "name": "Obfuscate password"
    },
    {
        "table": "sys_script_execution_history",
        "tableName": "Script Execution History",
        "sys_id": "807208e92ffb48d0ddadfe7cf699b696",
        "name": "Created 2020-03-26 05:55:27"
    },
    {
        "table": "sys_script_include",
        "tableName": "Script Include",
        "sys_id": "326b53699f3010008f88ed93ee4bcc2b",
        "name": "ScrumSecurityManagerDefault"
    },
    {
        "table": "sys_script_include",
        "tableName": "Script Include",
        "sys_id": "3f7d6f17537103003248cfa018dc347c",
        "name": "PwdResetPageInfo"
    },
    {
        "table": "sys_script_include",
        "tableName": "Script Include",
        "sys_id": "57042a36932012001aa1f4b8b67ffb95",
        "name": "TourBuilderUtility"
    },
    {
        "table": "sys_script_include",
        "tableName": "Script Include",
        "sys_id": "88c548dc37010100dcd48c00dfbe5d2e",
        "name": "SnmpIdentityInfoParser"
    },
    {
        "table": "sys_script_include",
        "tableName": "Script Include",
        "sys_id": "8dbee16b530203003248cfa018dc349e",
        "name": "PwdResetPageInfo_V2"
    },
    {
        "table": "sys_script_include",
        "tableName": "Script Include",
        "sys_id": "91a92c70733023008b516cb63cf6a79e",
        "name": "CommunityCacheUtilSNCJSC"
    },
    {
        "table": "sys_script_include",
        "tableName": "Script Include",
        "sys_id": "a7ac57b7c710320003fa9c569b976312",
        "name": "MIDUserConnectivity"
    },
    {
        "table": "sys_script_include",
        "tableName": "Script Include",
        "sys_id": "c54c989f37612100dcd48c00dfbe5df4",
        "name": "CiSchema"
    },
    {
        "table": "sys_script_include",
        "tableName": "Script Include",
        "sys_id": "ee735ba66713220089ec9a6617415a75",
        "name": "CommunityForumImpl"
    },
    {
        "table": "sys_script_include",
        "tableName": "Script Include",
        "sys_id": "eee51271eb223100c46ac2eef106fed4",
        "name": "AddScriptedRESTVersionAjax"
    }
]

Now, there are obviously better ways to format and display that information, but that would be an entire project on its own, so we’ll just save that exercise for another time.

Customizing the Data Table Widget

“The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore, all progress depends on the unreasonable man.”
George Bernard Shaw

After spending some time playing around with my Data Table Widget Content Selector, I realized that there were a few things that I wanted out of the Data Table widget that just weren’t on its list of features. For one thing, the entire row was the source of the link to further details. In the standard ServiceNow UI, the first column on any given list is the link to further details on the subject of the the row, and links in other columns took you to details related to that specific column. On a list of Incidents, for example, clicking on the link on the first column will take you to the Incident, but clicking on a link in any other column, say Assignment Group or Location, will bring up information on the Assignment Group or Location. I wanted to have a Service Portal list that behaved in the same manner. I started rooting around in the code for the Data Table and realized that this would be more than a simple hack, so I decided to clone the widget and create my own SNH Data Table.

One thing that I discovered while playing around with my copy of the Data Table widget was that there was a minor bug in widget related to the fields option, which was the subject of my earlier hack of the Data Table from URL definition widget. Throughout the widget, this option is referenced as fields, but in the actual widget options, it is named field_list. The statement that copied fields from the options didn’t really copy anything, as the actual data was stored under the variable name field_list. So the first thing that I ended up doing with my copy was to change this:

if (!data.fields) {
	if (data.view)
		data.fields = $sp.getListColumns(data.table, data.view);
	else
		data.fields = $sp.getListColumns(data.table);
}

… to this:

if (!data.fields) {
	if (data.field_list) {
		data.fields = data.field_list;
	} else if (data.view) {
		data.fields = $sp.getListColumns(data.table, data.view);
	} else {
		data.fields = $sp.getListColumns(data.table);
	}
}

That seemed to have rectified that little shortcoming, so now back to my intended purpose, which was to provide column-level links rather than the current row-level link. First things first: I needed to find out if we had enough information on hand to provide the links, or if we needed to add some code to gather up more details for columns that could contain a link. Unfortunately, the code the code that gathers up the row data doesn’t reveal much detail on what, exactly, is gathered up for each field in each row:

while (gr._next()) {
	var record = {};
	$sp.getRecordElements(record, gr, data.fields);
	. . .
}

I would have to dig out the code for $sp.getRecordElements() to know what data was being pulled for each field, which sounded a lot like a major hunting expedition. Instead, I took the lazy way out (my favorite!), and just dumped out the result with a gs.info(JSON.Stringify(record)); statement right after the $sp.getRecordElements(). After that, all I had to do was to bring up a list using the widget and then dig through syslog.list. What I learned was that each field in the record object contained a type, a value, and a display_value, but not the name of the table for the reference fields, which I would need if I was going to create a link. So, I needed to add a little code to pick up that extra bit of info for each field where type=reference. There was actually some similar code right above where I needed to insert my own, so what I ended up adding turned out to be a bastardized copy of the preceeding logic:

for (var f in data.fields_array) {
	var fld = data.fields_array[f];
	if (gr.isValidField(fld)) {
		if (record[fld].type == 'reference') {
			record[fld].table = gr.getElement(fld).getED().getReference();
		}
	}
}

Basically, we just loop through all of the fields in the record, look for those where type=reference and then go fetch the reference table from the source GlideRecord. Now that we have everything that we need in the underlying data, the last thing that we need to do will be to alter the HTML to provide individual column links instead of a single link for the entire row. That seems like a good place to start, next time out