Content Selector Configuration Editor, Part II

“If you concentrate on small, manageable steps you can cross unimaginable distances.”
Shaun Hick

Before I was sidetracked by my self-inflicted issues with my Configurable Data Table Widget Content Selector, I was just about to dive into the red meat of my new Content Selector Configuration Editor. Now that those issues have been resolved, we can get back to the fun stuff. As those of you who have been following along at home will recall, there are three distinct sections of the JSON object used to configure the content selector: 1) Perspective, 2) State, and 3) Table. Both the Perspective and State sections are relatively simple arrays of objects, but the Table section is much more complex, having properties for every State of every Table in every Perspective. Since I like to start out with the simple things first, my plan is to build out the Perspective section first, work out all of the kinks, and then pretty much clone that working model to create the State section. Once we get through all of that, then we can deal with the more complicated Table section.

As usual, we will start out with the visual components first and try to get things to look halfway decent before we crawl under the hood and wire everything together. Perspectives have only three properties, a Label, a Name, and an optional list of Roles to limit access to the Perspective. We should be able to lay all of this out in a relatively simple table, with one row for each Perspective. Here is what I came up with:

<div>
  <h4 class="text-primary">${Perspectives}</h4>
</div>
<div>
  <table class="table table-hover table-condensed">
    <thead>
      <tr>
        <th style="text-align: center;">Label</th>
        <th style="text-align: center;">Name</th>
        <th style="text-align: center;">Roles</th>
        <th style="text-align: center;">Edit</th>
        <th style="text-align: center;">Delete</th>
      </tr>
    </thead>
    <tbody>
      <tr ng-repeat="item in c.data.config.perspective">
        <td data-th="Name">{{item.label}}</td>
        <td data-th="Label">{{item.name}}</td>
        <td data-th="Roles">{{item.roles}}</td>
        <td data-th="Edit" style="text-align: center;"><img src="/images/edittsk_tsk.gif" ng-click="editPerspective($index)" alt="Click here to edit the details of this Perspective" title="Click here to edit the details of this Perspective" style="cursor: pointer;"/></td>
        <td data-th="Delete" style="text-align: center;"><img src="/images/delete_row.gif" ng-click="deletePerspective($index)" alt="Click here to permanently delete this Perspective" title="Click here to permanently delete this Perspective" style="cursor: pointer;"/></td>
      </tr>
    </tbody>
  </table>
</div>
<div style="width: 100%; text-align: right;">
  <button ng-click="editPerspective('new')" class="btn btn-primary ng-binding ng-scope" role="button" title="Click here to add a new Perspective">Add a new Perspective</button>
</div>

In addition to the columns for the three properties, I also added a column for a couple of action icons, one to edit the Perspective and one to delete the Perspective. I thought about putting input elements directly in the table for editing the values, but I decided that I would prefer to keep everything read-only unless you specifically asked to edit one of the rows. If you do want to edit one of the rows, then I plan on popping up a simple modal dialog where you can make your changes (we’ll get to that a little later).

I also added a button down at the bottom that you can use to add a new Perspective, which should pop up the same modal dialog without any existing values. Here’s what the layout looks like so far:

Perspective section of the configuration object editor

That’s not too bad. I think that it looks good enough for now. Our action icons reference nonexistent client-side functions right now, though, so next we ought to build those out. The Delete process looks like it might be the simplest of the two, so let’s say we start there. For starters, it’s always a good practice to pop up a confirm dialog before actually deleting anything, and I always like to use the spModal confirm option for that as opposed to a simple Javascript confirm. Since deleting the Perspective will also wipe out any Table information defined for that Perspective, we will want to warn them of that as well. Here is what I came up with:

$scope.deletePerspective = function(i) {
	var confirmMsg = '<b>Delete Perspective</b>';
	confirmMsg += '<br/>Deleting the ';
	confirmMsg += c.data.config.perspective[i].label;
	confirmMsg += ' Perspective will also delete all information for every State of every Table in the Perspective.';
	confirmMsg += '<br/>Are you sure you want to delete this Perspective?';
	spModal.confirm(confirmMsg).then(function(confirmed) {
		if (confirmed) {
			c.data.config.table[c.data.config.perspective[i].name] = null;
			c.data.config.perspective.splice(i, 1);
		}
	});
};

If the operator confirms the delete action, then we first null out all of the Table information for that Perspective, and then we slice out the Perspective from the list. We have to do things in that order. If we removed the Perspective first, then we would lose access to the name, which is needed to null out the Table data. Here is what the confirm dialog looks like on the page:

Perspective Delete Confirmation pop-up

That takes care of the easy one. Now on to the Edit action. For this one, we will use spModal as well, but instead of the confirm method we will be using the open method to launch a small Perspective Editor widget. The open method has an argument called shared that we can use to pass data to and from the widget. Here is the code to launch the widget and collect the updated data when it closes:

$scope.editPerspective = function(i) {
	var shared = {roles:{}};
	if (i != 'new') {
		shared.label = c.data.config.perspective[i].label;
		shared.name = c.data.config.perspective[i].name;
		shared.roles.value = c.data.config.perspective[i].roles;
		shared.roles.displayValue = c.data.config.perspective[i].roles;
	}
	spModal.open({
		title: 'Perspective Editor',
		widget: 'b83b9f342f3320104425fcecf699b6c3',
		shared: shared
	}).then(function() {
		if (i == 'new') {
			c.data.config.table[shared.name] = [];
			i = c.data.config.perspective.length;
			c.data.config.perspective.push({});
		} else {
			if (shared.name != c.data.config.perspective[i].name) {
				c.data.config.table[shared.name] = c.data.config.table[c.data.config.perspective[i].name];
				c.data.config.table[c.data.config.perspective[i].name] = null;
			}
		}
		c.data.config.perspective[i].name = shared.name;
		c.data.config.perspective[i].label = shared.label;
		c.data.config.perspective[i].roles = shared.roles.value;
	});
};

Since this function is intended to be used for both new and existing Perspectives, we have to check to see which one it is in a couple of places. Before we open the widget dialog, we will need to build a new object if this is a new addition, and after the dialog closes, we will have to establish a Table array for the new Perspective as well as establish an index value and an empty object at that index in the Perspective array. Also, if this is an existing Perspective and they have changed the name of the Perspective, then we need to move all of the associated Table information from the old name to the new name and get rid of everything under the old name. Other than that, the new and existing edit processes are pretty much the same.

This takes care of the client side function to launch the widget, but we still need to build the widget. That might get a little involved, and we have already covered quite a bit, so this may be a good place to stop for now. We’ll tackle that Perspective Editor widget first thing next time out, which should wrap up the Perspective section. Maybe we will even have time to clone it all and finish up the State section as well.

Configurable Data Table Widget Content Selector, Corrected

“No great thing is created suddenly.”
Epictetus

While playing around with my new Content Selector Configuration Editor, I ran into a few errors in my Configurable Data Table Widget Content Selector when working in the Portal Page Designer. The problem that I ran into was that errors in widget prevented the widget from appearing in the container, which then prevented you from accessing the widget controls that let you edit the widget options or delete the widget. I had run into something similar before with my Dynamic Service Portal Breadcrumbs, so I pretty much knew what was going on — I just needed to hunt down the specific error. In this particular case, it turned out to me more than one error, and fixing the first one still did not solve the problem completely.

The problem occurs when first dragging the widget onto a page. Before you have an opportunity to edit the widget options and specify a configuration script,the widget tries to run without a configuration script and then it crashes because it has no configuration script. Here is the offending code:

if (!c.data.table) {
	if (c.data.config.defaults.table) {
		refreshPage(c.data.config.defaults.table, c.data.config.defaults.perspective, c.data.config.defaults.state);
	} else {
		window.location.search = '';
	}
}

The problem is that second line that wants to grab the default table value from the defaults object in the configuration. Since we haven’t had a chance to specify a configuration script just yet, there is no defaults object in the configuration, and attempting to access the table property of that nonexistent object will earn you a NullPointerException. That’s not good! Before checking for the table property, we need to first check to see if the defaults object exists. This modification should do the trick:

if (!c.data.table) {
	if (c.data.config.defaults && c.data.config.defaults.table) {
		refreshPage(c.data.config.defaults.table, c.data.config.defaults.perspective, c.data.config.defaults.state);
	} else {
		$location.search('');
	}
}

That keeps the widget from crashing, but there is still no content displayed on the screen in the Page Designer, so I decided to add a little something to the top of the HTML that would only show if there was no configuration script specified. That code looks like this:

<div ng-hide="options && options.configuration_script">
  <div class="alert alert-danger">
    ${You must specify a configuration script using the widget option editor}
  </div>
</div>

Now when you drag a brand new copy of the widget onto the canvas in the Page Designer, you get this:

Content Selector widget with no configuration script specified

That takes care of an empty configuration script name, but what if you enter a name for script that isn’t a valid Script Include? Well, that crashes the widget code as well, so we will need to fix that, too. This time, the issue is on the server side, where we were assuming that we would always get an instance of whatever script was specified. As you can see from the following code snippet, we don’t even bother to check to see if something was returned from the Instantiator:

var configurator = instantiator.getInstance(options.configuration_script);
data.config = configurator.getConfig($sp);
data.config.authorizedPerspective = getAuthorizedPerspectives();
establsihDefaults();

That’s another easy fix, though. We just need to check to make sure that it is there before we attempt to use it.

var configurator = instantiator.getInstance(options.configuration_script);
if (configurator != null) {
	data.config = configurator.getConfig($sp);
	data.config.authorizedPerspective = getAuthorizedPerspectives();
	establishDefaults();
}

Here is another instance where it would be good to let the developer know that something is amiss, so I added yet anothe DIV to the top of the HTML:

<div ng-show="options && options.configuration_script && !data.config.defaults">
  <div class="alert alert-danger">
    {{options.configuration_script}} ${is not a valid Script Include}
  </div>
</div>

Here’s how that looks in action:

Error message when a invalid Script Include is specified

That should resolve all of the errors that I have discovered so far. Here is the corrected Update Set, which should replace all of the broken parts in the last one and get things working again.

Content Selector Configuration Editor

“Nothing in the world can take the place of Persistence. Talent will not; nothing is more common than unsuccessful men with talent. Genius will not; unrewarded genius is almost a proverb. Education will not; the world is full of educated derelicts. Persistence and Determination alone are omnipotent.”
Calvin Coolidge

Some time ago, I built a little Service Portal widget designed to allow a User to select various sets of records to be display in the Data Table widget on a portal page. I called this widget the Configurable Data Table Widget Content Selector because I designed it to be driven by an external JSON configuration object that could be specified as a widget option. Of course, I ended up just hard-coding the first one during my development and had to go back in later and fix it so that you could actually do that, but now that all works — you just have to build a new JSON configuration object if you want to use the widget on a different page for another purpose. That JSON object is a little complicated, though, so it occurred to me that it would be even better if there was some kind of input screen with some validation that would help you put one of those together. I did that not too long ago for the sn-record-picker, which seems to have worked out fairly well, so I was imagining something very similar, but maybe a little more complex.

There are three main sections to the Content Selector: 1) Perspectives, 2) States, and 3) Tables. My configuration wizard, then, would need a simple section where you could set up your Perspectives, another simple section where you could set up your States, and then a more complicated section where you could set up the Tables for every State of every Perspective. That third section sounds a little overwhelming at first glance, but like most complicated issues, if we break it down into its component parts and focus on one thing at time, we should be able to work through it.

The sn-record-picker Helper was just another portal widget, so that seemed like a decent approach to this endeavor as well. I called it the Content Selector Configurator, and placed it alone on a page of the same name for testing. To begin the process, we need to select an existing applicable Script Include, or enter a name if you want to create a brand new one. We can use an sn-record-picker for selecting an existing one, or to make things even easier, an snh-form-field of type reference, which is just a wrapper around the sn-record-picker that includes all of the labels and validation stuff. To limit the selection to just those Script Includes that are relevant to this process, we can filter on the field that contains the actual script looking for the code that extends the base class. The full snh-form-field element looks like this:

<snh-form-field
  snh-label="Content Selector Configuration"
  snh-model="c.data.script"
  snh-name="script"
  snh-type="reference"
  snh-help="Select the Content Selector Configuration that you would like to edit."
  snh-change="scriptSelected();"
  snh-required="true"
  placeholder="Choose a Content Selector Configuration"
  table="'sys_script_include'"
  default-query="'active=true^scriptCONTAINSObject.extendsObject(ContentSelectorConfig'"
  display-field="'name'"
  search-fields="'name'"
  value-field="'api_name'"/>

… and renders out like this:

Script Include picker

When the selection is made, the snh-change attribute will invoke the scriptSelected() client-side function, so we will need to code that out as well to handle the choice that was made. All of the work necessary to handle the selection will actually occur on the server side, so the client side function just has to kick things over there.

$scope.scriptSelected = function() {
	c.server.update();
};

Over on the server side, things are a little more complicated. We need to use the name of the script to get an instance of the script, and for that, we can use our old friend, the Instantiator. Once we have an instance of the script, we can call the getConfig() function to get a copy of the current configuration object. but before we do that, we have to make sure that we have an input object and we don’t already have a config object. All together, the code looks like this:

if (input) {
	if (!data.config && input.script && input.script.value) {
		data.scriptInclude = input.script.value;
		if (data.scriptInclude.startsWith('global.')) {
			data.scriptInclude = data.scriptInclude.split('.')[1];
		}
		var instantiator = new Instantiator();
		instantiator.setRoot(this);
		var configScript = instantiator.getInstance(data.scriptInclude);
		data.config = configScript.getConfig($sp);
	}
}

Now that we have all of that out of the way, we can work on the actual wizard itself. I wrapped all of the HTML related to selecting a config script in one DIV and then made another, currently empty DIV for the wizard. I used complementary ng-show attributes to hide the wizard until the script was selected, and then hide the selection components once the choice was made. The whole thing now looks like this:

<snh-panel title="'${Content Selector Configuration Editor}'" class="panel-primary">
  <form id="form1" name="form1" ng-submit="save();" novalidate>
    <div class="row" ng-show="!c.data.script.value">
      <div class="col-sm-12">
        <snh-form-field
          snh-label="Content Selector Configuration"
          snh-model="c.data.script"
          snh-name="script"
          snh-type="reference"
          snh-help="Select the Content Selector Configuration that you would like to edit."
          snh-change="scriptSelected();"
          snh-required="true"
          placeholder="Choose a Content Selector Configuration"
          table="'sys_script_include'"
          default-query="'active=true^scriptCONTAINSObject.extendsObject(ContentSelectorConfig'"
          display-field="'name'"
          search-fields="'name'"
          value-field="'api_name'"/>
      </div>
    </div>
    <div class="row" ng-show="c.data.script.value">
      (the wizard lives here)
    </div>
  </form>
</snh-panel>

I still have to add in the ability to create a new script from scratch, but I think I will deal with that later as I am anxious to jump into the wizard itself. I’ll circle back and toss that in before we wrap things up, but right now I just want to get on to fun stuff. That’s an entirely different process, though, so this seems like a good place to stop for now. We’ll jump straight into the wizard next time out.

Fun with Webhooks, Improved

“I think it’s very important to have a feedback loop, where you’re constantly thinking about what you’ve done and how you could be doing it better.”
Elon Musk

So far, I have had relatively good luck playing around with my Simple Webhooks app, and have been able to post content to other systems such as Slack and MS Teams in addition to the test cases that I sent over to webhook.site. One thing that I did notice, though, was that my portal page for editing the details of a Webhook was missing a couple of items found on the corresponding form in the main UI. On the form for a Webhook in the main UI, I built a UI Action that you can use to send a test POST to your URL, and the form also includes a Delete button that you can use to get rid of your Webhook when you no longer need it or want it. The current version of the portal page has neither of those features, so I decided that it was time to add those in.

The first order of business, then, was to add the two buttons to the HTML, right after the existing Save button:

&nbsp;
<button ng-show="c.data.sysId" ng-click="testURL()" class="btn btn-default ng-binding ng-scope" role="button" title="Click here to send a test POST to this URL">Test URL</button>
&nbsp;
<button ng-show="c.data.sysId" ng-click="deleteThis()" class="btn btn-default ng-binding ng-scope" role="button" title="Click here to permanently delete this webhook">Delete</button>

I didn’t want them showing up on new records, since there is no point in deleting a record that you haven’t created yet, so I added an ng-show attribute based on the presence of an existing sys_id. Other than that, it’s just a basic copy and paste of the other button code with some minor modifications. Here’s how it looks rendered out:

New buttons added to the form HTML

The new buttons reference new client-side functions, so next we will need to add those to the existing client-side script. Here are the two that I came up with:

$scope.testURL = function() {
	spModal.confirm('Send a test POST to this URL?').then(function(confirmed) {
		if (confirmed) {
			c.data.action = 'test';
			c.server.update();
		}
	});
};

$scope.deleteThis = function() {
	if (c.data.sysId) {
		spModal.confirm('Permanetly delete this Webhook?<br/>(This cannot be undone.)').then(function(confirmed) {
			if (confirmed) {
				c.data.action = 'delete';
				c.server.update().then(function(response) {
					goBack();
				});
			}
		});
	} else {
		goBack();
	}
};

I ended up putting a Confirm pop-up on both of them, even though technically the URL test is not destructive. I just thought that it might be nice to confirm that you really want send something over to another system before you actually did it. I also added the c.data.action variable so that once we were over on the server side, that code would know what to do. In our previous version, the only call to the server side was that Save button, so there was no question what needed to be done. But now that we have multiple possible actions, everyone — including Save — will need to register their intentions by setting this variable to some known value (save, test, or delete) before invoking c.server.update(). All of the actual work to perform the save, test, and delete actions is done on the server side, so let’s pop over there next.

To begin, I pulled out all of the existing Save logic and put it into a function of its own. Then I added the following conditional step, assuming that I would have similar functions for the other two actions:

	if (input) {
		if (input.action == 'save') {
			save(input);
		} if (input.action == 'test') {
			test(input);
		} if (input.action == 'delete') {
			deleteThis(input);
		}
	} else {
		. . . 

The Delete function turned out to be pretty basic stuff:

function deleteThis(input) {
	whrGR.get(input.sysId);
	whrGR.deleteRecord();
	gs.addInfoMessage('Your Webhook data has been deleted.');
}

Most of the code for the Test URL button I just stole from the existing UI Action built for the same purpose. Much of that is buried in the Script Include anyway, so that turned out to be fairly simple as well:

function test(input) {
	whrGR.get(input.sysId);
	var wru = new WebhookRegistryUtils();
	var result = wru.testURL(whrGR);
	if (result.status == '200') {
		gs.addInfoMessage('URL "' + input.url + '" was tested successfully.');
	} else {
		gs.addErrorMessage('URL test failed: ' + JSON.stringify(result, null, '<br/> '));
	}
}

That’s about all there was to that. Technically, you cannot really call this an enhancement since it is functionality that should have been in there from the start. Let’s just call it a much needed improvement. Here’s the new Update Set.

Automatically Link Referenced Tasks, Improved

“The secret of change is to focus all of your energy not on fighting the old, but on building the new”
Socrates

After running my LinkReferencedTasks Business Rule for a while, it has become apparent that there was a flaw in my approach. Whenever someone separates different task numbers with a special character, neither one gets picked up in the process. For example, if you enter something like REQ0010005/RITM0010009, the process that converts all special characters to an empty string ends up converting that to REQ0010005RITM0010009, which starts with REQ, but is not a valid Task number. A little Corrective Maintenance should solve that problem. Let’s convert this:

text = text.replace(/\n/g, ' ');
text = text.replace(/[^A-Z0-9 ]/g, '');

… to this:

text = text.replace(/[^A-Z0-9 ]/g, ' ');

Originally, I had converted all line feeds to spaces and then all characters that were not letters, numbers, or spaces to an empty string. Once I decided to change all characters that were not letters, numbers, or spaces to a space, I no longer needed the line above, as line feeds fall into that same category. That solved that problem.

However, while I was in there, I decided to do a little Perfective Maintenance as well. By changing all of the special characters to single spaces, I thought that I might end up with several spaces in a row, which would result in one or more empty strings in my array of words. To screen those out, I thought about discarding words with a length of zero, but then it occurred to me that short words of any kind will not be task numbers, so I settled on an arbitrary minimum length of 6 instead. Now the code that unduplicates the list of words looks like this:

var unduplicated = {};
for (var i=0; i<words.length; i++) {
	var thisWord = words[i];
	if (thisWord.length > 5) {
		unduplicated[thisWord] = thisWord;
	}
}

This should reduce the clutter quite a bit and minimize the number of words that need to be checked to see if they start with one of the specified prefixes.

And finally, I added an Enhancement, which I had actually thought about earlier, but didn’t implement. This enhancement allows you to specify the relationship type instead of just accepting the hard-coded Investigates relationship that I had put in the original. For backward compatibility, I did not want to make that mandatory, so I kept that as a default for those implementations that did not want to specify their own. The new version of the Script Include now looks like this:

var TaskReferenceUtils = Class.create();
TaskReferenceUtils.prototype = {
    initialize: function() {
    },

	linkReferencedTasks: function(taskGR, prefix, relationship) {
		if (!relationship) {
			relationship = 'd80dc65b0a25810200fe91a7c64e9cac';
		}
		var text = taskGR.short_description + ' ' + taskGR.description;
		text = text.toUpperCase();
		text = text.replace(/[^A-Z0-9 ]/g, ' ');
		var words = text.split(' ');
		var unduplicated = {};
		for (var i=0; i<words.length; i++) {
			var thisWord = words[i];
			if (thisWord.length > 5) {
				unduplicated[thisWord] = thisWord;
			}
		}
		for (var word in unduplicated) {
			for (var x in prefix) {
				if (word.startsWith(prefix[x])) {
					this._findTask(word, taskGR.getUniqueValue(), relationship);
				}
			}
		}
	},

	_findTask: function(number, child) {
		var taskGR = new GlideRecord('task');
		if (taskGR.get('number', number)) {
			this._documentRelationship(taskGR.getUniqueValue(), child, relationship);
		}
	},

	_documentRelationship: function(parent, child, relationship) {
		var relGR = new GlideRecord('task_rel_task');
		relGR.addQuery('parent', parent);
		relGR.addQuery('child', child);
		relGR.query();
		if (!relGR.next()) {
			relGR.initialize();
			relGR.parent = parent;
			relGR.child = child;
			relGR.type = relationship;
			relGR.insert();
		}
	},

    type: 'TaskReferenceUtils'
};

So, a little fix here, a little improvement there, and a brand new feature over there, and suddenly we have a new version better than the last. Stuff’s getting better. Stuff’s getting better every day. Here’s the improved Update Set.

Automatically Link Referenced Tasks

“It is necessity and not pleasure that compels us.”
Dante Alighieri

Occasionally, we will get an Incident Ticket that references another task present in the system. This is usually a status request on a Service Catalog Request or a problem with a recent Change. It would be nice to be able to just click on those recognizable task numbers, but both the Short Description and the Description are plain text fields where that is not an option. There is, however, a handy out-of-the-box UI Formatter that you can include on your Incident Form that provides the capability to link other tasks to your Incident. The name of the Formatter is Task Relations, and I like to drag it onto the Incident Form in the Forms Designer right underneath the Formatter for Contextual Search Results. Once you have that Formatter present on your Incident Form, you can click on the green plus sign and link other tasks of various types to your Incident.

That’s a really cool feature that allows you click on the related tasks to open them up, but being the lazy developer that I am, I would prefer not to have to go through all of the manual work to set up all of the links to the things mentioned in the text of the ticket. In my ideal world, the system would be smart enough to read the text, recognize a task number, and then build the link for me so that I don’t have to do all of that work by hand, and also to make sure that I did not miss anything. How hard could that be?

My thought was that I could create a Business Rule linked to the Incident table that would examine the two primary text fields (Short Description and Description), look for anything that appeared to be a task number, search the Task table to see if it really was a task number, and if so, build the link for me. It seemed like a relatively simple thing to do, so I went to work.

It felt as if there might be a lot of code involved, so instead of putting all of that in the Business Rule itself, I decided to build a Script Include that I could call from the Business Rule. I thought that if I could make the function generic enough, I might be able to reuse it for other Business Rules linked to different Task tables. Plus, putting all of the code in the Script Include keeps the Business Rule a lot cleaner. Here are the things that I thought that I needed to do in order to get this all to work:

  • Combine the two text fields on the Incident into a single string variable for examination,
  • Convert the text to upper case,
  • Convert all line feeds to spaces,
  • Remove all characters that are not letters, numbers, or spaces,
  • Split the string by spaces creating an array of words,
  • Unduplicate the array of words so that we only looked at each unique word once,
  • Examine each word to see if it started with a known task number prefix,
  • Read the Task table for every word that started with a known task number prefix, and
  • Build a relationship record for every Task record that was found on the Task table.

All we need to do now is turn that into code.

Combine the two text fields on the Incident into a single string variable for examination:

var text = taskGR.short_description + ' ' + taskGR.description;

Convert the text to upper case:

text = text.toUpperCase();

Convert all line feeds to spaces:

text = text.replace(/\n/g, ' ');

Remove all characters that are not letters, numbers, or spaces:

text = text.replace(/[^A-Z0-9 ]/g, '');

Split the string by spaces creating an array of words:

var words = text.split(' ');

Unduplicate the array of words so that we only looked at each unique word once:

var unduplicated = {};
for (var i=0; i<words.length; i++) {
	var thisWord = words[i];
	unduplicated[thisWord] = thisWord;
}

Examine each word to see if it started with a known task number prefix:

for (var word in unduplicated) {
	for (var x in prefix) {
		if (word.startsWith(prefix[x])) {
			this._findTask(word, taskGR.getUniqueValue());
		}
	}
}

Read the Task table for every word that started with a known task number prefix:

_findTask: function(number, child) {
	var taskGR = new GlideRecord('task');
	if (taskGR.get('number', number)) {
		this._documentRelationship(taskGR.getUniqueValue(), child);
	}
},

Build a relationship record for every Task record that was found on the Task table:

_documentRelationship: function(parent, child) {
	var relGR = new GlideRecord('task_rel_task');
	relGR.addQuery('parent', parent);
	relGR.addQuery('child', child);
	relGR.query();
	if (!relGR.next()) {
		relGR.initialize();
		relGR.parent = parent;
		relGR.child = child;
		relGR.type = 'd80dc65b0a25810200fe91a7c64e9cac';
		relGR.insert();
	}
},

Before I create a relationship record, I want to make sure that there isn’t already a relationship record out there, so I do a quick query just to check before I commit to inserting a new one. The two records don’t need to be linked more than once. I also hard-coded the relationship type, which works for my current purpose, but if I ever want to expand this out to other use cases, I may eventually want to pass that in as an argument as I did with the task number prefixes. Here is the whole thing, all put together:

var TaskReferenceUtils = Class.create();
TaskReferenceUtils.prototype = {
    initialize: function() {
    },

	linkReferencedTasks: function(taskGR, prefix) {
		var text = taskGR.short_description + ' ' + taskGR.description;
		text = text.toUpperCase();
		text = text.replace(/\n/g, ' ');
		text = text.replace(/[^A-Z0-9 ]/g, '');
		var words = text.split(' ');
		var unduplicated = {};
		for (var i=0; i<words.length; i++) {
			var thisWord = words[i];
			unduplicated[thisWord] = thisWord;
		}
		for (var word in unduplicated) {
			for (var x in prefix) {
				if (word.startsWith(prefix[x])) {
					this._findTask(word, taskGR.getUniqueValue());
				}
			}
		}
	},

	_findTask: function(number, child) {
		var taskGR = new GlideRecord('task');
		if (taskGR.get('number', number)) {
			this._documentRelationship(taskGR.getUniqueValue(), child);
		}
	},

	_documentRelationship: function(parent, child) {
		var relGR = new GlideRecord('task_rel_task');
		relGR.addQuery('parent', parent);
		relGR.addQuery('child', child);
		relGR.query();
		if (!relGR.next()) {
			relGR.initialize();
			relGR.parent = parent;
			relGR.child = child;
			relGR.type = 'd80dc65b0a25810200fe91a7c64e9cac';
			relGR.insert();
		}
	},

    type: 'TaskReferenceUtils'
};

Now that we our Script Include, we need to build the Business Rule that calls it. For my purpose, I added a Business Rule to the Incident table and called it LinkReferencedTasks. I checked the Advanced checkbox and made it active async on Insert whenever there was text in either of the two description fields. I could have also triggered it on Update as well, but in my experience, the Incident description is usually captured when the Incident is created an rarely updated after that.

LinkReferencedTasks Business Rule

Under the Advanced tab, I entered the following script:

(function executeRule(current, previous) {
	new TaskReferenceUtils().linkReferencedTasks(current, ['REQ','RITM','SCTASK','CHG']);
})(current, previous);

In addition to the current GlideRecord, you also need to pass in a string array of task table prefixes which will be used somewhat like a filter to only link tasks that start with those values. If you want a different mix of task types, you can just update that list. That’s it. We are done. Well, maybe we had better test it out first, but the building part is done anyway. Testing should be simple enough: we just need to find an existing task that we want to reference and then include that task number in the text of a new incident. Let’s do that now.

New test Incident with references to other tasks in the Short Description field

Now all we have to do is hit that Submit button and see if those tasks referenced in the text are automatically linked to the Incident. Seems as if a little drum-roll would be appropriate here …

Test results for the new Business Rule and Script Include

Well, nothing ever goes right the very first time! It looks like we managed to create a link to the RITM, but not to the Request. Fortunately, it turns out that this is a tester error and not a developer error. When I typed in the Request number, I missed a zero. The actual Request number is REQ0010005, not REQ001005. Of course, my explanation for that is that I did that on purpose to demonstrate that it will only link real requests, and if your request number is not a real request, then it won’t bother to attempt to create a link to it. That’s it — I did it on purpose — it was all part of the plan. You know that you have to test for failure as well as success — I just did it all in one test because I am so efficient. Sometimes I even amaze myself!

Anyway, it all seems to work, so for those of you who like to play along at home, here’s an Update Set with all of the parts.

Update: There is a better (corrected) version here.

Fun with Webhooks, Part X

“Control is for beginners.”
Ane Størmer

I’ve been playing around with our little Incident Webhook subsystem to make sure that everything works, and to make sure that I had finally developed all of the pieces that I had intended to build. For the most part, I’m quite happy with what we have put together during this exercise, but like most end users who finally get their hands on something that they have ordered, now that I have a working model in my hands and have tried to use if for various things, I can envision a number of different enhancements that would make things even better. Still, what we have is pretty nice all on its own, although I did break down and make just a few minor adjustments.

One thing that I had thought about doing, but didn’t, was to skip the confirmation pop-up on the custom Webhook Registry page’s Cancel button when no changes had been made to the form. Going through that a few times was enough to motivate me to put that in there, and I like this version much better. While I was in there, I also built a goBack() function to house the code for returning to the previous page, and then called that function wherever it was appropriate. This didn’t really save that much in the way of code, since the current goBack() logic is only one line itself, but it consolidates the logic in a single place if I ever want to wire in support for something like my Dynamic Breadcrumbs. The entire client side code for the Webhook Registry widget now looks like this:

function WebhookRegistry($scope, $location, spModal) {
	var c = this;

	$scope.cancel = function() {
		if ($scope.form1.$dirty) {
			spModal.confirm('Abandond your changes and return to your Webhooks?').then(function(confirmed) {
				if (confirmed) {
					goBack();
				}
			});
		} else {
			goBack();
		}
	};

	$scope.save = function() {
		if ($scope.form1.$valid) {
			c.server.update().then(function(response) {
				goBack();
			});
		} else {
			$scope.form1.$setSubmitted(true);
		}
	};

	function goBack() {
		$location.search('id=my_webhooks');
	}
}

One other thing that I noticed when attempting to integrate with various other targets is that many sites are looking for a property named text as opposed to message. I ended up renaming my message field to text to be more compatible with this convention, but it would really be nice to be able to pick and chose what properties you would like to have in your payload, as well as being able to specify what you wanted them to be named. That’s on my wish list for a future version for sure.

Something that I meant to include in this version, but forgot to do, was to emulate the Test URL UI Action on the Webhook Registry widget so that Service Portal users could have that same capability on that portal page. That was definitely on my plan to include, but I just spaced it out when I was putting that all together. I definitely want to be sure to include that at some point in the near future. I would do it now, but I already built the Update Set and I’m just too lazy to go back and fix it now.

One other thing that is on my wish list for some future version is the ability to set this up for more than just the Incident table. I thought about just switching over to the Task table, which includes Incident as well as quite a few other things derived from Task, but the base Task table does not include the Incident’s Caller or the Request’s Requested for, so there would have to be some special considerations included to cover that. The Task table has Opened by, but that’s not really the same thing when you are dealing with folks calling in and dealing with an Agent entering their information. I thought about adding some additional complexity to cover that, but in the end I just put all of that on my One Day … list and left well enough alone.

Based on what I first set out to do, I think it all came out OK, though. Yes, there are quite a few more things that we could add to make it applicable to a broader domain, and there are a number of things that we could do to make it more flexible, user-friendly, and user-customizable, but it’s a decent start. Certainly good enough to warrant the release of an initial version, which you can download here. Since this is a scoped app, I did not bundle any of the dependencies in the Update Set, so if you want to try this out in your own instance as is, you will need to also grab the latest version of SNH Form Fields and SNH ServiceNow Events, which you can find here. All in all, I am happy with the way that it came out, but I am also looking forward to making it even better one day, after I have spent some time attempting to use it as it is today.

Update: There is a better (improved) version here.

Fun with Webhooks, Part IX

“You have brains in your head. You have feet in your shoes. You can steer yourself any direction you choose.”
Theodor Geisel

We still need to test the My Webhooks portal page that we built last time, but before we do that, I wanted to first build out the page referenced in a couple of links on that page so that we could test everything together. My initial thought for that page was to build out a brand new portal widget for that purpose using our old friends SNH Panel and SNH Form Fields. Before I did that, though, it occurred to me that it might be faster to just use the stock form portal page, passing in the name of the table, and potentially a sys_id for existing records. There were a number of things that I did not like about that idea, but I thought that I could overcome those with some UI Policies and some default values for a couple of table fields. I played around with that a bit and found another thing that I didn’t really like, which was that saving a record left you still on the form page and did not bring you back to the My Webhooks page, which I thought was rather annoying. It seemed as though I might be able to mitigate that by adding my Dynamic Service Portal Breadcrumbs to the top of each page, but then I ran into another problem that I could not work around related to the Document ID field. At that point, I gave up and went back to my original plan, which is where I should have started in the first place.

So, here is the HTML for my new Webhook Registry portal widget:

<snh-panel class="panel panel-primary" title="'${Webhook Registry}'">
  <form id="form1" name="form1" novalidate>
    <div class="row">
      <div class="col-xs-12 col-sm-6">
        <snh-form-field
          snh-model="c.data.number"
          snh-name="number"
          snh-label="Number"
          snh-type="text"
          readonly="readonly"/>
        <snh-form-field
          snh-model="c.data.type"
          snh-name="type"
          snh-label="Type"
          snh-type="select"
          snh-required="true"
          snh-choices='[{"label":"Single Item","value":"single"},{"label":"Caller / Requester","value":"requester"},{"label":"Assignment Group","value":"group"},{"label":"Assignee","value":"assignee"}]'/>
      </div>
      <div class="col-xs-12 col-sm-6">
        <snh-form-field
          snh-model="c.data.owner"
          snh-name="owner"
          snh-label="Owner"
          snh-type="text"
          readonly="readonly"/>
        <snh-form-field
          snh-model="c.data.active"
          snh-name="active"
          snh-label="Active"
          snh-type="checkbox"/>
      </div>
    </div>
    <div class="row">
      <div class="col-sm-12">
        <snh-form-field
          snh-model="c.data.url"
          snh-name="url"
          snh-label="URL"
          snh-type="url"
          snh-required="true"/>
      </div>
    </div>
    <div class="row">
      <div class="col-xs-12 col-sm-6">
        <snh-form-field
          snh-model="c.data.document_id"
          snh-name="document_id"
          snh-label="Item"
          snh-type="reference"
          snh-required="c.data.type=='single'"
          table="'incident'"
          display-field="'number'"
          search-fields="'number'"
          value-field="'sys_id'"
          ng-show="c.data.type=='single'"/>
        <snh-form-field
          snh-model="c.data.person"
          snh-name="person"
          snh-label="Person"
          snh-type="reference"
          snh-required="c.data.type=='assignee' || c.data.type=='requester'"
          table="'sys_user'"
          default-query="'active=true'"
          display-field="'name'"
          search-fields="'name'"
          value-field="'sys_id'"
          ng-show="c.data.type=='assignee' || c.data.type=='requester'"/>
        <snh-form-field
          snh-model="c.data.group"
          snh-name="group"
          snh-label="Group"
          snh-type="reference"
          snh-required="c.data.type=='group'"
          table="'sys_user_group'"
          default-query="'active=true'"
          display-field="'name'"
          search-fields="'name'"
          value-field="'sys_id'"
          ng-show="c.data.type=='group'"/>
      </div>
      <div class="col-xs-12 col-sm-6">
        <snh-form-field
          snh-model="c.data.authentication"
          snh-name="authentication"
          snh-label="Authentication"
          snh-type="select"
          snh-required="true"
          snh-choices='[{"label":"None","value":"none"},{"label":"Basic","value":"basic"}]'/>
        <snh-form-field
          snh-model="c.data.username"
          snh-name="username"
          snh-label="Username"
          snh-type="text"
          snh-required="c.data.authentication=='basic'"
          ng-show="c.data.authentication=='basic'"/>
        <snh-form-field
          snh-model="c.data.password"
          snh-name="password"
          snh-label="Password"
          snh-type="password"
          snh-required="c.data.authentication=='basic'"
          ng-show="c.data.authentication=='basic'"/>
      </div>
    </div>
  </form>

  <div style="width: 100%; padding: 5px 50px; text-align: center;">
    <button ng-click="cancel()" class="btn btn-default ng-binding ng-scope" role="button" title="Click here to abandon this update and return to your webhooks">Cancel</button>
    &nbsp;
    <button ng-click="save()" class="btn btn-primary ng-binding ng-scope" role="button" title="Click here to save your input">Save</button>
  </div>
</snh-panel>

There’s nothing too magical there; just a bunch of SNH Form Fields wrapped inside of an SNH Panel. To mirror the UI Policies on the ServiceNow side of things, I used ng-show attributes to hide unneeded fields, and when those fields where required, I used the exact same criteria for the snh-required attribute, which kept it from telling me to complete fields that I couldn’t even see. With just that alone, I could throw the widget onto a page and bring it up, just to see what it looked like.

The new Webhook Registry widget layout

Not too bad, all things considered. Of course, this is just the layout. We still have to put the code underneath this presentation layer. We will definitely need some server side code to read and update the database, but I like to do the easy things first, so let’s start on the client side and throw in the code for the Cancel button. That just takes you right back the My Webhooks page, so that should be pretty simple, although we should bake in a little confirmation pop-up, just to be sure that the operator really does want to abandon their work. We can do that with a plain Javascript confirm, but I like the spModal version much better.

$scope.cancel = function() {
	spModal.confirm('Abandond your changes and return to your Webhooks?').then(function(confirmed) {
		if (confirmed) {
			$location.search('id=my_webhooks');
		}
	});
};

Technically, I should have checked to make sure that something was at least altered before I popped up that confirmation, and if not, just whisked you straight away to the My Webhooks page without asking. I may actually do that at some point, but this works for now. Unlike the Cancel button, the Save button will require some server side code, but we can still code out the client side while we are here and then tackle that next. Here is the code for the Save button.

$scope.save = function() {
	if ($scope.form1.$valid) {
		c.server.update().then(function(response) {
			$location.search('id=my_webhooks');
		});
	} else {
		$scope.form1.$setSubmitted(true);
	}
};

Here we check to make sure that there are no validation errors on the form before invoking the server side code, and then we return to the My Webhooks page once the server process has completed. If there are validation errors, then we set the form status to submitted to reveal all of the field errors to the user. SNH Form Fields hide validation errors until you touch the field or the form has been submitted, so setting the form to submitted here reveals any validation errors present for fields that have not yet been touched.

On the server side, we essentially have two events to handle: 1) widget initialization and 2) handling a Save action. The Save action involves input from the client side, so we know that if there is input present, we are doing a Save; otherwise, we are initializing the widget. At initialization, we need to look for a sys_id parameter in the URL, which tells us that we are updating an existing record. If there isn’t one, then we are adding a new record. For existing records, we need to go get the data and for new records, we need to initialize certain fields. Here is all that code:

data.sysId = $sp.getParameter("sys_id");
if (data.sysId) {
	whrGR.get(data.sysId);
	data.number = whrGR.getDisplayValue('number');
	data.type = whrGR.getValue('type');
	data.url = whrGR.getValue('url');
	data.document_id = {value: whrGR.getValue('document_id'), displayValue: whrGR.getDisplayValue('document_id')};
	data.group = {value: whrGR.getValue('group'), displayValue: whrGR.getDisplayValue('group')};
	data.person = {value: whrGR.getValue('person'), displayValue: whrGR.getDisplayValue('person')};
	data.owner = whrGR.getDisplayValue('owner');
	data.active = whrGR.getValue('active')=='1'?true:false;
	data.authentication =  whrGR.getValue('authentication');
	data.username = whrGR.getValue('username');
	data.password = whrGR.getValue('password');
} else {
	data.owner = gs.getUserDisplayName();
	data.active = true;
	data.document_id = {};
	data.group = {};
	data.person = {};
}

Similarly, when we do a Save, we need to know whether we are doing an update or an insert, which we can again tell by the presence of a sys_id. If we are updating, we need to go out and get the current record, and then in all cases, we need to move the data from the screen to the record and then save it. Here is all of that code:

if (input.sysId) {
	whrGR.get(input.sysId);
}
whrGR.type = input.type;
whrGR.url = input.url;
whrGR.document_id = input.document_id.value;
whrGR.group = input.group.value;
whrGR.person = input.person.value;
whrGR.setValue('active', input.active?'1':'0');
whrGR.authentication = input.authentication;
whrGR.username = input.username;
whrGR.password = input.password;
if (input.sysId) {
	whrGR.update();
} else {
	whrGR.insert();
}

That’s pretty much it for the widget. Just to make sure that it works, we can pull up an existing record and see what shows up on the screen.

Webhook Registry widget with existing record

Once I pulled the record up, I switched the Authentication to Basic and then hit the Save button, just to see if the form validation was working. So far, so good, but there is obviously a lot more testing to do, including the integration with the My Webhooks page. Still, things are looking pretty good at this point.

I’m not quite ready to put out an Update Set at the moment, as there is still quite a bit of testing that I would like to do first. Hopefully, though, I won’t find anything too major and I can drop the whole package next time out.

Fun with Webhooks, Part VIII

“If you are working on something that you really care about, you don’t have to be pushed. The vision pulls you.”
Steve Jobs

When we last parted, there was an open question on the table regarding the focus of this next installment. Implementing Basic Authentication and moving on to the My Webhooks Service Portal widget were two of the options, but no decision was made at the time as to which way we wanted to turn next. Now that we have finally made it here, the good news is that we don’t have to choose. Adding support for Basic Authentication turned out to be so simple that it looks like we are going to have time for both. Check it out:

if (whrGR.getValue('authentication') == 'basic') {
	request.setBasicAuth(whrGR.getValue('user_name'), whrGR.getValue('password'));
}

That’s it. I just added those few lines of code to the postWebhook function of our Script Include and BAM! — now we support Basic Authentication. Pretty sweet!

So now we can devote the remainder of the post to our new My Webhooks widget. As I think I mentioned earlier, this should be a fairly easy clone of the existing My Delegates widget, so the first thing that I did was to make a copy of that guy to have something with which to start. Then I hacked up the HTML to adapt it to our current application.

<table class="table table-hover table-condensed">
  <thead>
    <tr>
      <th style="text-align: center;">ID</th>
      <th style="text-align: center;">Table</th>
      <th style="text-align: center;">Type</th>
      <th style="text-align: center;">Reference</th>
      <th style="text-align: center;">Active</th>
      <th style="text-align: center;">Remove</th>
    </tr>
  </thead>
  <tbody>
    <tr ng-repeat="item in c.data.listItems track by item.id | orderBy: 'id'" ng-hide="item.removed">
      <td data-th="ID"><a href="?id=webhook_registry&sys_id={{item.sys_id}}">{{item.id}}</a></td>
      <td data-th="Table">{{item.table}}</td>
      <td data-th="Type">{{item.type}}</td>
      <td data-th="Reference">{{item.reference}}</td>
      <td data-th="Active" style="text-align: center;"><img src="/images/check32.gif" style="width: 16px; height: 16px;" ng-show="item.active"/></td>
      <td data-th="Remove" style="text-align: center;"><img src="/images/delete_row.gif" ng-click="deleteWebhook($index)" alt="Click here to permanently delete this Webhook" title="Click here to permanently delete this Webhook" style="cursor: pointer;"/></td>
    </tr>
  </tbody>
</table>

With that out of the way, we now need to replace the code that gathers up the Delegates with code that will gather up the Webhooks. Seems simple enough …

function fetchList() {
	var list = [];
	var whrGR = new GlideRecord('x_11556_simple_web_webhook_registry');
	whrGR.addQuery('owner', data.userID);
	whrGR.orderBy('number');
	whrGR.query();
	while (whrGR.next()) {
		var thisWebhook = {};
		thisWebhook.sys_id = whrGR.getValue('sys_id');
		thisWebhook.id = whrGR.getDisplayValue('number');
		thisWebhook.table = whrGR.getDisplayValue('table');
		thisWebhook.type = whrGR.getDisplayValue('type');
		thisWebhook.active = whrGR.getValue('active');
		if (thisWebhook.type == 'Single Item') {
			thisWebhook.reference = whrGR.getDisplayValue('document_id');
		} else if (thisWebhook.type == 'Assignment Group') {
			thisWebhook.reference = whrGR.getDisplayValue('group');
		} else {
			thisWebhook.reference = whrGR.getDisplayValue('person');
		}
		list.push(thisWebhook);
	}
	return list;
}

There is still a lot of work to do, but I like to try things every so often before I get too far along, just to make sure that things are going OK, so let’s do that now. This widget could easily go on the existing User Profile page just like the My Delegates widget, but it could also go on a page of its own. Since we are just trying things on for size right now, let’s just create a simple Portal Page and put nothing on it but our new widget. Let’s call it my_webhooks, drop our widget right in the middle of it, and go check it out on the Service Portal.

Initial My Webhooks widget

Well, that’s not too bad. We don’t really need the Save and Cancel buttons in this instance, but we do need a way to create a new Webhook, so maybe we can replace those with a single Add Webhook button. The links don’t go anywhere just yet and the delete icons don’t do anything, but as far as the layout goes, it looks pretty good. I think it’s a good start so far. Let’s swap out the buttons and then we can wrap things up with the client-side code.

The left-over code in the button area from the My Delegates widget looks like this:

<div style="width: 100%; padding: 5px 50px; text-align: center;">
  <button ng-click="saveDelegates()" class="btn btn-primary ng-binding ng-scope" role="button" title="Click here to save your changes">Save</button>
   
  <button ng-click="returnToProfile()" class="btn ng-binding ng-scope" role="button" title="Click here to cancel your changes">Cancel</button>
</div>

Let’s replace it with this:

<div style="width: 100%; padding: 5px 50px; text-align: center;">
  <button ng-click="newWebhook()" class="btn btn-primary ng-binding ng-scope" role="button" title="Click here to create a new Webhook">Add New</button>
</div>

Now we just need to overhaul the code on the client side to handle both the adding and deleting of our webhooks. The code to support the Add New button should be pretty straightforward; we just need to link to the same (currently nonexistent!) page that we link to from the main table, just without any sys_id to indicate that this is a new record request. This should handle that nicely:

$scope.newWebhook = function() {
	$location.search('id=webhook_registry');
};

As for the delete operation, we will have to bounce over to the server side to handle that one, but first we should confirm that was really the intent of the operator with a little confirmation pop-up. We could use the stock Javascript confirm feature here, but I like the look of the spModal version better, so let’s go with that.

$scope.deleteWebhook = function(i) {
	spModal.confirm('Delete Webhook ' + c.data.listItems[i].id + '?').then(function(confirmed) {
		if (confirmed) {
			c.data.toBeRemoved = i;
			$scope.server.update().then(function(response) {
				reloadPage();
			});
		}
	});
};

We’ll need some server side code to handle the actual deletion of the record, but that should be simple enough.

if (input) {
	data.listItems = input.listItems;
	if (input.toBeRemoved) {
		deleteWebhook(input.toBeRemoved);
	}
}

function deleteWebhook(i) {
	var whrGR = new GlideRecord('x_11556_simple_web_webhook_registry');
	if (whrGR.get(data.listItems[i].sys_id)) {
		whrGR.deleteRecord();
	}
}

We still need to test everything, but before we do that, we should go ahead and build the webhook_registry page that we have been pointing at so that we can fully test those links as well. That sounds like a good project for our next installment, so I think we will wrap things up right here for now and then start off next time with our new page followed by some end to end testing.

Fun with Webhooks, Part VII

“The question isn’t who is going to let me; it’s who is going to stop me.”
Ayn Rand

Now that we have proven that the essential elements of our Subflow all work as intended, it’s time finish out the remainder of the flow’s activities, which include logging the HTTP POST and Response details as well as reporting any undesired results to Event Management. Let’s start with logging the activity.

The first thing that we will need in order to record our Webhook POSTs is a table in which to store the data. As we did with our original Webhook Registry table, we can navigate to System Definition -> Tables and click on the New button to bring up the Table definition screen.

New Webhook Log table

And once again we will give it a Label and let the system generate the associated Name. We will also want to uncheck the Create module checkbox again to prevent the generation of a number of artifacts for which we have no use. Once the table has been defined, we can start adding fields, and the first field that we will want to add is a Reference to the Webhook Registry table. Every log record will be linked to the registry for which the activity was POSTed, so we will want to establish that relationship with a Reference field that we can label Registry.

The other Reference field that we will want is a link back to the original Incident that is the subject of the POST. Since we set things up in a way that would allow us to support tables other than Incident, we will want to do this with a Document ID field rather than a direct reference to the Incident table. This time, when we configure the Dependent Field, we can dot walk through the registry reference to get to the table name column in that related table. This will save us from having to have a column on the log table for the name of the table that holds the record associated with the Document ID, and it will ensure that all of the Document IDs related to each Registry will only come from the table associated with that Registry.

Selecting the Registry record’s Table column as the Dependent Field

The rest of the fields in the log table are just what we sent over, and what we received in response:

  • Payload – The data that we will be POSTing
  • URL – The URL to which we will be POSTing our Payload
  • Status – The HTTP Response Code returned by the target server
  • Body – The body of the message returned by the target server
  • Error – The error flag
  • Error Code – The error code
  • Error Message – The error message
  • Parse Error – Any error that occurred while parsing the body of the response

After defining all of the fields on the table, I brought up the table’s form and arranged all of the fields on the screen in a manner that I thought was most appropriate.

Webhook Log form layout

Those of you who are paying close attention will also have noticed that I added the JSON View Dictionary Attribute to both the Payload and Body fields, just to make reading the JSON content a little easier.

Now that we have a table, we can start putting records into it. We will do this in our Subflow, right after we POST the payload. This is just a simple, out-of-the-box Create Record action that we can configure using data pills from various other steps.

Logging the Webhook POST and Response

In addition to capturing everything related to each POST, the other thing that we wanted to do was to capture any issues that might come up during this process. We are already aware of two possible issues, one being a bad response and the other being a good response, but with an unparsable response body. After we do the POST and log the result, we can throw in a few more conditionals to pick those up, and then add a Log Event Action to the flow for each.

Complete Subflow with Event logging for error conditions

Whenever we log an Event, we will want to capture as much information as we can about what went wrong. In this case, however, we have already logged everything about the transaction to the Webhook Log table, so really all we will need to provide is a way to find that record. Putting the sys_id in the additional_info field should do the trick. Here is how I populated all of the data for the Event:

Event Log data

That should complete the Subflow, at least for now. We may end up adding some other features in the future, but for now, this accomplishes everything that we set out to do. We still need to do a lot more testing to verify that all of these various branches in the tree work out as we are hoping, but the building part should be done now, at least for this portion.

As far as the remaining development goes, we still have to build out the My Webhooks portal widget and we also need to go back into the Script Include and add support for Basic Authentication. We also need to add code, and possibly additional fields in the registry record, for any other authentication protocols that we would like to support. So once again we find ourselves at a crossroads: we can either jump into the Service Portal world and start working on our widget, or we can turn our attentions to authentication and finish things up in that area. There is no need to make any decision on that today, though. We’ll figure all of that out when we meet again.