Collaboration Store, Part XXV

“Controlling complexity is the essence of computer programming.”
Brian Kernighan

Last time, we realized that we had a little bit of rework to do, and it turns out that we actually have to do a couple of things: 1) insert the missing step (attaching the XML to the version record), and 2) modify the ending point if the instance is the Host instance (there is no need to send the records to the Host instance if you are the Host instance). The first part is easy enough; just insert one more DIV for our missing step and then renumber all of the ones that follow:

<div class="row" id="phase_4" style="visibility: hidden; display: none;">
	<image id="loading_4" src="/images/loading_anim4.gif" style="width: 16px; height: 16px;"/>
	<image id="success_4" src="/images/check32.gif" style="width: 16px; height: 16px; visibility: hidden; display: none;"/>
	<image id="error_4" src="/images/delete_row.gif" style="width: 16px; height: 16px; visibility: hidden; display: none;"/>
	<span style="margin-left: 10px; font-weight:bold;">
		Attaching the Update Set XML to the Version record
	</span>
</div>
<div class="row" id="phase_5" style="visibility: hidden; display: none;">
	<image id="loading_5" src="/images/loading_anim4.gif" style="width: 16px; height: 16px;"/>
	<image id="success_5" src="/images/check32.gif" style="width: 16px; height: 16px; visibility: hidden; display: none;"/>
	<image id="error_5" src="/images/delete_row.gif" style="width: 16px; height: 16px; visibility: hidden; display: none;"/>
	<span style="margin-left: 10px; font-weight:bold;">
		Sending the Application record to the Host instance
	</span>
</div>
<div class="row" id="phase_6" style="visibility: hidden; display: none;">
	<image id="loading_6" src="/images/loading_anim4.gif" style="width: 16px; height: 16px;"/>
	<image id="success_6" src="/images/check32.gif" style="width: 16px; height: 16px; visibility: hidden; display: none;"/>
	<image id="error_6" src="/images/delete_row.gif" style="width: 16px; height: 16px; visibility: hidden; display: none;"/>
	<span style="margin-left: 10px; font-weight:bold;">
		Sending the Version record to the Host instance
	</span>
</div>
<div class="row" id="phase_7" style="visibility:hidden; display: none;">
	<image id="loading_7" src="/images/loading_anim4.gif" style="width: 16px; height: 16px;"/>
	<image id="success_7" src="/images/check32.gif" style="width: 16px; height: 16px; visibility: hidden; display: none;"/>
	<image id="error_7" src="/images/delete_row.gif" style="width: 16px; height: 16px; visibility: hidden; display: none;"/>
	<span style="margin-left: 10px; font-weight:bold;">
		Sending the Update Set XML to the Host instance
	</span>
</div>

And of course, we have to insert the missing step in our Script Include, which at this point is just another empty placeholder like all of the others. We’ll build out the details later as we come to that step.

For controlling the point at which we stop doing stuff, we will need to know if this instance is the Host instance or one of the Client instances. The easiest way to do that is to compare our scoped Host instance property with the stock instance_name property.

var isHost = gs.getProperty('instance_name') == gs.getProperty('x_11556_col_store.host_instance');

Then we just need to modify our original conditional statement that only assumed 6 steps and looked like this:

if (answer.phase < 7) {

… to one that will do all 7 steps for a Client instance and only the first 4 steps for a Host instance.

if (answer.phase < 5 || (answer.phase < 8 && !answer.isHost)) {

With that out of the way, we can return to building out the missing steps in the Script Include, starting with the newly added fourth step, which is to attach the Update Set XML to the version record. As you may recall, we already created an attachment record when we generated the XML, so now all we need to do is to transfer that attachment to our new version record. For that, we can return to our old friend, the GlideSysAttachment. This time, instead of creating the attachment record, we will be copying the attachment from one record to another.

var gsa = new GlideSysAttachment();
var newSysId = gsa.copy('sys_app', answer.appSysId, 'x_11556_col_store_member_application_version', answer.versionId);

Once we have copied the attachment from the Scoped Application record to the version record, we will want to delete the attachment record linked to the Scoped Application.

gsa.deleteAttachment(answer.attachmentId);

The last thing that we will need to do will be to update the attachment sys_id in our transfer object so that we will have the ID of the right attachment later on when we go to send it over to the Host.

answer.attachmentId = newSysId;

That makes this a relatively simple step in terms of code. The whole thing looks like this:

processPhase4: function(answer) {
	var gsa = new GlideSysAttachment();
	var newSysId = gsa.copy('sys_app', answer.appSysId, 'x_11556_col_store_member_application_version', answer.versionId);
	gsa.deleteAttachment(answer.attachmentId);
	answer.attachmentId = newSysId;

	return answer;
},

If the instance doing the publishing is a Host instance, this would actually be the end of the process, as far as publishing is concerned. We will still have to notify all of the other instances of the new version, but that’s an entirely separate process that we will deal with at some point in the future. But for our new Publish to Collaboration Store action, this is all that needs to be done if you are the Host. For all of the other instances, the application, version, and attachment records will all have to be sent over to the Host as a part of this process. We’ll get started on those steps of the process next time out.

Collaboration Store, Part XXIV

“It is a myth that we can get systems ‘right the first time.’ Instead, we should implement only today’s stories, then refactor and expand the system to implement new stories tomorrow. This is the essence of iterative and incremental agility. Test-driven development, refactoring, and the clean code they produce make this work at the code level.”
Robert Cecil Martin (Uncle Bob)

In our last installment, we restructured the way in which the client-side code interacted with the server-side Script Include and then we went ahead and built out that first step. Now we just need to keep marching down through the list of steps until we have tackled them all. The next step in the list is to either update the existing member application record, or in the case of a first-time publication, create a brand new member application record. To build that record, we will need to grab the data from the original sys_app record, we should grab that guy right off of the bat.

processPhase2: function(answer) {
	var sysAppGR = new GlideRecord('sys_app');
	if (sysAppGR.get(answer.appSysId)) {
		...
	} else {
		answer = this.processError(answer, 'Invalid Application sys_id: ' + answer.appSysId);
	}

	return answer;
},

As we did before, we throw in a simple check, just to make sure that we were actually able to fetch the record. The next thing that we will need to do is to see if this is a new record or not, and if so, create that initial entry.

var mbrAppGR = new GlideRecord('x_11556_col_store_member_application');
if (answer.mbrAppId == 'new') {
	mbrAppGR.initialize();
	mbrAppGR.provider.setDisplayValue(gs.getProperty('instance_name'));
	mbrAppGR.setValue('application', answer.appSysId);
	mbrAppGR.insert();
	answer.mbrAppId = mbrAppGR.getUniqueValue();
	mbrAppGR.initialize();
}

With a new record created if needed, the rest of the update logic can work for both new and existing records, since we just converted the new record to an existing record by inserting the bare minimum data to kick things off.

if (mbrAppGR.get(answer.mbrAppId)) {
	mbrAppGR.setValue('name', sysAppGR.getValue('name'));
	mbrAppGR.setValue('description', sysAppGR.getValue('short_description'));
	mbrAppGR.setValue('current_version', sysAppGR.getValue('version'));
	mbrAppGR.setValue('active', true);
	mbrAppGR.update();
} else {
	answer = this.processError(answer, 'Invalid Member Application sys_id: ' + answer.mbrAppId);
}

That’s all there is to that. Putting it all together, the entire processPhase2 function looks like this:

processPhase2: function(answer) {
	var sysAppGR = new GlideRecord('sys_app');
	if (sysAppGR.get(answer.appSysId)) {
		var mbrAppGR = new GlideRecord('x_11556_col_store_member_application');
		if (answer.mbrAppId == 'new') {
			mbrAppGR.initialize();
			mbrAppGR.provider.setDisplayValue(gs.getProperty('instance_name'));
			mbrAppGR.setValue('application', answer.appSysId);
			mbrAppGR.insert();
			answer.mbrAppId = mbrAppGR.getUniqueValue();
			mbrAppGR.initialize();
		}
		if (mbrAppGR.get(answer.mbrAppId)) {
			mbrAppGR.setValue('name', sysAppGR.getValue('name'));
			mbrAppGR.setValue('description', sysAppGR.getValue('short_description'));
			mbrAppGR.setValue('current_version', sysAppGR.getValue('version'));
			mbrAppGR.setValue('active', true);
			mbrAppGR.update();
		} else {
			answer = this.processError(answer, 'Invalid Member Application sys_id: ' + answer.mbrAppId);
		}
	} else {
		answer = this.processError(answer, 'Invalid Application sys_id: ' + answer.appSysId);
	}

	return answer;
},

That one was pretty easy compared to what we went through last time, so let’s go ahead and do another. The next step, which is to create the application version record, is even easier, as every version record is an insert, so we don’t have to check for new or existing.

processPhase3: function(answer) {
	var sysAppGR = new GlideRecord('sys_app');
	if (sysAppGR.get(answer.appSysId)) {
		var versionGR = new GlideRecord('x_11556_col_store_member_application_version');
		versionGR.setValue('member_application', answer.mbrAppId);
		versionGR.setValue('version', sysAppGR.getValue('version'));
		versionGR.setValue('installed', true);
		versionGR.insert();
		answer.versionId = versionGR.getUniqueValue();
	} else {
		answer = this.processError(answer, 'Invalid Application sys_id: ' + answer.appSysId);
	}

	return answer;
},

The next thing that we need to do is attach the XML to the version record now that it exists. This is actually something that was not on our original list of things to do, so before we jump into that, we are going to need to correct that oversight. That’s a little bit of rework to code that we have already had to rework a few times already, but that’s the way these things go sometimes. I’m not really feeling all that excited about reworking that one more time right at the moment, though, so let’s say we jump right into that next time out.

Collaboration Store, Part XXII

“Doubt is an uncomfortable condition, but certainty is a ridiculous one.”
Voltaire

Now that we have the face of our new pop-up window laid out, it’s time to build out the code underneath that will do all of the work. Since this is all server-side activity, we will need some kind of client accessible Script Include that we can call from the client side for each step of the process. Just to do a little testing on the structure and the process, I created a stubbed-out version of a Script Include that I can call from the client-side code to do a little testing before we get too far into the weeds.

var ApplicationPublisher = Class.create();
ApplicationPublisher.prototype = Object.extendsObject(global.AbstractAjaxProcessor, {

    processPhaseClient: function() {
		var phase = this.getParameter('sysparm_phase');
		var mbrAppId = this.getParameter('sysparm_mbr_app_id');
		var appSysId = this.getParameter('sysparm_app_sys_id');
		var updSetId = this.getParameter('sysparm_upd_set_id');
		var origAppId = this.getParameter('sysparm_orig_app_id');
		return this.processPhase(phase, mbrAppId, appSysId, updSetId, origAppId);
	},

    processPhase: function(phase, mbrAppId, appSysId, updSetId, origAppId) {
		return 'success';
	},

    type: 'ApplicationPublisher'
});

At this point, all it does is return the value ‘success’ each time that it is called. This will be enough to prove out all of the client-side code before we dig into the actual work on the server side. For now, we just want to prove that the design is functional before we invest too much time in the actual tasks ahead of us.

On the client side, we will want to create a generic function that will work essentially the same for every step of the process. To keep track of where we are, I created a variable called phase, which is basically just the number of the current step. I called this function processPhase.

function processPhase(phase, mbrAppId, appSysId, updSetId, origAppId) {
	var ga = new GlideAjax('ApplicationPublisher');
	ga.addParam('sysparm_name', 'processPhaseClient');
	ga.addParam('sysparm_phase', phase);
	ga.addParam('sysparm_mbr_app_id', mbrAppId);
	ga.addParam('sysparm_app_sys_id', appSysId);
	ga.addParam('sysparm_upd_set_id', updSetId);
	ga.addParam('sysparm_orig_app_id', origAppId);
	ga.getXMLAnswer(function (answer) {
		if (answer == 'success') {
			hideElement('loading_' + phase);
			showElement('success_' + phase);
			phase++;
			if (phase < 7) {
				showElement('phase_' + phase);
				processPhase(phase, mbrAppId, appSysId, updSetId, origAppId);
			} else {
				showElement('done_button');
			}
		} else {
			showElement('done_button');
		}
	});
}

This is a recursive function that calls itself for every step until all six steps have been completed or there was some kind of an error. If the response from the Ajax call is ‘success’, then the loading icon for that step is hidden and replaced with the success icon, and then the phase is incremented. If we haven’t reached the end of the steps, then the HTML block for the next step is revealed and the process is repeated. If we reach the end of the steps, or if the Ajax call returns anything other than success, then the process comes to an end and the Done button is revealed. There is no effort on the client side to communicate any error or completion message to the operator. The assumption is that all messaging will be handled on the server side.

For the hideElement and showElement functions, I just stole some old code from some other component where I needed to do the same thing. There’s not much exciting here, but just for the sake of including everything, here it is:

function showElement(elementName) {
	var elem = gel(elementName);
	elem.style.visibility = 'visible';
	elem.style.display = '';
}

function hideElement(elementName) {
	var elem = gel(elementName);
	elem.style.visibility = 'hidden';
	elem.style.display = 'none';
}

Even though the processPhase function calls itself once it gets going, something has to initially get the ball rolling to kicks things off. For that, we can use an onload function to initialize all of the values and make that first call.

function onLoad() {
	var phase = 1;
	var mbrAppId = gel('mbr_app_id').value;
	var appSysId = gel('app_sys_id').value;
	var updSetId = gel('upd_set_id').value;
	var origAppId = gel('mbr_app_id').value;
	processPhase(phase, mbrAppId, appSysId, updSetId, origAppId);
}

Although that seemed like a good plan, when I ran my first test run, nothing happened. After doing a little online research, I discovered that the onLoad function is not a natural thing for a UI Page, and if I wanted something to run on load, I needed to add just a bit more code.

addLoadEvent(function() {  
	onLoad();
});

That was much better! Now after calling the server side six times in succession, the final rendition of the pop-up screen looked like this:

Pop-up structure test results

That proves that the client-side process functions as we intended and the stubbed-out Script Include is getting called and is returning the hard-coded ‘success’ value. This should now complete the work on the UI Page itself, so all that is left is to finish out the server-side Script Include to perform all of the tasks itemized in our list of steps. Next time, we can get started on the fist step, which will involve calling the global function that we created earlier to convert the Update Set to XML.

Collaboration Store, Part XX

“It’s always too soon to quit!”
Norman Vincent Peale

Now that we have the needed global Script Include out of the way, we can turn our attention to building the Publish to Collaboration Store UI Action. Before we do that, though, we are going to need a couple more tables, one for the published applications, and another for the application versions. I won’t waste a lot of time here on the process of building a new table; that’s pretty basic Now Platform stuff that you can find just about anywhere. I also did not spend a lot of time creating the tables. At this point, they are very basic and just enough to get things going. I am sure at some point I will be coming back around and adding more valuable fields, but for right now, I just included the bare necessities to create this initial publication process.

Here is the form for the base application table:

Base Member Application table input form

… and here is the form for the application versions:

Member Application Version table input form

There is an obvious one-to-many relationship between the application record and all of the various version records for an application. In practice, all of the versions would be listed under an application as a Related List on the application form. We can show examples of that once we build up some data.

Now that that is done, we can get back to our UI Action. Since our action is going to be very similar to the stock Publish to Update Set… action, the easiest way to create ours is to pull up the stock action and use the context menu to select Insert and Stay to clone the action.

Select Insert and Stay from the context menu to clone the UI Action

Now we just have to rename the action and update the script, which actually turned out to be more modifications that I had anticipated, primarily because our action is not in the global scope. First, we had to rename the function to avoid a conflict with the original action with which we will be sharing the page. Also, the gel function and the window object are not available to scoped actions, and of course, we needed to point to our own UI Page instead of the original and put in our own title.. Here is the modified script:

function publishToCollaborationStore() {
	var sysId = g_form.getUniqueValue();
	var dialogClass = GlideModal ? GlideModal : GlideDialogWindow;
	var dd = new dialogClass("x_11556_col_store_publish_app_dialog_cs");
	dd.setTitle(new GwtMessage().getMessage('Publish to Collaboration Store'));
	dd.setPreference('sysparm_sys_id', sysId);
	dd.setWidth(500);
	dd.render();
}

I also removed all of the conditionals for now, since some of that code was not available to a scoped UI Action, and we will end up having our own conditions, anyway, with which we can deal later on. Right now, I am still just focused on seeing if I can get all of this to work.

The next thing that we need to do is to build our own UI Page, on which we can also get a head start by cloning the original using the same Insert and Stay method. The modifications here are a little more extensive, as we want to do much more than just create an Update Set. One thing that we will want to do is check to make sure that the version number is not one that has already been published. To support that, I added some code at the top to pull all of the current versions so that I can stick them into a hidden field.

var existingVersions = '';
var mbrAppId = 'new';
var appSysId = jelly.sysparm_sys_id;
var appGR = new GlideRecord('sys_app');
appGR.get(appSysId);
var appName = appGR.getDisplayValue();
var appVersion = appGR.version || '';
var appDescription = appGR.short_description;
var mbrAppGR = new GlideRecord('x_11556_col_store_member_application');
if (mbrAppGR.get('application', appSysId)) {
	mbrAppId = mbrAppGR.getUniqueValue();
	var versionGR = new GlideRecord('x_11556_col_store_member_application_version');
	versionGR.addQuery();
	versionGR.query('member_application', mbrAppId);
	while (versionGR.next()) {
		existingVersions += '|' + versionGR.getDisplayValue('version');
	}
	existingVersions += '|';
}

To pass this value to the client side for validation, I added a hidden field to the HTML for the page:

<input id="existing_versions" type="hidden" value="$[existingVersions]"/>

To check the validity on the client side, I retained the original version validity check and then added my own check of the existing versions right underneath using the same alert approach to delivering the bad news:

this.existingVersions = gel('existing_versions').value;
...
var vd = g_form.validators.version;
if (!vd)
	console.log("WARNING.  Cannot find validator for version field in parent form.");
else {
	var answer = vd.call(g_form, this.versionField.value);
	if (answer != true) {
		alert(answer);
		return null;
	}
}
if (this.existingVersions.indexOf('|' + versionField.value + '|') != -1) {
	alert('Version ' + versionField.value + ' has already been published to the Collaboration Store');
	return null;
}

The last thing that we have to change is what happens once the Update Set has been created. In the original version that we cloned, once the Update Set has been produced, the operator is then taken to the Update Set form to see the newly created Update Set. We don’t want to do that, as we still have much work to do to, including the following:

  • Update or Create an Application record
  • Create a new Version record linked to the Application record
  • Convert the Update Set to XML
  • Attach the XML to the Version Record
  • Send all of the updated/created records over to the Host instance, including the attached XML

We will need some kind of vehicle in which to perform all of the these tasks and keep the operator updated as to the progress and success or failure of each one of the steps. That sounds like quite a bit of work, so I think that will be a good topic for our next installment.

Collaboration Store, Part XIX

“I like taking things apart and putting them back together. Tinkering. I’d be a professional tinkerer. Tinkerbell. I think that’s what they’re called.”
Chris Carmack

When we took a peek under the hood of the two UI Actions that produced an Update Set from a Scoped Application and turned it into an XML file, it looked like we could leverage most of the code to accomplish our own objective of publishing an app to the Host instance. Unfortunately, after experimenting a bit with various modifications, it appears that some things simply cannot be done within the scope of our Collaboration Store app. Two of the Script Includes involved can only be invoked from another global component (UpdateSetExport and ExportWithRelatedLists), and the delete statement that is executed after the XML file is produced is also something that can only be done in the global scope. I did not really want to create any components outside of our application scope, but in this instance, it doesn’t look like I have any other option.

Given this new tidbit of information, my new goal is to create a single global component and limit the contents of that component to just the things that are absolutely necessary to be in the global scope. I may even try to create this component in the global scope programmatically as part of the set-up process so that a separate global Update Set is not needed, but for now, let’s just see if we can get it to work. I called my new component CollaborationStoreGlobalUtils, and set the Accessible from to All application scopes so that it could be called from any of our scoped components.

var CollaborationStoreGlobalUtils = Class.create();
CollaborationStoreGlobalUtils.prototype = {
    initialize: function() {
    },

	type: 'CollaborationStoreGlobalUtils'
};

All of the things that needed to be in the global scope seemed to be centered around the process to convert an Update Set to an XML file, so I created a function called updateSetToXML to which I passed the GlideRecord for the Update Set. The purpose of the function was to turn the Update Set into XML and return the XML text. Most of the code I lifted straight out of the Publish to XML UI Action and the Processor that it called.

updateSetToXML: function(updateSetGR) {
	var updateSetExport = new UpdateSetExport();
	var rusSysId = updateSetExport.exportUpdateSet(updateSetGR);
	var remoteUpdateGR = new GlideRecord('sys_remote_update_set');
	remoteUpdateGR.get(rusSysId);
	var exporter = new ExportWithRelatedLists('sys_remote_update_set', rusSysId);
	exporter.addRelatedList('sys_update_xml', 'remote_update_set');
	var fakeResponse = this.responseObject();
	exporter.exportRecords(fakeResponse);
	remoteUpdateGR.deleteRecord();
	return fakeResponse.getOutputStreamText();
}

This is an abbreviated version of the code lifted from the stock components, as I removed things like role checking, the validity checking of various GlideRecords, and the value of passed parameters. My thinking is that those things can all be handled in our scoped components before we get to this point, so this function only needs to contain those things that are required to be in the global scope. Since the code that we are borrowing was intended to be used to send the XML file created to the servlet response output stream, I did have to introduce a little bit of creative hackery to replace the expected g_response object parameter in the exportRecords function with an object of my own design.

To construct an object that would be accepted as valid input to the exportRecords function, I dug through the code in that function looking for all of properties and methods that it was expecting to find. Most of those were basic setter functions used to define the HTTP response, and since we weren’t actually going to have an HTTP response, I was able to just include those in the object as empty functions that did nothing and returned nothing.

addHeader: function() {
},
setHeader: function() {
},
setContentType: function() {
},

The one method that did expect a return was the getOutputStream function, and it was expected to return a valid Java output stream object. Back in my Java developer days, whenever I needed to convert an output stream to a string, I always used the java.io.ByteArrayOutputStream class, so I set up a local variable that was an instance of that class and then added a getter to return that variable.

outputStream: new Packages.java.io.ByteArrayOutputStream(),
getOutputStream: function() {
	return this.outputStream;
}

Utilizing the global Packages object is another thing that cannot be done in a Scoped Application, but since we already made the commitment to build this global Script Include, that didn’t turn out to be a problem here. The last thing that I needed to do was to convert the output stream object to a string once the XML was assembled, so I created this additional nonstandard function to obtain the text:

getOutputStreamText: function() {
	var dataAsString = Packages.java.lang.String(this.outputStream);
	dataAsString = String(dataAsString);
	return dataAsString;
}

That’s the entire fake g_response object, and the function that provides the object looks like this:

responseObject: function() {
	return {
		outputStream: new Packages.java.io.ByteArrayOutputStream(),
		addHeader: function() {
		},
		setHeader: function() {
		},
		setContentType: function() {
		},
		getOutputStream: function() {
			return this.outputStream;
		},
		getOutputStreamText: function() {
			var dataAsString = Packages.java.lang.String(this.outputStream);
			dataAsString = String(dataAsString);
			return dataAsString;
		}
	};
}

So far, those are the only two functions that I have had to include in my global component. The entire thing at this stage of the process now looks like this:

var CollaborationStoreGlobalUtils = Class.create();
CollaborationStoreGlobalUtils.prototype = {
    initialize: function() {
    },

	updateSetToXML: function(updateSetGR) {
		var updateSetExport = new UpdateSetExport();
		var rusSysId = updateSetExport.exportUpdateSet(updateSetGR);
		var remoteUpdateGR = new GlideRecord('sys_remote_update_set');
		remoteUpdateGR.get(rusSysId);
		var exporter = new ExportWithRelatedLists('sys_remote_update_set', rusSysId);
		exporter.addRelatedList('sys_update_xml', 'remote_update_set');
		var fakeResponse = this.responseObject();
		exporter.exportRecords(fakeResponse);
		remoteUpdateGR.deleteRecord();
		return fakeResponse.getOutputStreamText();
	},

	responseObject: function() {
		return {
			outputStream: new Packages.java.io.ByteArrayOutputStream(),
			addHeader: function() {
			},
			setHeader: function() {
			},
			setContentType: function() {
			},
			getOutputStream: function() {
				return this.outputStream;
			},
			getOutputStreamText: function() {
				var dataAsString = Packages.java.lang.String(this.outputStream);
				dataAsString = String(dataAsString);
				return dataAsString;
			}
		};
	},

	type: 'CollaborationStoreGlobalUtils'
};

Now that we have that little piece out of the way, we can start building our Publish to Collaboration Store UI Action and the associated UI Page that will be launched in the modal pop-up when that action is selected. Unless we have some additional feedback from the set-up process testing, we’ll jump right into that next time out.

Collaboration Store, Part XV

“The beginning is the most important part of the work.”
Plato

With the completion of the last piece of the registration service, the only remaining component of the set-up process is the Client instance function that utilizes the registration service provided by the Host instance. This function will actually be quite similar to the function that we just created to inform one instance about another. This time, we will be invoking the Scripted REST API instead of a stock REST API, but the process is virtually the same.

Before we make the call, however, we need to take care of few little items. First, we need to create the Service Account needed to access the instance, and then we need to grab a couple of our System Properties. We already created a function to establish the Service Account, so all we need to do is to call that function and then grab the two property values.

this.createUpdateWorker(mbrGR.getUniqueValue());
var host = gs.getProperty('x_11556_col_store.host_instance');
var token = gs.getProperty('x_11556_col_store.active_token');

Now we can build the payload from the instance GlideRecord that was passed to the function.

var payload = {};
payload.sys_id = mbrGR.getUniqueValue();
payload.name = mbrGR.getDisplayValue('name');
payload.instance = mbrGR.getDisplayValue('instance');
payload.email = mbrGR.getDisplayValue('email');
payload.description = mbrGR.getDisplayValue('description');

At this point, we can create a new instance of our old friend, the sn_ws.RESTMessageV2 object, and then populate it with all of the relevant information.

var request = new sn_ws.RESTMessageV2();
request.setHttpMethod('post');
request.setBasicAuth(this.WORKER_ROOT + host, token);
request.setRequestHeader("Accept", "application/json");
request.setEndpoint('https://' + host + '.service-now.com/api/x_11556_col_store/v1/register');
request.setRequestBody(JSON.stringify(payload));

… and as we did before, we obtain the response object by invoking the execute method.

var response = request.execute();

Now all we have to do is to populate the result object with the information contained in the response.

result.responseCode = response.getStatusCode();
result.bodyText = response.getBody();
try {
	result.body = JSON.parse(response.getBody());
} catch(e) {
	//
}
if (response.getErrorCode()) {
	result.error = response.getErrorMessage();
	result.errorCode = response.getErrorCode();
} else if (result.responseCode != '202') {
	result.error = 'Invalid HTTP Response Code: ' + result.responseCode;
} else {
	mbrGR.accepted = new GlideDateTime();
	mbrGR.update();
}

The complete function, including the return of the result, looks like this:

registerWithHost: function(mbrGR) {
	var result = {};

	this.createUpdateWorker(mbrGR.getUniqueValue());
	var host = gs.getProperty('x_11556_col_store.host_instance');
	var token = gs.getProperty('x_11556_col_store.active_token');
	var payload = {};
	payload.sys_id = mbrGR.getUniqueValue();
	payload.name = mbrGR.getDisplayValue('name');
	payload.instance = mbrGR.getDisplayValue('instance');
	payload.email = mbrGR.getDisplayValue('email');
	payload.description = mbrGR.getDisplayValue('description');
	var request = new sn_ws.RESTMessageV2();
	request.setHttpMethod('post');
	request.setBasicAuth(this.WORKER_ROOT + host, token);
	request.setRequestHeader("Accept", "application/json");
	request.setEndpoint('https://' + host + '.service-now.com/api/x_11556_col_store/v1/register');
	request.setRequestBody(JSON.stringify(payload));
	var response = request.execute();
	result.responseCode = response.getStatusCode();
	result.bodyText = response.getBody();
	try {
		result.body = JSON.parse(response.getBody());
	} catch(e) {
		//
	}
	if (response.getErrorCode()) {
		result.error = response.getErrorMessage();
		result.errorCode = response.getErrorCode();
	} else if (result.responseCode != '202') {
		result.error = 'Invalid HTTP Response Code: ' + result.responseCode;
	} else {
		mbrGR.accepted = new GlideDateTime();
		mbrGR.update();
	}

	return result;
}

This final function completes the initial set-up process for our new Scoped Application. The application still doesn’t do anything in the way of sharing components between instances, but it’s a start. Next time, we will figure out where we go from here.

Collaboration Store, Part XIV

“I find that the harder I work, the more luck I seem to have.”
Thomas Jefferson

With the completion of the asynchronous Subflow, we now need to turn our attention to the Script Include function that was referenced in the custom Action built for the repeating step in the Subflow. This function needs to tell each existing instance about the new instance and tell the new instance about all of the existing instances. Both tasks can actually be accomplished with the same code, leveraging the out-of-the-box REST API for ServiceNow tables. Basically, what we will be doing will be to insert a new record into the instance table for both operations, which can be easily handled with the stock API.

Assuming that we are passed a GlideRecord for both the instance to contact and the instance to be shared, the first thing that we will want to do is to create an object containing all of the relevant information about the instance to be shared.

var payload = {};
payload.instance = instanceGR.getDisplayValue('instance');
payload.name = instanceGR.getDisplayValue('name');
payload.description = instanceGR.getDisplayValue('description');
payload.email = instanceGR.getDisplayValue('email');
payload.accepted = instanceGR.getDisplayValue('accepted');
payload.active = instanceGR.getDisplayValue('active');
payload.host = instanceGR.getDisplayValue('host');

Once we have constructed the payload object from the shared instance GlideRecord, we will want to construct the end point URL using the instance name from the target instance GlideRecord.

var url = 'https://' + targetGR.getDisplayValue('instance') + '.service-now.com/api/now/table/x_11556_col_store_member_organization';

Now we need to build a new RESTMessageV2 with the information that we have assembled for this action.

var request = new sn_ws.RESTMessageV2();
request.setEndpoint(url);
request.setHttpMethod('POST');
request.setBasicAuth(this.WORKER_ROOT + targetGR.getDisplayValue('instance'), targetGR.getDisplayValue('token'));
request.setRequestHeader('Content-Type', 'application/json');
request.setRequestHeader('Accept', 'application/json');
request.setRequestBody(JSON.stringify(payload, null, '\t'));

To obtain the response object, we just need to invoke the execute method on the newly created RESTMessageV2 object.

var response = request.execute();

… and now we just have to examine the response to populate the result object.

result.status = response.getStatusCode();
result.body = response.getBody();
if (result.body) {
	try {
		result.obj = JSON.parse(result.body);
	} catch (e) {
		result.parse_error = e.toString();
	}
}
result.error = response.haveError();
if (result.error) {
	result.error_code = response.getErrorCode();
	result.error_message = response.getErrorMessage();
}

Once that has been completed, all that is left is to return the result object back to the caller. All together, the function looks like this:

publishInstance: function(instanceGR, targetGR) {
	var result = {};

	var payload = {};
	payload.instance = instanceGR.getDisplayValue('instance');
	payload.name = instanceGR.getDisplayValue('name');
	payload.description = instanceGR.getDisplayValue('description');
	payload.email = instanceGR.getDisplayValue('email');
	payload.accepted = instanceGR.getDisplayValue('accepted');
	payload.active = instanceGR.getDisplayValue('active');
	payload.host = instanceGR.getDisplayValue('host');
	var url = 'https://' + targetGR.getDisplayValue('instance') + '.service-now.com/api/now/table/x_11556_col_store_member_organization';
	var request = new sn_ws.RESTMessageV2();
	request.setEndpoint(url);
	request.setHttpMethod('POST');
	request.setBasicAuth(this.WORKER_ROOT + targetGR.getDisplayValue('instance'), targetGR.getDisplayValue('token'));
	request.setRequestHeader('Content-Type', 'application/json');
	request.setRequestHeader('Accept', 'application/json');
	request.setRequestBody(JSON.stringify(payload, null, '\t'));
	var response = request.execute();
	result.status = response.getStatusCode();
	result.body = response.getBody();
	if (result.body) {
		try {
			result.obj = JSON.parse(result.body);
		} catch (e) {
			result.parse_error = e.toString();
		}
	}
	result.error = response.haveError();
	if (result.error) {
		result.error_code = response.getErrorCode();
		result.error_message = response.getErrorMessage();
	}

	return result;
}

We can now utilize this function in the function called by our custom Action. That function is passed the two instance names, so we will need to fetch the GlideRecords for those instances, and then pass those GlideRecords to the publishInstance function, once to tell the existing instance about the new instance, and then again to tell the new instance about the existing instance.

publishNewInstance: function(new_instance, target_instance) {
	instanceGR = new GlideRecord('x_11556_col_store_member_organization');
	instanceGR.get('instance', new_instance);
	targetGR = new GlideRecord('x_11556_col_store_member_organization');
	targetGR.get('instance', target_instance);
	this.publishInstance(instanceGR, targetGR);
	this.publishInstance(targetGR, instanceGR);
}

That completes all of the parts for the registration service. However, we still need to build the function that calls this Host service from the new Client instance. That will be the subject of our next installment.

Collaboration Store, Part XIII

“First, solve the problem. Then, write the code.”
John Johnson

Today we need to build out the Subflow that we referenced in the Script Include function that we built last time out. To create a new Subflow, open up the Flow Designer, click on the New button to bring up the selection list, and select Subflow from the list.

Creating a new Subflow

When the initial form pops up, all you really need to enter is the name of the Subflow, which we already referred to in our Script Include function as New_Collaboration_Store_Instance.

New Subflow properties form

Once you submit the initial properties form, the new Subflow will appear in the list of Subflows, and from there you can bring it up in full edit mode. In edit mode, we can add the one Input to the Subflow, the name of the new instance.

Subflow Inputs and Outputs

Since we will be launching this Subflow to run on its own in the background, there is no need for any Outputs.

Once we configure the Inputs and Outputs, we can move on to the steps of the Subflow. Our first step will be to gather up all of the records in the table of instances except for two: the Host instance, which has already been updated, and the new instance, which already knows about itself.

Database query step

We select the Look Up Records Action, select the Member Organization table, and then define two Conditions to get the records that we want: 1) the host value is false, and 2) the instance is not the instance that we brought in as Input. It really doesn’t matter for our purposes what sequence the records come in, but I went ahead and sorted the records by instance, just so they we will always work through them in the same order.

Our next step, then is to loop through the records. We do that with a For Each Item Flow Control step, setting the items to the records obtained in the first step.

Looping through the retrieved records

Now we have the basic structure of the Subflow; we just need to perform the tasks necessary to notify each host of the new instance, and notify the new instance of each host within the the For Each Item loop. This could all be done with additional Subflow steps, but I took the easy way out here and just built another function in our Script Include to handle the REST API calls to the instances. In fact, I built two functions, one to make the call, and another to call that function twice, once to tell the existing host about the new host, and then again to inform the new host of the existing host. To run that script, I had to create a custom Action, which is just a simple Script Action that calls the function, passing in the names of the two instances (existing, from the current record, and new, from the Subflow input). Once I built the custom Action, I was able to select it from the list and then configure it.

Custom Script Action step

That completes the Subflow, but once again, we have referenced a function in our Script Include that does not exist, so we will have to get into that in our next installment.

Collaboration Store, Part XII

“The slogan ‘press on’ has solved and always will solve the problems of the human race.”
Calvin Coolidge

In the previous installment in this series, we created a new Scripted REST API Resource and referenced another nonexistent function in our Script Include. Now it is time to create that function, which will perform some of the work required to register a new client instance and then hand off the remaining tasks to an asynchronous Subflow so that the function can return the results without waiting for all of the other instances to be notified of the new client instance. The only thing to be done in the function will be to insert the new client instance into the database and kick off the Subflow. But before we do that, we need to first check to make sure that the Client has not already registered with the Host.

var result = {body: {error: {}, status: 'failure'}};

var mbrGR = new GlideRecord('x_11556_col_store_member_organization');
if (mbrGR.get('instance', data.instance)) {
	result.status = 400;
	result.body.error.message = 'Duplicate registration error';
	result.body.error.detail = 'This instance has already been registered with this store.';
} else {
	...

As we did before, we construct our result object with the expectation of failure, since there are more error conditions than the one successful path through the logic. In the case of an instance that has already been registered, we respond with a 400 Bad Request HTTP Response Code and accompanying error details. If the instance is not already in the database, then we attempt to insert it.

mbrGR.initialize();
mbrGR.name = data.name;
mbrGR.instance = data.instance;
mbrGR.email = data.email;
mbrGR.description = data.description;
mbrGR.token = data.sys_id;
mbrGR.active = true;
mbrGR.host = false;
mbrGR.accepted = new GlideDateTime();
if (mbrGR.insert()) {
	result.status = 202;
	delete result.body.error;
	result.body.info = {};
	result.body.info.message = 'Registration complete';
	result.body.info.detail = 'This instance has been successfully registered with this store.';
	result.body.status = 'success';
	...

If the new record was inserted successfully, then we response with a 202 Accepted HTTP Response Code, indicating that the registration was accepted, but the complete registration process (notifying all of the other instances) is not yet complete. At this point, all we have left to do is to initiate the Subflow to handle the rest of the process. We haven’t built the Subflow just yet, but for the purposes of this exercise, we can just assume that it is out there and then we can build it out later. There a couple of different ways to launch an asynchronous Subflow in the background, the original way, and the newer, preferred method. Both methods use the Scripted Flow API. Here is the way that we used to do this:

sn_fd.FlowAPI.startSubflow('New_Collaboration_Store_Instance', {new_instance: data.instance});

… and here is way that ServiceNow would like you to do it now:

sn_fd.FlowAPI.getRunner()
	.subflow('New_Collaboration_Store_Instance')
	.inBackground()
	.withInputs({new_instance: data.instance})
	.run();

Right now, both methods will work, and I’m still using the older, simpler way, but one day I’m going to need to switch over.

There should never be a problem inserting the new record, but just in case, we make that a conditional, and if for some reason it fails, we respond with a 500 Internal Server Error HTTP Response Code.

result.status = 500;
result.body.error.message = 'Internal server error';
result.body.error.detail = 'There was a problem processing this registration request.';

That’s it for all of the little parts and pieces. Here is the entire function, all put together.

processRegistrationRequest: function(data) {
	var result = {body: {error: {}, status: 'failure'}};

	var mbrGR = new GlideRecord('x_11556_col_store_member_organization');
	if (mbrGR.get('instance', data.instance)) {
		result.status = 400;
		result.body.error.message = 'Duplicate registration error';
		result.body.error.detail = 'This instance has already been registered with this store.';
	} else {
		mbrGR.initialize();
		mbrGR.name = data.name;
		mbrGR.instance = data.instance;
		mbrGR.email = data.email;
		mbrGR.description = data.description;
		mbrGR.token = data.sys_id;
		mbrGR.active = true;
		mbrGR.host = false;
		mbrGR.accepted = new GlideDateTime();
		if (mbrGR.insert()) {
			result.status = 202;
			delete result.body.error;
			result.body.info = {};
			result.body.info.message = 'Registration complete';
			result.body.info.detail = 'This instance has been successfully registered with this store.';
			result.body.status = 'success';
			sn_fd.FlowAPI.startSubflow('New_Collaboration_Store_Instance', {new_instance: data.instance});
		} else {
			result.status = 500;
			result.body.error.message = 'Internal server error';
			result.body.error.detail = 'There was a problem processing this registration request.';
		}
	}

	return result;
}

Now we have completed the nonexistent function that was referenced in the REST API Resource, but we have also now referenced a new nonexistent Subflow that we will need to build out before things are complete. That sounds like a good subject for our next installment.

Collaboration Store, Part XI

Never give up on a dream just because of the time it will take to accomplish it. The time will pass anyway.”
Earl Nightingale

We have one more function left in our Script Include that was referenced in the set-process function of our widget that still needs to be created. Last time, we completed the function that creates the Service Account, and now we need to build the one that calls the new instance registration process on the Host instance, which also does not exist as yet, either. In fact, before we build the function that uses the service, we should probably create the service first. This will be another Scripted REST API similar to the one that we already created for the Host info service, but this one will be much more complicated.

When we register a new Client instance with the Host, the following things should occur:

  • The new Client instance should be added to the Host database,
  • The new Client instance data should be pushed to the databases of all of the existing instances, and
  • The new Client instance should be updated with a full list of all of the existing instances.

When this process is complete, all instances, including the new instance, should have a full list of every other instance registered with the Host. However, it seems to me that the registration process itself should not have to wait until all of the instances have been made aware of each other. Once we insert the new instance into the Host database, we should be able to respond to the caller that the registration has been completed, and then pushing the details out to all of the instances could be handled off-line in some other async process. But before we worry too much about all of that, let’s get to creating the service.

We need to create another Scripted REST API Resource, but we can tuck it underneath same the Scripted REST API Service that we already created for our earlier info resource. This one we will call register, which will give it the following URI:

/api/x_11556_col_store/v1/register

This time, we will set the HTTP Method to POST, since we will be accepting input from the client instance in JSON format in the body of the request.

/register Scripted REST API Resource

For our Script, we will push the majority of the code over into yet another new function in our Script Include, and then use the results to populate the response.

(function process(request, response) {
	var data = {};
	if (request.body && request.body.data) {
		data = request.body.data;
	}
	var result = new CollaborationStoreUtils().processRegistrationRequest(data);
	response.setBody(result.body);
	response.setStatus(result.status);
})(request, response);

Unlike our earlier /info service, we will want to require authentication for this service (which is why we created the Service Accounts), so under the Security tab, we will check the Requires authentication checkbox.

/register Scripted REST API Resource Security Tab

Under the Content Negotiation tab, we will set both the Supported request formats and the Supported response formats to application/json, as we will expect the caller to send us a JSON string containing the details of their instance, and we will be responding with a JSON string containing the outcome of the registration process.

/register Scripted REST API Resource Content Negotiation Tab

Once we Save the new resource, it should appear in the Related List at the bottom of the service form.

Our first two Scripted REST API Resources

With that out of the way, now we need to turn our attention to building out the non-existent Script Include function that we referenced in the resource’s script. That’s where all of magic will happen, and quite a lot will go on there, so that seems like a good subject for our next installment.