Collaboration Store, Part XLVII

“Relinquish your attachment to the known, step into the unknown, and you will step into the field of all possibilities.”
Deepak Chopra

While we wait patiently for the testing results of the instance sync process to come trickling in, we should go ahead and get started on the application installation process. As I laid out earlier, there are a couple of different ways to go here, and I really do not like either one. However, as much as I would like to have come up with a preferable third alternative, nothing has really come to mind, despite the fact that I have been dragging out the instance sync process for much longer than I should have while I searched in vain for a better solution. We have to keep pushing forward, though, so enough waiting. Let’s get into it.

I have decided to go with the client-side approach, mainly because I don’t want to get involved with the security issues related to attempting to emulate a user session on the server side. Both approaches involve quite a bit more hackery than I really care to include in something that is supposed to be a viable product to be distributed and used by others, but at this point, I don’t see any other way. The client-side approach involves downloading the Update Set XML file from the server, and then presenting it back as if it were a local file uploaded to the import XML process. So, to start with, we will need a Script Include function that will deliver the XML to the client side. Since this will be a client callable Script Include, we will want to create a new one rather than add another function to our existing utilities. We will call this new Script Include ApplicationInstaller, and check the Client callable checkbox when we create it. Our lone function will be called getXML and it will accept a single parameter for the sys_id of the attachment record, and then use that sys_id to fetch the record and return both the file name and the file contents in a JSON string response.

var ApplicationInstaller = Class.create();
ApplicationInstaller.prototype = Object.extendsObject(global.AbstractAjaxProcessor, {

	getXML: function() {
		var answer = {};
		var sysId = this.getParameter("attachment_id");
		if (sysId) {
			var sysAttGR = new GlideRecord('sys_attachment');
			if (sysAttGR.get(sysId)) {
				var gsa = new GlideSysAttachment();
				answer.fileName = sysAttGR.getDisplayValue('file_name');
				answer.xml = gsa.getContent(sysAttGR);
			}
		}
		return JSON.stringify(answer);
	},

    type: 'ApplicationInstaller'
});

There’s nothing too extraordinary about the code here. We do leverage our old friend the GlideSysAttachment to grab the contents of the file, but that’s nothing that we have not already done before. Other than that, it’s pretty basic stuff.

Now that we have a way to fetch the name and contents of the attached file, we will need a way to launch the process, which will either be a clone of the existing upload.do page or the original upload.do page with a little creative hackery to bend it to our will. I hate to mess with the original, but I also don’t want to create an entire second copy if I don’t have to. But let’s not get too far ahead of ourselves just yet. Right now, we just need to build a launcher for the process, and we can easily change the page that we launch once we decide which way to go.

To launch the installation process, we can add a UI Action to the version form, which we can simply label Install. We will want to make this action conditional, since we would not want to see this option on a version that has already been installed. At this point, we don’t need to worry about the code behind the action; we can just create it and see how it renders out on the page.

New UI Action “Install”

After saving the new action, we can pull up a version page and see how it looks.

New UI Action rendered on the version page

Now we need to add some code to make the action actually do something. Basically, we just want to jump over to whatever version of the upload.do page we end up pursuing, but before we do that we need to include a parameter. The client callable Script Include that we just created takes an attachment record sys_id as a parameter, but out action is sitting on the version record, not the attachment record. We will need to hunt down the attachment record to pull out the sys_id and then we can pass that along to our destination so that it can be used to make the GlideAjax call to our Script Include function. Here is the UI Action script to make all of that happen.

var attachmentGR = new GlideRecord('sys_attachment');
attachmentGR.addQuery('table_name', 'x_11556_col_store_member_application_version');
attachmentGR.addQuery('table_sys_id', current.sys_id);
attachmentGR.addQuery('content_type', 'CONTAINS', 'xml');
attachmentGR.query();
if (attachmentGR.next()) {
	action.setRedirectURL('/upload.do?attachment_id=' + attachmentGR.getUniqueValue());
} else {
	gs.addErrorMessage('No Update Set XML file found attached to this version');
}

Once again, this is pretty basic stuff that doesn’t require a whole lot of explanation. We query the attachment table for an XML file attached to the current record, and if we find it, we construct a URL and jet off to that page. If not, we throw up an error message, but that really shouldn’t happen unless something has gone horribly wrong somewhere along the line.

So now we have built enough parts to get us to the page that will actually fetch the XML from the server and then push it back up again. I need to make a decision as to whether I should hack up the original upload.do page or just use it as a guide to make one of my own. I’ll need to give that some thought, so let’s pause for now and we’ll jump into that next time out.

Collaboration Store, Part XLIV

“I am so clever that sometimes I don’t understand a single word of what I am saying.”
Oscar Wilde

Last time, we completed the shared functions in the CollaborationStoreUtils Script Include that will push the various record types from one instance to another. These functions will work to push data from a Client instance to the Host as well as push data from the Host to any of the Clients. Now that we have those in place, we can get back to the business of syncing up the Client instances with the Host. We have already built the code to sync up the list of instances, as well as the code to sync up the applications for each instance. Now we need to create the code to sync up the versions for each application. We will use the same approach that we used for all of the others, and much of the code will look very similar to that which we have already put into place.

Once again, we will start by gathering up all of the records associated with the active higher-level record, fetching the versions associated with the current application just as we previously gathered up all of the applications associated with the providing instance. Other than the table and query parameters, this should look quite familiar to those of you who have been following along at home.

var versionList = [];
var versionGR = new GlideRecord('x_11556_col_store_member_application_version');
versionGR.addQuery('member_application.name', thisApplication);
versionGR.addQuery('member_application.provider.instance', thisInstance);
versionGR.query();
while (versionGR.next()) {
	versionList.push(versionGR.getDisplayValue('version'));
}

If there are version records present on the Host, then we need to use the REST API to go gather up the same list from the target Client. As usual, we start out by building our RESTMessageV2 request object.

var request  = new sn_ws.RESTMessageV2();
request.setHttpMethod('get');
request.setBasicAuth(this.CSU.WORKER_ROOT + targetGR.getDisplayValue('instance'), targetGR.getDisplayValue('token'));
request.setRequestHeader("Accept", "application/json");
request.setEndpoint('https://' + targetGR.getDisplayValue('instance') + '.service-now.com/api/now/table/x_11556_col_store_member_application_version?sysparm_fields=version%2Csys_id&sysparm_query=member_application%3D' + remoteAppId);

Once we have the request constructed, we execute it and check the response.

var response = request.execute();
if (response.haveError()) {
	gs.error('InstanceSyncUtils.syncVersions - Error returned from attempt to fetch version list: ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
} else if (response.getStatusCode() == '200') {
	var jsonString = response.getBody();
	var jsonObject = {};
	try {
		jsonObject = JSON.parse(jsonString);
	} catch (e) {
		gs.error('InstanceSyncUtils.syncVersions - Unparsable JSON string returned from attempt to fetch version list: ' + jsonString);
	}
	if (Array.isArray(jsonObject.result)) {
		...
	} else {
		gs.error('InstanceSyncUtils.syncVersions - Invalid response body returned from attempt to fetch version list: ' + response.getBody());
	}
} else {
	gs.error('InstanceSyncUtils.syncVersions - Invalid HTTP response code returned from attempt to fetch version list: ' + response.getStatusCode());
}

Assuming all went well, the last thing that we need to do is to loop through all of the versions present on the Host and see if they are also present on the Client. If not, then we need to send them over, which we can now do with our new shared functions in CollaborationStoreUtils. And of course, whether we had to push the version over or not, we will need to check to see if the version record has the attached Update Set XML file.

for (var i=0; i<versionList.length; i++) {
	var thisVersion = versionList[i];
	var remoteVerId = '';
	for (var j=0; j<jsonObject.result.length && remoteVerId == ''; j++) {
		if (jsonObject.result[j].version == thisVersion) {
			remoteVerId = jsonObject.result[j].sys_id;
		}
	}
	if (remoteVerId == '') {
		remoteVerId = this.sendVersion(targetGR, thisApplication, thisInstance, thisVersion, remoteAppId);
	}
	this.syncAttachments(targetGR, thisVersion, remoteVerId);
}

Putting it all together, the entire function looks like this:

syncVersions: function(targetGR, thisApplication, thisInstance, remoteAppId) {
	var versionList = [];
	var versionGR = new GlideRecord('x_11556_col_store_member_application_version');
	versionGR.addQuery('member_application.name', thisApplication);
	versionGR.addQuery('member_application.provider.instance', thisInstance);
	versionGR.query();
	while (versionGR.next()) {
		versionList.push(versionGR.getDisplayValue('version'));
	}
	if (versionList.length > 0) {
		var request  = new sn_ws.RESTMessageV2();
		request.setHttpMethod('get');
		request.setBasicAuth(this.CSU.WORKER_ROOT + targetGR.getDisplayValue('instance'), targetGR.getDisplayValue('token'));
		request.setRequestHeader("Accept", "application/json");
		request.setEndpoint('https://' + targetGR.getDisplayValue('instance') + '.service-now.com/api/now/table/x_11556_col_store_member_application_version?sysparm_fields=version%2Csys_id&sysparm_query=member_application%3D' + remoteAppId);
		var response = request.execute();
		if (response.haveError()) {
			gs.error('InstanceSyncUtils.syncVersions - Error returned from attempt to fetch version list: ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
		} else if (response.getStatusCode() == '200') {
			var jsonString = response.getBody();
			var jsonObject = {};
			try {
				jsonObject = JSON.parse(jsonString);
			} catch (e) {
				gs.error('InstanceSyncUtils.syncVersions - Unparsable JSON string returned from attempt to fetch version list: ' + jsonString);
			}
			if (Array.isArray(jsonObject.result)) {
				for (var i=0; i<versionList.length; i++) {
					var thisVersion = versionList[i];
					var remoteVerId = '';
					for (var j=0; j<jsonObject.result.length && remoteVerId == ''; j++) {
						if (jsonObject.result[j].version == thisVersion) {
							remoteVerId = jsonObject.result[j].sys_id;
						}
					}
					if (remoteVerId == '') {
						remoteVerId = this.sendVersion(targetGR, thisApplication, thisInstance, thisVersion, remoteAppId);
					}
					this.syncAttachments(targetGR, thisVersion, remoteVerId);
				}
			} else {
				gs.error('InstanceSyncUtils.syncVersions - Invalid response body returned from attempt to fetch version list: ' + response.getBody());
			}
		} else {
			gs.error('InstanceSyncUtils.syncVersions - Invalid HTTP response code returned from attempt to fetch version list: ' + response.getStatusCode());
		}
	} else {
		gs.info('InstanceSyncUtils.syncVersions - No versions to sync for application ' + thisApplication + ' on instance ' + thisInstance);
	}
}

Now we have referenced two new functions that will need to built out as well, sendVersion and syncAttachments. The sendVersion function will be pretty much a clone of the sendApplication function that we already built, and it will call our new pushVersion function in CollaborationStoreUtils.

sendVersion: function(targetGR, thisApplication, thisInstance, thisVersion, remoteAppId) {
	var sysId = '';

	var versionGR = new GlideRecord('x_11556_col_store_member_application_version');
	versionGR.addQuery('member_application.name', thisApplication);
	versionGR.addQuery('member_application.provider.instance', thisInstance);
	versionGR.addQuery('version', thisVersion);
	versionGR.query();
	if (versionGR.next()) {
		gs.info('InstanceSyncUtils.sendApplication - Sending version ' + thisVersion + ' of application ' + versionGR.getDisplayValue('member_application') + ' to target instance ' + targetGR.getDisplayValue('instance'));
		var result = this.CSU.pushVersion(versionGR, targetGR, remoteAppId);
		if (result.error) {
			gs.error('InstanceSyncUtils.sendVersion - Error occurred attempting to push version ' + thisVersion + ' of application ' + remoteAppId + ' : ' + result.errorCode + '; ' + result.errorMessage);
		} else if (result.status != 200 && result.status != 201) {
			gs.error('InstanceSyncUtils.sendVersion - Invalid HTTP response code returned from attempt to push version ' + thisVersion + ' of application ' + remoteAppId + ' : ' + result.status);
		} else if (!result.obj) {
			gs.error('InstanceSyncUtils.sendVersion - Invalid response body returned from attempt to push version ' + thisVersion + ' of application ' + remoteAppId + ' : ' + result.body);
		} else {
			sysId = result.obj.sys_id;
		}
	}

	return sysId;
}

The other missing referenced function is syncAttachments. This one will be similar to the syncVersions function above, but because we will be working with attached files, there will be some minor differences in the tools that we will use to make all of that happen. We will get into all of that next time out.

Collaboration Store, Part XLIII

“You don’t start out writing good stuff. You start out writing crap and thinking it’s good stuff, and then gradually you get better at it. That’s why I say one of the most valuable traits is persistence.”
Octavia E. Butler

Last time, we built a function to send over an application record modeled after a similar function that we already had to send over and instance record. We can do the same thing now for a version record as well as the Update Set XML attachment. Since those are always inserts, we can skip the extra step of looking to see if the record is already there, and these functions will end looking an awful lot like the original function that sent over an instance record. Here is a completed function to send over a version record.

pushVersion: function(versionGR, targetGR, remoteAppId) {
	var result = {};

	var payload = {};
	payload.member_application = remoteAppId;
	payload.version = versionGR.getDisplayValue('version');
	payload.built_on = versionGR.getDisplayValue('built_on');
	result.url = 'https://' + targetGR.getDisplayValue('instance') + '.service-now.com/api/now/table/x_11556_col_store_member_application_version';
	result.method = 'POST';
	var request = new sn_ws.RESTMessageV2();
	request.setEndpoint(result.url);
	request.setHttpMethod(result.method);
	request.setBasicAuth(this.WORKER_ROOT + targetGR.getDisplayValue('instance'), targetGR.getDisplayValue('token'));
	request.setRequestHeader('Content-Type', 'application/json');
	request.setRequestHeader('Accept', 'application/json');
	request.setRequestBody(JSON.stringify(payload, null, '\t'));
	var response = request.execute();
	result.status = response.getStatusCode();
	result.body = response.getBody();
	if (result.body) {
		try {
			result.obj = JSON.parse(result.body);
		} catch (e) {
			result.parse_error = e.toString();
		}
	}
	result.error = response.haveError();
	if (result.error) {
		result.error_code = response.getErrorCode();
		result.error_message = response.getErrorMessage();
	}

	return result;
}

And here is the completed function for sending over the Update Set XML file that is attached to the version.

pushAttachment: function(attachmentGR, targetGR, remoteVerId) {
	var result = {};

	var gsa = new GlideSysAttachment();
	result.url = 'https://';
	result.url += targetGR.getDisplayValue('instance');
	result.url += '.service-now.com/api/now/attachment/file?table_name=x_11556_col_store_member_application_version&table_sys_id=';
	result.url += remoteVerId;
	result.url += '&file_name=';
	result.url += attachmentGR.getDisplayValue('file_name');
	result.method = 'POST';
	var request = new sn_ws.RESTMessageV2();
	request.setEndpoint(result.url);
	request.setHttpMethod(result.method);
	request.setBasicAuth(this.WORKER_ROOT + targetGR.getDisplayValue('instance'), targetGR.getDisplayValue('token'));
	request.setRequestHeader('Content-Type', attachmentGR.getDisplayValue('content_type'));
	request.setRequestHeader('Accept', 'application/json');
	request.setRequestBody(gsa.getContent(sysAttGR));
	var response = request.execute();
	result.status = response.getStatusCode();
	result.body = response.getBody();
	if (result.body) {
		try {
			result.obj = JSON.parse(result.body);
		} catch (e) {
			result.parse_error = e.toString();
		}
	}
	result.error = response.haveError();
	if (result.error) {
		result.error_code = response.getErrorCode();
		result.error_message = response.getErrorMessage();
	}

	return result;
}

All together, the four complementary functions in the CollaborationStoreUtils now look like this.

pushInstance: function(instanceGR, targetGR) {
	var result = {};

	var payload = {};
	payload.instance = instanceGR.getDisplayValue('instance');
	payload.name = instanceGR.getDisplayValue('name');
	payload.description = instanceGR.getDisplayValue('description');
	payload.email = instanceGR.getDisplayValue('email');
	payload.accepted = instanceGR.getDisplayValue('accepted');
	payload.active = instanceGR.getDisplayValue('active');
	payload.host = instanceGR.getDisplayValue('host');
	result.url = 'https://' + targetGR.getDisplayValue('instance') + '.service-now.com/api/now/table/x_11556_col_store_member_organization';
	result.method = 'POST';
	var request = new sn_ws.RESTMessageV2();
	request.setEndpoint(result.url);
	request.setHttpMethod(result.method);
	request.setBasicAuth(this.WORKER_ROOT + targetGR.getDisplayValue('instance'), targetGR.getDisplayValue('token'));
	request.setRequestHeader('Content-Type', 'application/json');
	request.setRequestHeader('Accept', 'application/json');
	request.setRequestBody(JSON.stringify(payload, null, '\t'));
	var response = request.execute();
	result.status = response.getStatusCode();
	result.body = response.getBody();
	if (result.body) {
		try {
			result.obj = JSON.parse(result.body);
		} catch (e) {
			result.parse_error = e.toString();
		}
	}
	result.error = response.haveError();
	if (result.error) {
		result.error_code = response.getErrorCode();
		result.error_message = response.getErrorMessage();
	}

	return result;
},

pushApplication: function(applicationGR, targetGR, remoteSysId) {
	var result = {};
 
	result.url = 'https://' + targetGR.getDisplayValue('instance') + '.service-now.com/api/now/table/x_11556_col_store_member_application?sysparm_fields=sys_id&sysparm_query=provider.instance%3D' + applicationGR.getDisplayValue('provider.instance') + '%5Ename%3D' + encodeURIComponent(applicationGR.getDisplayValue('name'));
	result.method = 'GET';
	var request = new sn_ws.RESTMessageV2();
	request.setEndpoint(result.url);
	request.setHttpMethod(result.method);
	request.setBasicAuth(this.WORKER_ROOT + targetGR.getDisplayValue('instance'), targetGR.getDisplayValue('token'));
	request.setRequestHeader('Content-Type', 'application/json');
	request.setRequestHeader('Accept', 'application/json');
	var response = request.execute();
	result.status = response.getStatusCode();
	result.body = response.getBody();
	if (result.body) {
		try {
			result.obj = JSON.parse(result.body);
		} catch (e) {
			result.parse_error = e.toString();
		}
	}
	result.error = response.haveError();
	if (result.error) {
		result.error_code = response.getErrorCode();
		result.error_message = response.getErrorMessage();
	}
	if (!result.error && !result.parse_error && result.status == 200) {
		var remoteAppId = '';
		if (result.obj.result && result.obj.result.length > 0) {
			remoteAppId = result.obj.result[0].sys_id;
		}
		result = {};
		var payload = {};
		payload.name = applicationGR.getDisplayValue('name');
		payload.description = applicationGR.getDisplayValue('description');
		payload.current_version = applicationGR.getDisplayValue('current_version');
		payload.active = 'true';
		if (remoteAppId > '') {
			result.url = 'https://' + targetGR.getDisplayValue('instance') + '.service-now.com/api/now/table/x_11556_col_store_member_application/' + remoteAppId;
			result.method = 'PUT';
			result.remoteAppId = remoteAppId;
		} else {
			result.url = 'https://' + targetGR.getDisplayValue('instance') + '.service-now.com/api/now/table/x_11556_col_store_member_application';
			result.method = 'POST';
			payload.provider = remoteSysId;
		}
		request  = new sn_ws.RESTMessageV2();
		request.setEndpoint(result.url);
		request.setHttpMethod(result.method);
		request.setBasicAuth(this.WORKER_ROOT + targetGR.getDisplayValue('instance'), targetGR.getDisplayValue('token'));
		request.setRequestHeader('Content-Type', 'application/json');
		request.setRequestHeader('Accept', 'application/json');
		request.setRequestBody(JSON.stringify(payload, null, '\t'));
		response = request.execute();
		result.status = response.getStatusCode();
		result.body = response.getBody();
		if (result.body) {
			try {
				result.obj = JSON.parse(result.body);
			} catch (e) {
				result.parse_error = e.toString();
			}
		}
		result.error = response.haveError();
		if (result.error) {
			result.error_code = response.getErrorCode();
			result.error_message = response.getErrorMessage();
		}
	}
 
	return result;
},

pushVersion: function(versionGR, targetGR, remoteAppId) {
	var result = {};

	var payload = {};
	payload.member_application = remoteAppId;
	payload.version = versionGR.getDisplayValue('version');
	payload.built_on = versionGR.getDisplayValue('built_on');
	result.url = 'https://' + targetGR.getDisplayValue('instance') + '.service-now.com/api/now/table/x_11556_col_store_member_application_version';
	result.method = 'POST';
	var request = new sn_ws.RESTMessageV2();
	request.setEndpoint(result.url);
	request.setHttpMethod(result.method);
	request.setBasicAuth(this.WORKER_ROOT + targetGR.getDisplayValue('instance'), targetGR.getDisplayValue('token'));
	request.setRequestHeader('Content-Type', 'application/json');
	request.setRequestHeader('Accept', 'application/json');
	request.setRequestBody(JSON.stringify(payload, null, '\t'));
	var response = request.execute();
	result.status = response.getStatusCode();
	result.body = response.getBody();
	if (result.body) {
		try {
			result.obj = JSON.parse(result.body);
		} catch (e) {
			result.parse_error = e.toString();
		}
	}
	result.error = response.haveError();
	if (result.error) {
		result.error_code = response.getErrorCode();
		result.error_message = response.getErrorMessage();
	}

	return result;
},

pushAttachment: function(attachmentGR, targetGR, remoteVerId) {
	var result = {};

	var gsa = new GlideSysAttachment();
	result.url = 'https://';
	result.url += targetGR.getDisplayValue('instance');
	result.url += '.service-now.com/api/now/attachment/file?table_name=x_11556_col_store_member_application_version&table_sys_id=';
	result.url += remoteVerId;
	result.url += '&file_name=';
	result.url += attachmentGR.getDisplayValue('file_name');
	result.method = 'POST';
	var request = new sn_ws.RESTMessageV2();
	request.setEndpoint(result.url);
	request.setHttpMethod(result.method);
	request.setBasicAuth(this.WORKER_ROOT + targetGR.getDisplayValue('instance'), targetGR.getDisplayValue('token'));
	request.setRequestHeader('Content-Type', attachmentGR.getDisplayValue('content_type'));
	request.setRequestHeader('Accept', 'application/json');
	request.setRequestBody(gsa.getContent(sysAttGR));
	var response = request.execute();
	result.status = response.getStatusCode();
	result.body = response.getBody();
	if (result.body) {
		try {
			result.obj = JSON.parse(result.body);
		} catch (e) {
			result.parse_error = e.toString();
		}
	}
	result.error = response.haveError();
	if (result.error) {
		result.error_code = response.getErrorCode();
		result.error_message = response.getErrorMessage();
	}

	return result;
}

Looking at them all side by side now, it appears that we could consolidate them even further by creating a single function that accepted a URL and a method, and maybe a payload, as that seems to be the only difference between them. I’m not really up for that at the moment, but it looks as if we could actually make that work one day when we had nothing better to do. For now, though, let’s just leave things as they are and get back to our InstanceSyncUtils and finish building out the code that will sync the version and attachment records in the same way that we did for the instances and the applications. These will utilize our new push functions when needed, but we still need to build out the logic that does the compare and determines what, if anything, needs to be sent over.

As you may recall, our strategy is to loop through the instances, and for each instance, loop through the applications provided by that instance. Similarly, for each application, we will loop through each version of that application, and for each version, we will make sure that there is an associated Update Set XML file attached. Whenever we find anything missing, we will utilize the appropriate push function above to send over the missing artifact. Since we have already done that for instances and applications, it should be relatively straightforward to copy that existing code and modify it as needed for the different types of records involved. Still, it is a little bit of work, so let’s save that for our next installment.

Collaboration Store, Part XLII

“You don’t get results by focusing on results. You get results by focusing on the actions that produce results.”
Mike Hawkins

Last time out, we threw together the code that will sync up the list of apps for a particular instance on the Host with the list of apps for that instance on the remote Client. We stopped short of building out the code that will send the missing applications over, though, and today we need to wrap that up. Those of you following along at home will recall that we already have a couple of functions that do that, and we really don’t want to create yet another one, so we need to see if we can somehow collapse those two into one that can serve the needs of all three use cases. The basic premise on the application updates is to first look and see if the application is already present on the target instance, and if so, update it; otherwise, insert it. Both of our existing functions take that approach, so we will want to maintain that functionality in the new shared version.

The other thing that we wanted to do was to model the existing publishApplication function’s approach to handling the interactions between the instance making the call and the instance that is being called. Basically, the shared code just makes the calls and returns the results in an object that can be evaluated by the caller. The shared code takes no action on any errors that might be encountered; it simply reports back to the calling module everything that happened. It is up to the caller to take whatever action might be necessary to respond to any issues reported. We will want to maintain that approach in our new function as well.

So, just as with the existing publishApplication function, we start out by creating the result object that we will be returning to the caller.

pushApplication: function(applicationGR, targetGR, remoteSysId) {
	var result = {};
 
	...
 
	return result;
}

Something that was not in the original version of the publishApplication function was the insertion of the URL and the method used in the result object. It occurred to me that this might be useful information to the caller, so I decided to add it into this one, and I think I will go back and add that into the original as well. For fetching the applications, which is the next thing that we need to do, that looks like this:

result.url = 'https://' + targetGR.getDisplayValue('instance') + '.service-now.com/api/now/table/x_11556_col_store_member_application?sysparm_fields=sys_id&sysparm_query=provider.instance%3D' + applicationGR.getDisplayValue('provider.instance') + '%5Ename%3D' + encodeURIComponent(applicationGR.getDisplayValue('name'));
result.method = 'GET';

Once we have set the value of those two properties in the result object, then we can use them to build our request object.

var request = new sn_ws.RESTMessageV2();
request.setEndpoint(result.url);
request.setHttpMethod(result.method);

And before we pull the trigger, we also have to set up the standard authentication and content headers.

request.setBasicAuth(this.WORKER_ROOT + targetGR.getDisplayValue('instance'), targetGR.getDisplayValue('token'));
request.setRequestHeader('Content-Type', 'application/json');
request.setRequestHeader('Accept', 'application/json');

Once our request object is fully configured, then we just need to pull the trigger and complete our result object with the data and/or errors returned.

var response = request.execute();
result.status = response.getStatusCode();
result.body = response.getBody();
if (result.body) {
	try {
		result.obj = JSON.parse(result.body);
	} catch (e) {
		result.parse_error = e.toString();
	}
}
result.error = response.haveError();
if (result.error) {
	result.error_code = response.getErrorCode();
	result.error_message = response.getErrorMessage();
}

At this point, if there was any kind of error, then we just want to return that back to the caller so that they can take whatever action is appropriate for that particular context. Assuming everything worked as it should, though, the next thing that we are going to want to do is to see if the application was returned in the array of JSON objects sent back in response, and if so, snag the sys_id from the data.

if (!result.error && !result.parse_error && result.status == 200) {
	var remoteAppId = '';
	if (result.obj.result && result.obj.result.length > 0) {
		remoteAppId = result.obj.result[0].sys_id;
	}
	...
}

At this point, we are about to make an entirely new request of the target instance, so we want to reset the result object back to an empty object and start the building process all over again from scratch. The fetching of the application was successful (even if the application was not found on the target instance, finding that out was a success), so we do not need to retain anything from that activity other than the application’s sys_id on the target instance, if it happens to be there. Once we reset the result object, we can start building our payload to send over from the data on the application record.

result = {};
var payload = {};
payload.name = applicationGR.getDisplayValue('name');
payload.description = applicationGR.getDisplayValue('description');
payload.current_version = applicationGR.getDisplayValue('current_version');
payload.active = 'true';

At this point, we need to determine if we are doing an insert or an update based on the presence of a remote sys_id for this application.

if (remoteAppId > '') {
	result.url = 'https://' + targetGR.getDisplayValue('instance') + '.service-now.com/api/now/table/x_11556_col_store_member_application/' + remoteAppId;
	result.method = 'PUT';
	result.remoteAppId = remoteAppId;
} else {
	result.url = 'https://' + targetGR.getDisplayValue('instance') + '.service-now.com/api/now/table/x_11556_col_store_member_application';
	result.method = 'POST';
	payload.provider = remoteSysId;
}

Now we build our new request object, once again using the result.url and result.method along with all of the other standard elements of our other requests. Also, since this request will be shipping data over to the target instance, we will need to set the request body with the stringified payload.

request  = new sn_ws.RESTMessageV2();
request.setEndpoint(result.url);
request.setHttpMethod(result.method);
request.setBasicAuth(this.WORKER_ROOT + targetGR.getDisplayValue('instance'), targetGR.getDisplayValue('token'));
request.setRequestHeader('Content-Type', 'application/json');
request.setRequestHeader('Accept', 'application/json');
request.setRequestBody(JSON.stringify(payload, null, '\t'));

Then, as before, we need to execute the request and populate our result object with the response.

response = request.execute();
result.status = response.getStatusCode();
result.body = response.getBody();
if (result.body) {
	try {
		result.obj = JSON.parse(result.body);
	} catch (e) {
		result.parse_error = e.toString();
	}
}
result.error = response.haveError();
if (result.error) {
	result.error_code = response.getErrorCode();
	result.error_message = response.getErrorMessage();
}

Once again, we do not take any action in response to any of these potential errors; we simply report them back to the caller in the result object and let them decide what, if anything, they want to do about it. These functions are not built to take action directly. They simply Observe and Report.

All together, this new function looks like this:

pushApplication: function(applicationGR, targetGR, remoteSysId) {
	var result = {};
 
	result.url = 'https://' + targetGR.getDisplayValue('instance') + '.service-now.com/api/now/table/x_11556_col_store_member_application?sysparm_fields=sys_id&sysparm_query=provider.instance%3D' + applicationGR.getDisplayValue('provider.instance') + '%5Ename%3D' + encodeURIComponent(applicationGR.getDisplayValue('name'));
	result.method = 'GET';
	var request = new sn_ws.RESTMessageV2();
	request.setEndpoint(result.url);
	request.setHttpMethod(result.method);
	request.setBasicAuth(this.WORKER_ROOT + targetGR.getDisplayValue('instance'), targetGR.getDisplayValue('token'));
	request.setRequestHeader('Content-Type', 'application/json');
	request.setRequestHeader('Accept', 'application/json');
	var response = request.execute();
	result.status = response.getStatusCode();
	result.body = response.getBody();
	if (result.body) {
		try {
			result.obj = JSON.parse(result.body);
		} catch (e) {
			result.parse_error = e.toString();
		}
	}
	result.error = response.haveError();
	if (result.error) {
		result.error_code = response.getErrorCode();
		result.error_message = response.getErrorMessage();
	}
	if (!result.error && !result.parse_error && result.status == 200) {
		var remoteAppId = '';
		if (result.obj.result && result.obj.result.length > 0) {
			remoteAppId = result.obj.result[0].sys_id;
		}
		result = {};
		var payload = {};
		payload.name = applicationGR.getDisplayValue('name');
		payload.description = applicationGR.getDisplayValue('description');
		payload.current_version = applicationGR.getDisplayValue('current_version');
		payload.active = 'true';
		if (remoteAppId > '') {
			result.url = 'https://' + targetGR.getDisplayValue('instance') + '.service-now.com/api/now/table/x_11556_col_store_member_application/' + remoteAppId;
			result.method = 'PUT';
			result.remoteAppId = remoteAppId;
		} else {
			result.url = 'https://' + targetGR.getDisplayValue('instance') + '.service-now.com/api/now/table/x_11556_col_store_member_application';
			result.method = 'POST';
			payload.provider = remoteSysId;
		}
		request  = new sn_ws.RESTMessageV2();
		request.setEndpoint(result.url);
		request.setHttpMethod(result.method);
		request.setBasicAuth(this.WORKER_ROOT + targetGR.getDisplayValue('instance'), targetGR.getDisplayValue('token'));
		request.setRequestHeader('Content-Type', 'application/json');
		request.setRequestHeader('Accept', 'application/json');
		request.setRequestBody(JSON.stringify(payload, null, '\t'));
		response = request.execute();
		result.status = response.getStatusCode();
		result.body = response.getBody();
		if (result.body) {
			try {
				result.obj = JSON.parse(result.body);
			} catch (e) {
				result.parse_error = e.toString();
			}
		}
		result.error = response.haveError();
		if (result.error) {
			result.error_code = response.getErrorCode();
			result.error_message = response.getErrorMessage();
		}
	}
 
	return result;
}

This should work now for our purpose, but if we want to also utilize this new function for the other two original purposes, we will need to refactor that code to call this function and then evaluate the response. Those actually work as they are right at the moment, so there is no rush to jump in and do that right now, but we don’t want to forget to take care of that one day, if for no other reason than to reduce the lines of code to be maintained.

That about wraps things up for the application records. Next time, we will see if we can do the same thing for the version records, and then we will jump into the Update Set attachments and that ought to wrap up this little side trip of creative avoidance that we took on so that we could ignore the issues with the application publishing for a while. One day, we need to get back into building out that third primary leg of this little stool, but now that we have headed down this intentional detour, we should finish this up first before we jump back onto the main road.

Collaboration Store, Part XLI

“If opportunity doesn’t knock, build a door.”
Milton Berle

Last time out we started work on the Script Include function that will sync up the data on a Client instance with that of the Host instance. We stopped short of consolidating a couple of cloned functions into a single shared function, but before we get to that effort, we need to add the code that syncs up the application table data. We already wrote the code that syncs up the instance table data, and since this operation is essentially the same process, we should be able lift most of the code from what we have already done.

As we did with the instances, we will need to gather up all of the applications on the Host associated with the instance being processed. This is basically the same process that we went through with the instance table, and the code will be very much the same except for the table and the query.

var applicationList = [];
var applicationGR = new GlideRecord('x_11556_col_store_member_application');
applicationGR.addQuery('provider.name', thisInstance);
applicationGR.query();
while (applicationGR.next()) {
	applicationList.push(applicationGR.getDisplayValue('name'));
}

We will also need to contact the Client instance and fetch all of the applications associated with the current instance being processed that are present on that system. Once again, we can turn to the REST API Explorer to generate an end point URL, and other than the actual URL, the request object needed for fetching the applications is again basically the same as that which we put together to fetch the instances from the Client.

var request  = new sn_ws.RESTMessageV2();
request.setHttpMethod('get');
request.setBasicAuth(this.CSU.WORKER_ROOT + instance, token);
request.setRequestHeader("Accept", "application/json");
request.setEndpoint('https://' + instance + '.service-now.com/api/now/table/x_11556_col_store_member_application?sysparm_fields=name%2Csys_id&sysparm_query=provider%3D' + remoteSysId);

Now that we have build our request object, we need to execute the request and check the results just as we did when gathered up the Client’s list of instances.

var response = request.execute();
if (response.haveError()) {
	gs.error('InstanceSyncUtils.syncApplications - Error returned from attempt to fetch application list: ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
} else if (response.getStatusCode() == '200') {
	var jsonString = response.getBody();
	var jsonObject = {};
	try {
		jsonObject = JSON.parse(jsonString);
	} catch (e) {
		gs.error('InstanceSyncUtils.syncApplications - Unparsable JSON string returned from attempt to fetch application list: ' + jsonString);
	}
	if (Array.isArray(jsonObject.result)) {
		...
	} else {
		gs.error('InstanceSyncUtils.syncApplications - Invalid response body returned from attempt to fetch application list: ' + response.getBody());
	}
} else {
	gs.error('InstanceSyncUtils.syncApplications - Invalid HTTP response code returned from attempt to fetch application list: ' + response.getStatusCode());
}

Assuming that all went well with gathering up all of the data on the Host and the Client, the next thing that we need to do is loop through all of the applications found on the Host for this instance and see if they are present on the Client as well. If not, we will need to push them over, which is where we will get into that code that we cloned that needs to be refactored into a single function so that we can have one process that can serve the two original purposes and our new purpose as well.

for (var i=0; i<applicationList.length; i++) {
	var thisApplication = applicationList[i];
	var remoteAppId = '';
	for (var j=0; j<jsonObject.result.length && remoteAppId == ''; j++) {
		if (jsonObject.result[j].name == thisApplication) {
			remoteAppId = jsonObject.result[j].sys_id;
		}
	}
	if (remoteAppId == '') {
		remoteAppId = this.sendApplication(targetGR, thisApplication, thisInstance, remoteSysId);
	}
	this.syncVersions(targetGR, thisApplication, remoteAppId);
}

This code references two functions that we have not yet created, sendApplication and syncVersions. We should be able to cut and paste the sendInstance function that we created earlier to build the sendApplication function with just a few minor modifications. Similarly, we should be able to clone the syncApplications function to serve as the initial basis of the syncVersions function. At this point, we are assuming that we have collapsed our two cloned functions that send applications over into a single function in the CollaborationStoreUtils Script Include modeled after the publishInstance function that we were using to send over an instance record, and we are assuming that it will return the very same type of object in response.

sendApplication: function(targetGR, thisApplication, thisInstance, remoteSysId) {
	var sysId = '';
	var applicationGR = new GlideRecord('x_11556_col_store_member_application');
	applicationGR.addQuery('name', thisApplication);
	applicationGR.addQuery('provider.name', thisInstance);
	applicationGR.query();
	if (applicationGR.next()) {
		var result = this.CSU.publishApplication(applicationGR, targetGR, remoteSysId);
		if (result.error) {
			gs.error('InstanceSyncUtils.sendApplication - Error occurred attempting to push application ' + thisApplication + ' : ' + result.errorCode + '; ' + result.errorMessage);
		} else if (result.status != 200) {
			gs.error('InstanceSyncUtils.sendApplication - Invalid HTTP response code returned from attempt to push application ' + thisApplication + ' : ' + result.status);
		} else if (!result.obj) {
			gs.error('InstanceSyncUtils.sendApplication - Invalid response body returned from attempt to push application ' + thisApplication + ' : ' + result.body);
		} else {
			sysId = result.obj.sys_id;
		}
	}
	return sysId;
}

Of course, for this to work, we actually have to create a function in CollaborationStoreUtils called publishApplication, but we already have two functions that do that very thing and a publishInstance function to use as a model. It shouldn’t take all that much to collapse all of that together into a single function that will work for everyone, so let’s make that the focus of our next installment.

Collaboration Store, Part XL

“If one does not know to which port one is sailing, no wind is favorable.”
Lucius Annaeus Seneca

Last time, we started building out the Script Include function that will do all of the heavy lifting for the process of periodically syncing up all of the Client instances with the Host. The next piece of code that we were going to tackle was the process of sending over a missing instance record to the Client. At the time, I was thinking that I had a couple of different versions of the code to do that already built out, one being cloned off of the other, but as it turns out, that is true for the application records, the version records, and the version attachments, but not the instance records. Currently, there is only one function that pushes over an instance record, and that function is called publishInstance and is found in the CollaborationStoreUtils Script Include.

publishInstance: function(instanceGR, targetGR) {
	var result = {};

	var payload = {};
	payload.instance = instanceGR.getDisplayValue('instance');
	payload.name = instanceGR.getDisplayValue('name');
	payload.description = instanceGR.getDisplayValue('description');
	payload.email = instanceGR.getDisplayValue('email');
	payload.accepted = instanceGR.getDisplayValue('accepted');
	payload.active = instanceGR.getDisplayValue('active');
	payload.host = instanceGR.getDisplayValue('host');
	var url = 'https://' + targetGR.getDisplayValue('instance') + '.service-now.com/api/now/table/x_11556_col_store_member_organization';
	var request = new sn_ws.RESTMessageV2();
	request.setEndpoint(url);
	request.setHttpMethod('POST');
	request.setBasicAuth(this.WORKER_ROOT + targetGR.getDisplayValue('instance'), targetGR.getDisplayValue('token'));
	request.setRequestHeader('Content-Type', 'application/json');
	request.setRequestHeader('Accept', 'application/json');
	request.setRequestBody(JSON.stringify(payload, null, '\t'));
	var response = request.execute();
	result.status = response.getStatusCode();
	result.body = response.getBody();
	if (result.body) {
		try {
			result.obj = JSON.parse(result.body);
		} catch (e) {
			result.parse_error = e.toString();
		}
	}
	result.error = response.haveError();
	if (result.error) {
		result.error_code = response.getErrorCode();
		result.error_message = response.getErrorMessage();
	}

	return result;
}

It takes two arguments, the instanceGR and the targetGR. In the current version of our new code, we do not have a GlideRecord for either the instance to be sent over or the instance to which the record is to be sent, but with a little refactoring, we can make that happen. To start with, I changed this code:

var instance = '';
var token = '';
var instanceList = [];
instanceGR = new GlideRecord('x_11556_col_store_member_organization');
instanceGR.addQuery('active', true);
instanceGR.query();
while (instanceGR.next()) {
    instanceList.push(instanceGR.getDisplayValue('name'));
    if (instanceGR.getUniqueValue() == instanceId) {
        instance = instanceGR.getDisplayValue('name');
        token = instanceGR.getDisplayValue('token');
    }
}

… to this:

var targetGR = new GlideRecord('x_11556_col_store_member_organization');
targetGR.get(instanceId);
var instance = targetGR.getDisplayValue('name');
var token = targetGR.getDisplayValue('token');
var instanceList = [];
var instanceGR = new GlideRecord('x_11556_col_store_member_organization');
instanceGR.addQuery('active', true);
instanceGR.query();
while (instanceGR.next()) {
	instanceList.push(instanceGR.getDisplayValue('name'));
}

This still gets me the target instance name and token as it did earlier, but it also gets me a GlideRecord for the target instance as well.

Then I changed this line:

remoteSysId = this.sendInstance(instance, token, thisInstance);

… to this:

remoteSysId = this.sendInstance(targetGR, thisInstance);

That took care of the refactoring. Now all I needed to do was to fetch a GlideRecord for the instance to be sent over, and then I could call the existing function in CollaborationStoreUtils and check the response.

sendInstance: function(targetGR, thisInstance) {
	var sysId = '';
	var instanceGR = new GlideRecord('x_11556_col_store_member_organization');
	if (instanceGR.get('name', thisInstance)) {
		var result = this.CSU.publishInstance(instanceGR, targetGR);
		if (result.error) {
			gs.error('InstanceSyncUtils.sendInstance - Error occurred attempting to push instance ' + thisInstance + ' : ' + result.errorCode + '; ' + result.errorMessage);
		} else if (result.status != 200) {
			gs.error('InstanceSyncUtils.sendInstance - Invalid HTTP response code returned from attempt to push instance ' + thisInstance + ' : ' + result.status);
		} else if (!result.obj) {
			gs.error('InstanceSyncUtils.sendInstance - Invalid response body returned from attempt to push instance ' + thisInstance + ' : ' + result.body);
		} else {
			sysId = result.obj.sys_id;
		}
	}
	return sysId;
}

Of course, I haven’t actually tested any of this as yet, but it looks like it ought to work out OK. If not, well then we will make a few adjustments.

The next thing on the list, then, is to build out the other referenced function that does not yet exist, the syncApplications function. Here is where we will eventually run into the need for that code that was cloned earlier, so we will want to clean all of that up finally, and the publishInstance function actually looks like a good model to use for the way in which it works, sending back all of the good and bad results in an object that can be evaluated by the caller where actions can be taken that are appropriate for the specific context.

Here is the original version of the code that sends over an application:

processPhase5: function(answer) {
	var mbrAppGR = new GlideRecord('x_11556_col_store_member_application');
	if (mbrAppGR.get(answer.mbrAppId)) {
		var host = gs.getProperty('x_11556_col_store.host_instance');
		var token = gs.getProperty('x_11556_col_store.active_token');
		var thisInstance = gs.getProperty('instance_name');
		var request  = new sn_ws.RESTMessageV2();
		request.setHttpMethod('get');
		request.setBasicAuth(this.WORKER_ROOT + host, token);
		request.setRequestHeader("Accept", "application/json");
		request.setEndpoint('https://' + host + '.service-now.com/api/now/table/x_11556_col_store_member_application?sysparm_fields=sys_id&sysparm_query=provider.instance%3D' + thisInstance + '%5Ename%3D' + encodeURIComponent(mbrAppGR.getDisplayValue('name')));
		var response = request.execute();
		if (response.haveError()) {
			answer = this.processError(answer, 'Error returned from Host instance: ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
		} else if (response.getStatusCode() == '200') {
			var jsonString = response.getBody();
			var jsonObject = {};
			try {
				jsonObject = JSON.parse(jsonString);
			} catch (e) {
				answer = this.processError(answer, 'Unparsable JSON string returned from Host instance: ' + jsonString);
			}
			if (!answer.error) {
				var payload = {};
				payload.name = mbrAppGR.getDisplayValue('name');
				payload.description = mbrAppGR.getDisplayValue('description');
				payload.current_version = mbrAppGR.getDisplayValue('current_version');
				payload.active = 'true';
				request  = new sn_ws.RESTMessageV2();
				request.setBasicAuth(this.WORKER_ROOT + host, token);
				request.setRequestHeader("Accept", "application/json");
				if (jsonObject.result && jsonObject.result.length > 0) {
					answer.hostAppId = jsonObject.result[0].sys_id;
					request.setHttpMethod('put');
					request.setEndpoint('https://' + host + '.service-now.com/api/now/table/x_11556_col_store_member_application/' + answer.hostAppId);
				} else {
					request.setHttpMethod('post');
					request.setEndpoint('https://' + host + '.service-now.com/api/now/table/x_11556_col_store_member_application');
					payload.provider = thisInstance;
				}
				request.setRequestBody(JSON.stringify(payload, null, '\t'));
				response = request.execute();
				if (response.haveError()) {
					answer = this.processError(answer, 'Error returned from Host instance: ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
				} else if (response.getStatusCode() != 200 && response.getStatusCode() != 201) {
					answer = this.processError(answer, 'Invalid HTTP Response Code returned from Host instance: ' + response.getStatusCode());
				} else {
					jsonString = response.getBody();
					jsonObject = {};
					try {
						jsonObject = JSON.parse(jsonString);
					} catch (e) {
						answer = this.processError(answer, 'Unparsable JSON string returned from Host instance: ' + jsonString);
					}
					if (!answer.error) {
						answer.hostAppId = jsonObject.result.sys_id;
					}
				}
			}
		} else {
			answer = this.processError(answer, 'Invalid HTTP Response Code returned from Host instance: ' + response.getStatusCode());
		}
	} else {
		answer = this.processError(answer, 'Invalid Member Application sys_id: ' + answer.appSysId);
	}

	return answer;
}

… and here is the other version that was cloned and hacked up based on the original:

publishNewVersion: function(newVersion, targetInstance, attachmentId) {
	var targetGR = new GlideRecord('x_11556_col_store_member_organization');
	if (targetGR.get('instance', targetInstance)) {
		var token = targetGR.getDisplayValue('token');
		var versionGR = new GlideRecord('x_11556_col_store_member_application_version');
		if (versionGR.get(newVersion)) {
			var canContinue = true;
			var targetAppId = '';
			var mbrAppGR = versionGR.member_application.getRefRecord();
			var request  = new sn_ws.RESTMessageV2();
			request.setHttpMethod('get');
			request.setBasicAuth(this.WORKER_ROOT + targetInstance, token);
			request.setRequestHeader("Accept", "application/json");
			request.setEndpoint('https://' + targetInstance + '.service-now.com/api/now/table/x_11556_col_store_member_application?sysparm_fields=sys_id&sysparm_query=provider.instance%3D' + mbrAppGR.getDisplayValue('provider.instance') + '%5Ename%3D' + encodeURIComponent(mbrAppGR.getDisplayValue('name')));
			var response = request.execute();
			if (response.haveError()) {
				gs.error('CollaborationStoreUtils.publishNewVersion: Error returned from Target instance ' + targetInstance + ': ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
				canContinue = false;
			} else if (response.getStatusCode() == '200') {
				var jsonString = response.getBody();
				var jsonObject = {};
				try {
					jsonObject = JSON.parse(jsonString);
				} catch (e) {
					gs.error('CollaborationStoreUtils.publishNewVersion: Unparsable JSON string returned from Target instance ' + targetInstance + ': ' + jsonString);
					canContinue = false;
				}
				if (canContinue) {
					var payload = {};
					payload.name = mbrAppGR.getDisplayValue('name');
					payload.description = mbrAppGR.getDisplayValue('description');
					payload.current_version = mbrAppGR.getDisplayValue('current_version');
					payload.active = 'true';
					request  = new sn_ws.RESTMessageV2();
					request.setBasicAuth(this.WORKER_ROOT + targetInstance, token);
					request.setRequestHeader("Accept", "application/json");
					if (jsonObject.result && jsonObject.result.length > 0) {
						targetAppId = jsonObject.result[0].sys_id;
						request.setHttpMethod('put');
						request.setEndpoint('https://' + targetInstance + '.service-now.com/api/now/table/x_11556_col_store_member_application/' + targetAppId);
					} else {
						request.setHttpMethod('post');
						request.setEndpoint('https://' + targetInstance + '.service-now.com/api/now/table/x_11556_col_store_member_application');
						payload.provider = mbrAppGR.getDisplayValue('provider.instance');
					}
					request.setRequestBody(JSON.stringify(payload, null, '\t'));
					response = request.execute();
					if (response.haveError()) {
						gs.error('CollaborationStoreUtils.publishNewVersion: Error returned from Target instance ' + targetInstance + ': ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
						canContinue = false;
					} else if (response.getStatusCode() != 200 && response.getStatusCode() != 201) {
						gs.error('CollaborationStoreUtils.publishNewVersion: Invalid HTTP Response Code returned from Target instance ' + targetInstance + ': ' + response.getStatusCode());
						canContinue = false;
					} else {
						jsonString = response.getBody();
						jsonObject = {};
						try {
							jsonObject = JSON.parse(jsonString);
						} catch (e) {
							gs.error('CollaborationStoreUtils.publishNewVersion: Unparsable JSON string returned from Target instance ' + targetInstance + ': ' + jsonString);
							canContinue = false;
						}
						if (canContinue) {
							targetAppId = jsonObject.result.sys_id;
						}
					}
				}
			} else {
				gs.error('CollaborationStoreUtils.publishNewVersion: Invalid HTTP Response Code returned from Target instance ' + targetInstance + ': ' + response.getStatusCode());
			}
			if (canContinue) {
				this.publishVersionRecord(targetInstance, token, versionGR, targetAppId, attachmentId);
			}
		} else {
			gs.error('CollaborationStoreUtils.publishNewVersion: Version record not found: ' + newVersion);
		}
	} else {
		gs.error('CollaborationStoreUtils.publishNewVersion: Target instance record not found: ' + targetInstance);
	}
}

Both functions accomplish the same thing, first checking to see if the application is already present on the remote instance, and then either adding it or updating it depending on whether or not it was already there. This is what we would want to do as well, so it seems as if we could collapse these two into a single, reusable function that would server our current purpose and these other two use cases as well. We could adopt the result object approach from the above publishInstance function and return the same form of object from our new function and then each user of the function could make their own choices as to what to do with the information returned. This is something that I have wanted to do for a while now, but it looks like it might be a little bit of work and some more refactoring of existing code, so let’s save all of that for our next installment.

Collaboration Store, Part XXXIX

“It’s not that I’m so smart, it’s just that I stay with problems longer.”
Albert Einstein

Last time, we built the Flow that will call our Script Include function to update all of the instances in the community that might be missing any artifacts. Now we need to get started on the actual function that will check in with each instance, gather its inventory of artifacts, compare it to that of the Host, and send over any missing items. As with most things, there are a number of ways to go about this, but my plan is to go through the instances first, and then work through the applications for each instance as each instance is being processed. Similarly, my intent is to work through the application versions as each application is being processed. To begin, we will need to fetch all of the instances found on the Host and stuff them into an array.

var instance = '';
var token = '';
var instanceList = [];
instanceGR = new GlideRecord('x_11556_col_store_member_organization');
instanceGR.addQuery('active', true);
instanceGR.query();
while (instanceGR.next()) {
	instanceList.push(instanceGR.getDisplayValue('name'));
	if (instanceGR.getUniqueValue() == instanceId) {
		instance = instanceGR.getDisplayValue('name');
		token = instanceGR.getDisplayValue('token');
	}
}

The other thing that this code accomplishes is to gather up the name and credentials for the target instance. The sys_id of the instance is passed to the function from the Flow, but to make the REST API calls, we need the actual name of the instance and the token. We could read that record directly before we started everything, but since we were spinning through the instance records anyway, I decided to just check each one and grab the data when we happened to be processing that particular instance. Of course, once we do that we need to check to make sure that we actually came across an instance with that sys_id before we go any further.

if (instance > '' && token > '') {
	this.fetchInstanceList(instance, token, instanceList);
} else {
	gs.error('InstanceSyncUtils.syncInstance - Requested instance not on file: ' + instanceId);
}

Assuming that we did come across the instance being synced, we now need to contact that instance and gather up all of the instances present on that system. For that, we can call on our old friend sn_ws.RESTMessageV2. But before we do that, we can use the REST API Explorer to come up with an end point URL that will get us the results that we need. We want to gather up all of the active instances from the Client instance, but we will only need a couple of the fields for our purpose here. We can specify those in the sysparm_fields parameter. I also like to alter the sysparm_limit parameter from the default of 1 to the alternative of 10, just to get more than one result in the output to help verify the query.

Using the REST API Explorer to generate an end point URL

Once we have entered all of the appropriate parameters, we can hit the Send button, which will produce the URL and also display some sample results in the Response Body section. After stripping off the server portion and removing the limitation parameter, we are left with the following value to use as our end point:

/api/now/table/x_11556_col_store_member_organization?sysparm_query=active%3Dtrue&sysparm_fields=name%2Csys_id

With our end point URL in hand, we can now build the request object.

fetchInstanceList: function(instance, token, instanceList) {
	var request = new sn_ws.RESTMessageV2();
	request.setHttpMethod('get');
	request.setBasicAuth(this.WORKER_ROOT + instance, token);
	request.setRequestHeader("Accept", "application/json");
	request.setEndpoint('https://' + instance + '.service-now.com/api/now/table/x_11556_col_store_member_organization?sysparm_query=active%3Dtrue&sysparm_fields=name%2Csys_id');
	...
}

Once the request object has been built, we can execute the request and check the results.

var response = request.execute();
if (response.haveError()) {
	gs.error('InstanceSyncUtils.syncInstance - Error returned from attempt to fetch instance list: ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
} else if (response.getStatusCode() == '200') {
	var jsonString = response.getBody();
	var jsonObject = {};
	try {
		jsonObject = JSON.parse(jsonString);
	} catch (e) {
		gs.error('InstanceSyncUtils.syncInstance - Unparsable JSON string returned from attempt to fetch instance list: ' + jsonString);
	}
	if (Array.isArray(jsonObject.result)) {
		...
	} else {
		gs.error('InstanceSyncUtils.syncInstance - Invalid response body returned from attempt to fetch instance list: ' + response.getStatusCode());
	}
} else {
	gs.error('InstanceSyncUtils.syncInstance - Invalid HTTP response code returned from attempt to fetch instance list: ' + response.getStatusCode());
}

Finally, if all has gone well, we can loop through the Host’s list of instances and then for every instance on the Host list, see if we can find it on the Client instance list. If we do not find it, then we need to push it over. Either way, once that has been checked and corrected if necessary, the next thing to do will be to check all of the applications present for that instance from the Host list.

for (var i=0; i<instanceList.length; i++) {
	var thisInstance = instanceList[i];
	var remoteSysId = '';
	for (var j=0; j<jsonObject.result.length && remoteSysId == ''; j++) {
		if (jsonObject.result[j].name == thisInstance) {
			remoteSysId = jsonObject.result[j].sys_id;
		}
	}
	if (remoteSysId == '') {
		remoteSysId = this.sendInstance(instance, token, thisInstance.name);
	}
	this.syncApplications(instance, token, thisInstance, remoteSysId);
}

We already have code to send over an instance record. In fact, we have already cloned that code and have a second version for another purpose. Rather than clone that code yet a third time, it’s long past the point for all of that to be consolidated into a single function that can work for any and all purposes. That seems like a bit of work, though, so let’s stop here and save that for our next time out.

Collaboration Store, Part XXXVIII

“You don’t have to be good to start … you just have to start to be good!”
Joe Sabah

Last time out we really did not accomplish anything of any consequence, but today let’s see if we can actually make some progress on something of real value. We need to create a process that will run every so often and check to make sure that all of the instances in the community have all of the latest content. We can use the Flow Designer to create and schedule a flow that will run daily on the Host instance and compare the artifacts present on each Client instance with the artifacts present on the Host, and then send over any missing items that never made it over originally for whatever reason. Doing it this way will eliminate the need to track and record errors when they happen, since we will just compare the Client inventory with that of the Host and correct any discrepancies. We don’t really need to know what failed or why.

We could make all of the REST API calls to the Client instances using the Integration Hub, but since that is a collection of optional products, not every instance will have all of those components installed and we would not want to make that a prerequisite for the product. To make this work on any ServiceNow configuration, we will want to roll our own calls via Javascript, and we will just use the Flow Designer to schedule a simple flow that loops through all of the Client instance records and then calls a Script Action that will do all of the heavy lifting. Before we build the Action though, we will want to build a Script Include function that can be called in the Action. Since this will be a significant script, we should probably create a new Script Include specific to this purpose rather than adding a number of new functions to any of our existing Script Includes. We can call our new Script Include InstanceSyncUtils and create a simple function called syncInstance with a single argument of the sys_id of the record for the instance to be synced. For now, we can just stub out the function with a simple logging statement and then circle back later to fill in the details. Right now, we just want to have something to call in our Flow Designer Action.

New Script Include function to be called in our new Action

With that out of the way, we can now fire up the Flow Designer and navigate to New -> Action. We will call our new Action Sync Instance and configure a single Input called Instance ID.

New Sync Instance Action with a single Input

For the Script step, we will just pass the Input to our new Script Include function:

(function execute(inputs, outputs) {
	var isu = new InstanceSyncUtils();
	isu.syncInstance(inputs.instance_id);
})(inputs, outputs);

There is currently nothing returned by the function, so there is no need to define any Outputs. At this point, all we need to do is to Save and Publish the Action before turning our attentions to the actual Flow itself.

To build a Flow, go back to the Home Page of the Flow Designer and select New -> Flow. On the resulting pop-up Flow Properties panel, enter all of the details for the Flow and set the Run As value to System User.

New Flow Properties

Since we want this Flow to run on its own every day, we can set the Trigger to Daily, and the Time to 12:30, which should run it in the middle of the lunch hour in the Host time zone, when most instances should be available.

Flow Trigger configuration

Since we only want this Flow to run on the Host instance, the first thing that we need to do is to create a Flow Variable that indicates whether or not this instance is the Host. To set the value of the variable, we add a Set Flow Variables step that uses the following script to determine if this instance is the Host based on System Properties.

return gs.getProperty('instance_name') == gs.getProperty('x_11556_col_store.host_instance');

The next step then will be a conditional that checks the value of the new Flow Variable.

Making sure that this is the Host instance

Failing this test will terminate the Flow, but if this is the Host instance, then the next thing that we need to do is gather up all of the Client instance records and then loop through them and call our new Action for each one in turn.

Find all records where Active is true and Host is false

Inside the following For Each Item loop, we can then invoke our new Action, passing in the sys_id of the current record.

Calling our new Action inside the For Each Item loop

That completes the work on the Flow, and all we need to do now is to Save and Activate the Flow. At this point, the Flow will kick off every day at 12:30 local time, but it won’t really do much except put a few entries in the System Log. The real work will be done in the Script Include, and since that’s a major effort in and of itself, we’ll leave that for our next exciting episode.

Collaboration Store, Part XXXII

“There are two ways to write error-free programs; only the third works.”
Alan J. Perlis

Now that we have all of the Javascript functions needed to push a new application version out to all of the Client instances, we need to create an Action that will call the root function and then create a Subflow that will utilize the new Action. Since we already created a similar Action for the process that shares new Client instances with existing Client instances, the easiest thing to do will be to bring that guy up in the Flow Designer and make a copy. To make a copy, pull up the Action that you want to copy, and use the ellipses menu to select Copy from the drop-down menu.

Making a copy of the Publish New Instance Action

Once we have our copy, we need to make the necessary changes to adapt it to our purpose. The publishNewVersion function that we just created takes three arguments (newVersion, targetInstance, and attachmentId), so we will need to modify the Action Inputs to bring in those values. The new Action Inputs now look like this:

Modified Action Inputs

Similarly, we will need to modify the Input Variables of the Script Step to pass along these values:

Modified Script Input Variables

The last thing that we need to do is modify the script itself. The existing script in the cloned Action was pretty simple, as it was just a call to one of our Script Include functions:

(function execute(inputs, outputs) {
	var csu = new CollaborationStoreUtils();
	csu.publishNewInstance(inputs.new_instance, inputs.target_instance);
})(inputs, outputs);

We also want to call a function, and it is in the very same Script Include, so our modification is just a matter of changing the name of the function along with the arguments passed.

(function execute(inputs, outputs) {
	var csu = new CollaborationStoreUtils();
	csu.publishNewVersion(inputs.new_version, inputs.target_instance, inputs.attachment_id);
})(inputs, outputs);

That’s it for the changes, so now all we have to do is Save and Publish the new Action and we are ready to utilize it in our new Subflow. Creating the Subflow will be another Copy operation, this time from our existing New_Collaboration_Store_Instance Subflow. We’ll call our copy Distribute_New_Application_Version, and once again, we will need to make some modifications to adapt it to our new purpose. The original Subflow had a single input, the New Instance. Instead of a new instance, we will be distributing a new application version, so we can replace the New Instance input with a New Version input that will contain the sys_id of the new version record. We will also need the sys_id of the Attachment record, so we will add an Attachment ID input as well.

Modified Inputs for the new Distribute_New_Application_Version Subflow

The target instances for this process will be the same as those for the original Subflow: all instances in the community except for the Host instance and the instance from which the data was provided (which could also be the Host in this case, since Host instances can publish applications just like any other Client instance). To filter out the data providing instance, we will need to add a new step to use the passed version record sys_id to fetch the GlideRecord that will lead us to the instance that produced the application.

New step to fetch the version record

This gives us the information that we need to complete the filter on the query of the instance table so that we can gather up all of the instances except for the Host and the instance that produced the application.

Modified query filter using the data available in the fetched version record

Now all we need to do is to replace the original custom Action inside of the loop with the new Action that we just created.

New Action added to push application artifacts to the target instance

This completes the changes needed to the cloned Subflow, so again, all we need to do is to Save and Publish and we are all ready to go. Now all we need to is to come up with a way to launch this Subflow whenever a new application version is sent over to the Host instance. We’ll take a look at that next time out, and hopefully release a new Update Set as well so that those of you that have been kind enough to participate in the testing can give this all a trial run.

Collaboration Store, Part XXIX

As a rule, software systems do not work well until they have been used, and have failed repeatedly, in real applications.”
David Lorge Parnas

Now that we have completed the initial version of the application publishing process, it would be nice to release a new Update Set so that the folks who are inclined to help out with the testing can try it all out. However, we have already received feedback from the earlier testing of the set-up process, and we should really address all of those issues before publishing a new version for further testing. Here are the items discovered during the testing of the first version of the software released earlier:

  • Installation error: Table ‘sys_hub_action_status_metadata’ does not exist
  • Not allowing update of property: x_11556_col_store.store_name
  • Not allowing update of property: x_11556_col_store.host_instance
  • In the setup, the instance name field doesn’t inform you that you only need the instance prefix, not the full url
  • You can only collaborate with one host

I was able to reproduce them all, and here is what I ended up doing for each:

Installation error: Table ‘sys_hub_action_status_metadata’ does not exist

I tried to find out some information on the purpose and use for this table, but I couldn’t really find anything that told me anything of value. I still believe that the ‘sys_hub_action_status_metadata’ table is related to a version or plugin that I have in my instance, but was not present in the instance on which the test installation was being performed. Since it didn’t seem as if it was anything that had anything to do with the operation of the application, I decided to just delete all references to it in the Update Set, just to avoid this issue. I’m not sure if that will cause any issues with any instances that actually do have this table, but it seemed like something worth a try and we’ll see what happens. If it causes a problem, I will not do that in the future and just throw in some release notes that say just ignore the error if it comes up. But let’s see if this works, first.

Not allowing update of property: x_11556_col_store.store_name
Not allowing update of property: x_11556_col_store.host_instance

This one took a little bit of research and a little bit of trial and error (mostly error!), but I eventually solved the problem by moving all updates to these properties into my global utilities so that the command was being executed by a global component instead of a scoped component. I did this once before with the gs.sleep() function, and this worked out just as well. Here is the function that I added to the global utilities:

setProperty: function(name, value) {
	gs.setProperty(name, value);
},

Once I created the function in the global utilities, it was just a matter of changing all gs.setProperty() function calls to csgu.setProperty() and that seemed to have done the trick.

In the setup, the instance name field doesn’t inform you that you only need the instance prefix, not the full url

Since the snh-form-field tag provides for a “help” attribute, which appears underneath the label, I simply updated the definition for that field to include a little help:

<snh-form-field
  snh-model="c.data.host_instance_id"
  snh-name="host_instance_id"
  snh-label="Host Instance ID"
  snh-help="Enter the instance ID only, not the full URL of the instance (https://{instance_id}.servicenow.com))"
  snh-required="c.data.instance_type == 'client'"
  ng-show="c.data.instance_type == 'client'"/>

This renders a little help text underneath the label, and looks like this when the page comes up:

Initial set-up screen with added help text for the Host Instance ID

That should resolve this issue.

You can only collaborate with one host

As I mentioned earlier when this was first reported, this is by design. Allowing multiple Host instances introduces a level of complexity with which I’m not quite ready to deal just yet. At this point, we will just file this one under Will not fix or Future release for now.

Other than these issues, I have not personally encountered or heard of any other issues with the set-up process, but that doesn’t mean that the testing is complete by any means. If anyone wants to join the testing process and you missed out on testing the earlier version, you will still have to work your way through the set-up process for any instances involved, so please report any issues with the set-up process as well as any issues with the application publication process.

Speaking of the application publication process, there is still yet another aspect of this process that remains to be developed. Once a new application has been published to the Host instance, the Host instance will need to push that version out to all of the other Client instances. This version does not include that functionality, but we will need to throw that in at some point. I just did not want to hold up the testing for that particular feature, since it really is a completely independent operation.

For those of you who are interested in participating in the testing, you will need this new Update Set, plus this additional global component that is not included in the scoped application. Also, if this is your first time installing the application, you will also need the latest version of snh-form-fields, which you can find here. As always, if you have any feedback, positive or negative, please leave the details in the comments. All information is welcome and much appreciated. Thanks in advance for your assistance.