Collaboration Store, Part LXXIX

“You just have to keep driving down the road. It’s going to bend and curve and you’ll speed up and slow down, but the road keeps going.”
Ellen DeGeneres

Last time, we got the ability to toggle between the card/tile view and table view working as it should, and we made an initial stab at making the local apps look a little different from the apps that could be or have been pulled down from the Host. Now we need to figure out what we want to happen when the operator clicks on any of the links that are present on the tiles or table rows. Currently, the main portion of the tile is clickable, and in the footer, the version number could be clickable as well and could potentially launch the install process. In fact, let’s take a look at that first.

Right now, if you want to install a version of a store application, you go to the version record and click on the Install button. Let’s take a quick peek at that UI Action and see how that works.

var attachmentGR = new GlideRecord('sys_attachment');
attachmentGR.addQuery('table_name', 'x_11556_col_store_member_application_version');
attachmentGR.addQuery('table_sys_id', current.sys_id);
attachmentGR.addQuery('content_type', 'CONTAINS', 'xml');
attachmentGR.query();
if (attachmentGR.next()) {
	action.setRedirectURL('/upload.do?attachment_id=' + attachmentGR.getUniqueValue());
} else {
	gs.addErrorMessage('No Update Set XML file found attached to this version');
}

Basically, it just links to the stock upload.do page (which we hacked up a bit sometime back) with the attachment sys_id as a parameter. Assuming that we had the attachment sys_id included along with the rest of the row data, we could simply build an anchor tag to launch the install such as the following.

<a href="/upload.do?attachment_id={{::item.attachmentId}}">

To get the attachment sys_id, we could steal some of the UI Action code above to build a function for that purpose.

function getAttachmentId(applicationId, version) {
	var attachmentId = '';

	var versionGR = new GlideRecord('x_11556_col_store_member_application_version');
	versionGR.addQuery('member_application', applicationId);
	versionGR.addQuery('version', version);
	versionGR.query();
	if (versionGR.next()) {
		var attachmentGR = new GlideRecord('sys_attachment');
		attachmentGR.addQuery('table_name', 'x_11556_col_store_member_application_version');
		attachmentGR.addQuery('table_sys_id', versionGR.getUniqueValue());
		attachmentGR.addQuery('content_type', 'CONTAINS', 'xml');
		attachmentGR.query();
		if (attachmentGR.next()) {
			attachmentId = attachmentGR.getUniqueValue();
		}
	}
		
	return attachmentId;
}

For local apps, or apps that are already installed and up to date, there is no action to take. But for apps that have not been installed, or those that have, but are not on the latest version, a link to the install process would be appropriate. Two mutually exclusive tags would cover both cases.

<span ng-if="item.local || item.state == 2" class="current-version" title="${Version} {{::item.version}}">v{{::item.version}}</span>
<a ng-if="!item.local && item.state != 2" class="{{['not-installed','earlier-version'][item.state]}}" href="/upload.do?attachment_id={{::item.attachmentId}}" title="${Click here to} {{item.state == 0?'install':'upgrade to'}} ${version} {{::item.version}}">v{{::item.version}}</a>

To test this out, we will need to publish an app on our Host instance and bring up the store on our Client instance to see how this looks.

Install/Upgrade link on nonlocal applications

That takes care of the install link. For the main action when clicking on the tile itself (or on the app name in the table view), we should have some way of displaying all of the details of the app. We could link to the existing form for the application table, but we might want something a little more formatted, and maybe even just a pop-up so that you don’t actually leave the shopping experience. Let’s see if we can throw something like that together next time out.

Collaboration Store, Part LXXIII

“A little more persistence, a little more effort, and what seemed hopeless failure may turn to glorious success.”
Elbert Hubbard

Last time, we went over some of the remaining problems with this iteration of the software, and stopped short of providing a remedy for one of the issues, the fact that there is no capacity to include an instance logo image during the initial set-up process. Today we will take a look at correcting that design flaw.

Not one to create anything from scratch when someone else might have already put in all of that effort, I looked around for a portal page that had an image upload feature already in place. It turns out that the stock user_profile page, which uses the User Profile widget, has just the sort of thing that I was thinking of.

Stock User Profile page with existing image upload feature

Starting with the HTML, I dug into the code for the page and identified this section as pertaining to the image and upload button.

<div class="row">
  <div class="avatar-extra-large avatar-container" style="cursor:default;">
    <div class="avatar soloAvatar bottom">
      <div class="sub-avatar mia" ng-style="avatarPicture"><i class="fa fa-user"></i></div>
    </div>
  </div>
</div>
<div class="row">
  <button ng-if="::connectEnabled()" ng-click="openConnectConversation()" type="button"
          class="btn btn-primary send-message"><span class="glyphicon glyphicon-comment pad-right"></span>${Message}</button>
  <!-- file upload -->
  <span ng-if="::data.isLoggedInUsersProfile">
    <input ng-show="false" type="file" accept="image/jpeg,image/png,image/bmp,image/x-windows-bmp,image/gif,image/x-icon,image/svg+xml" ng-file-select="attachFiles({files: $files})" />
    <button ng-click="uploadNewProfilePicture($event)"
            ng-keypress="uploadNewProfilePicture($event)" type="button"
            class="btn btn-default send-message">${Upload Picture}</button>
  </span>
</div>

It was all pretty close to what I wanted, but I did make a few little tweaks here and there and finally ended up with this.

<div class="row" style="padding: 15px;">
  <div class="avatar-extra-large avatar-container" style="cursor:default;">
    <div class="avatar soloAvatar bottom">
      <div class="sub-avatar mia" ng-style="instanceLogoImage"><i class="fa fa-image"></i></div>
    </div>
  </div>
</div>
<div class="row" style="padding: 15px;">
  <input ng-show="false" type="file" accept="image/jpeg,image/png,image/bmp,image/x-windows-bmp,image/gif,image/x-icon,image/svg+xml" ng-file-select="attachFiles({files: $files})" />
  <button ng-click="uploadInstanceLogoImage($event)"
          ng-keypress="uploadInstanceLogoImage($event)" type="button"
          class="btn btn-default send-message">${Upload Instance Logo Image}</button>
</div>

Without bothering to build out the referenced functions to make it all actually work, I could still pull the page up and see how it rendered out at this point, just to get an idea of how it was going to look.

New initial set-up page with added logo image feature

So that looks pretty good. Now we need to add the client-side code needed to make it work. As with the HTML, we will turn to the User Profile widget to find some code to start out with.

$scope.uploadNewProfilePicture = function($event) {
	$event.stopPropagation();
	if($event.keyCode === 9) {
		return;
	}
	var $el = $element.find('input[type=file]');
	$el.click();
}

$scope.attachFiles = function(files) {
	var file = files.files[0];

	var validImage = false;

	switch(file.type) {
		case 'image/jpeg':
		case 'image/png':
		case 'image/bmp':
		case 'image/x-windows-bmp':
		case 'image/gif':
		case 'image/x-icon':
		case 'image/svg+xml':
			validImage = true;
			break;
		default:
			break;
	}

	if(!validImage) {
		alert(file.name + " " + i18n.getMessage("isn't a recognized image file format"));
		return;
	}

	snAttachmentHandler.create("live_profile", $scope.data.liveProfileID).uploadAttachment(file, {
		sysparm_fieldname: "photo"
	}).then(function(response) {
		var obj = {};
		obj.newAvatarId = response.sys_id;
		$scope.avatarPicture = {
			'background-image': "url('" + response.sys_id + ".iix')",
			'color': 'transparent'
		};
		$rootScope.$broadcast("sp.avatar_changed", obj);
		// timeout is required for screen reader to pick up the message once file upload prompt is dismissed
		$timeout(function() {
		   spAriaUtil.sendLiveMessage(i18n.getMessage('Profile picture updated successfully'));
		}, 500);
	});
}

$scope.avatarPicture = "";

$http.get('/api/now/live/profiles/sys_user.' + $scope.data.sysUserID).then(function (response) {
	if (response.data.result && response.data.result.avatar){
		var avatarPicture = response.data.result.avatar.replace("?t=small", "");
		$scope.avatarPicture = {
			'background-image': "url('" + avatarPicture + "')",
			'color': 'transparent'
		};
	}
});

Once again, I went through the code and made a few minimal modifications to make things work for our purpose and ended up with this.

$scope.uploadInstanceLogoImage = function($event) {
	$event.stopPropagation();
	if($event.keyCode === 9) {
		return;
	}
	var $el = $element.find('input[type=file]');
	$el.click();
};

$scope.attachFiles = function(files) {
	var file = files.files[0];

	var validImage = false;

	switch(file.type) {
		case 'image/jpeg':
		case 'image/png':
		case 'image/bmp':
		case 'image/x-windows-bmp':
		case 'image/gif':
		case 'image/x-icon':
		case 'image/svg+xml':
			validImage = true;
			break;
		default:
			break;
	}

	if(!validImage) {
		alert(file.name + " " + i18n.getMessage("isn't a recognized image file format"));
		return;
	}

	snAttachmentHandler.create('x_11556_col_store_member_organization', $scope.data.sys_id).uploadAttachment(file, {
		sysparm_fieldname: "logo"
	}).then(function(response) {
		$scope.instanceLogoImage = {
			'background-image': "url('" + response.sys_id + ".iix')",
			'color': 'transparent'
		};
	});
};

$scope.instanceLogoImage = "";

All of that would work great just as it is except for one thing: the snAttachmentHandler.create function wants to attach the file to an existing record, and the current design of the process doesn’t actually create the record in the database until after the Host instance accepts the registration request. We had that same problem with the Update Set XML file during the application publishing process, and I ended up attaching the file to the system application record, and then copying it over to the version record later, once the version record was created. Here, though, there really isn’t a record to which it could be attached temporarily, so in order for this to work, I am going to have to go ahead and create the record up front, and then do an update instead of an insert once the Host has granted access to the community. That took a little bit of re-engineering, but once that was done, everything seemed to work out OK. This is something that really needs to be tested thoroughly, though, but just from my cursory check-out it appears to be functioning as desired.

Image upload function in operation

With that up and running, it is time to release yet another Update Set for testing purposes. As with the 0.7.1 release, this should be a straight drop-in replacement for the 0.7/0.7.1 Update Set, and all of the other artifacts should be OK just as they are.

More information on previewing, committing, and testing can be found here and here and here. As always, feedback of any kind in the comments section is welcome, encouraged, and very much appreciated. Any information on your experiences is always a treat, and will give us a little something to review next time out.

Collaboration Store, Part LXX

“Software bugs are like cockroaches; there are probably dozens hiding in difficult to reach places for every one you find and fix.”
Donald G. Firesmith

Last time, we went through the list of issues that have been reported so far, the biggest one being the fact that the REST API call to the Host instance is sending over the application logo image attachment instead of the Update Set XML file attachment. Since then, we have received some additional information in the form of the data logged to the REST API log file. Here is the entry of interest:

{
	“size_bytes”: “547670”,
	“file_name”: “logo”,
	“sys_mod_count”: “0”,
	“average_image_color”: “”,
	“image_width”: “”,
	“sys_updated_on”: “2022-08-02 16:55:55”,
	“sys_tags”: “”,
	“table_name”: “x_11556_col_store_member_application_version”,
	“sys_id”: “c227acc297855110b40ebde3f153aff3”,
	“image_height”: “”,
	“sys_updated_by”: “csworker1.dev69362”,
	“download_link”: “https://dev69362.service-now.com/api/now/attachment/c227acc297855110b40ebde3f153aff3/file”,
	“content_type”: “image/jpeg”,
	“sys_created_on”: “2022-08-02 16:55:55”,
	“size_compressed”: “247152”,
	“compressed”: “true”,
	“state”: “pending”,
	“table_sys_id”: “b127a88297855110b40ebde3f153afa6”,
	“chunk_size_bytes”: “700000”,
	“hash”: “8b5a07a6c0edf042df4b3c24e729036562985b705427ba7e33768566de94e96f”,
	“sys_created_by”: “csworker1.dev69362”
}

If you look at the table_name property, you can see that it is attaching something to the version record, and if you look at the file_name and content_type properties, you can see that it isn’t the Update Set XML file that it is sending over. So let’s take a look at the shared code that sends over the Update Set XML file attachment and see if we can see where things may have gone wrong.

pushAttachment: function(attachmentGR, targetGR, remoteVerId) {
	var result = {};

	var gsa = new GlideSysAttachment();
	result.url = 'https://';
	result.url += targetGR.getDisplayValue('instance');
	result.url += '.service-now.com/api/now/attachment/file?table_name=x_11556_col_store_member_application_version&table_sys_id=';
	result.url += remoteVerId;
	result.url += '&file_name=';
	result.url += attachmentGR.getDisplayValue('file_name');
	result.method = 'POST';
	var request = new sn_ws.RESTMessageV2();
	request.setEndpoint(result.url);
	request.setHttpMethod(result.method);
	request.setBasicAuth(this.WORKER_ROOT + targetGR.getDisplayValue('instance'), targetGR.getDisplayValue('token'));
	request.setRequestHeader('Content-Type', attachmentGR.getDisplayValue('content_type'));
	request.setRequestHeader('Accept', 'application/json');
	request.setRequestBody(gsa.getContent(attachmentGR));
	var response = request.execute();
	result.status = response.getStatusCode();
	result.body = response.getBody();
	if (result.body) {
		try {
			result.obj = JSON.parse(result.body);
		} catch (e) {
			result.parse_error = e.toString();
		}
	}
	result.error = response.haveError();
	if (result.error) {
		result.error_code = response.getErrorCode();
		result.error_message = response.getErrorMessage();
	}
	this.logRESTCall(targetGR, result);

	return result;
}

By this point in the process, the GlideRecord for the attachment has already been obtained from the database, so the problem has to be upstream from here. This is a shared function called from many places, but our problem is related to the application publishing process, so let’s take a look at the ApplicationPublisher Script Include and see if we can find where this function is called.

processPhase7: function(answer) {
	var gsa = new GlideSysAttachment();
	var attachmentGR = new GlideRecord('sys_attachment');
	if (attachmentGR.get(answer.attachmentId)) {
		var targetGR = this.getHostInstanceGR();
		var csu = new CollaborationStoreUtils();
		var result = csu.pushAttachment(attachmentGR, targetGR, answer.hostVerId);
		if (result.error) {
			answer = this.processError(answer, 'Error returned from Host instance: ' + result.error_code + ' - ' + result.error_message);
		} else if (result.parse_error) {
			answer = this.processError(answer, 'Unparsable JSON string returned from Host instance: ' + result.body);
		} else if (result.status != 200 && result.status != 201) {
			answer = this.processError(answer, 'Invalid HTTP Response Code returned from Host instance: ' + result.status);
		} else {
			answer.hostVerId = result.obj.result.sys_id;
		}
	} else {
		answer = this.processError(answer, 'Invalid attachment record sys_id: ' + answer.attachmentId);
	}

	return answer;
}

Here we are fetching the attachment record based on the sys_id in the answer object property called attachmentId. There isn’t much opportunity for things to go tango uniform with this particular code, so I think we have to assume that somewhere upstream of this logic the value of answer.attachmentId got set to the sys_id of the logo attachment instead of the sys_id of the Update Set XML file attachment. So it looks like we need to do a quick search for answer.attachmentId and see where this property may have gotten corrupted.

Since the version record does not yet exist when the Update Set XML file is generated, it is initially attached to the stock application record. Then, once the version record has been created, the attachment is copied from the application record to the version record, and then the original attachment file is removed from the stock application record. All of that seems to work, since the Update Set XML file is, in fact, attached to the version record on the original source instance; however, somewhere along the line, the sys_id of that attachment record in the answer object ends up being the sys_id of the logo image attachment record. Let’s take a look at that code.

processPhase4: function(answer) {
	var gsa = new GlideSysAttachment();
	var values = gsa.copy('sys_app', answer.appSysId, 'x_11556_col_store_member_application_version', answer.versionId);
	gsa.deleteAttachment(answer.attachmentId);
	if (values.length > 0) {
		var ids = values[values.length - 1].split(',');
		if (ids[1]) {
			answer.attachmentId = ids[1];
		} else {
			answer = this.processError(answer, 'Unrecognizable response from attachment copy: ' + JSON.stringify(values));
		}
	} else {
		answer = this.processError(answer, 'Unrecognizable response from attachment copy: ' +  JSON.stringify(values));
	}

	return answer;
}

This has to be the source of the problem. The copy method the GlideSysAttachment object doesn’t allow you to select what to copy; it arbitrarily copies all attachments from one record to another and returns an array of sys_id pairs (before and after for each attachment). The code above assumed that the last pair contained the sys_id that we were looking for, but apparently, that is not always the case. It looks like we need to examine every sys_id pair in the array, select the one that contains the XML file, grab that sys_id, and then delete all of the other attachments from the version record. That would mean replacing this:

var ids = values[values.length - 1].split(',');
if (ids[1]) {
	answer.attachmentId = ids[1];
}

… with this:

var origId = answer.attachmentId;
for (var i=0; i<values.length; i++) {
	var ids = values[i].split(',');
	if (ids[0] == origId) {
		answer.attachmentId = ids[1];
		gsa.deleteAttachment(origId);
	} else {
		gsa.deleteAttachment(ids[1]);
	}
}

Basically, this code loops through all of the sys_id pairs, looks for the one where the first sys_id matches the original, grabs the second sys_id of that pair for the new answer.attachmentId value, and then deletes the original attachment record. When the first sys_id does not match, then it deletes the copied attachment from the version record, as we did not want to copy that one anyway. We will have to do a little testing to prove this out, but hopefully this will resolve this issue.

Next time, we should have a new Update Set available with this, and a few other, minor corrections in it, and then we can do a little retesting and see if that resolves a few of these issues. As always, if anyone finds anything else that we need to address, please leave the details in the comments section below. All feedback is heartily welcomed!

Collaboration Store, Part XLV

“Progress isn’t made by early risers. It’s made by lazy men trying to find easier ways to do something.”
Robert Heinlein

Last time, we completed the code to sync up the version records, so now all we have left to do is to do basically the same thing for the Update Set XML files attached to the version records. The attachments are a slightly different animal, though, so we can’t just clone what we have done before as we were able to do when using the application record process as a basis for creating the version record process. For one thing, the relationship between instances and their applications is one-to-many, just as the relationship between applications and their various versions is one-to-many. Theoretically, there should only be one attached Update Set XML file for any given version. Technically, there is nothing stopping someone from attaching additional files to a version record, and in the future we may, in fact, want to attach things to a version record like release notes or user guides, but there should only be one Update Set for each version.

One thing that I ended up doing was to go back into the syncVersions function and add in the capability to send the sys_id of the version record to the syncAttachments function. Because the sys_attachments table uses both a table_name and a table_sys_id property to link to the attachment to its record, I couldn’t really do any dot-walking to come up with a query to find the right attachment record. So I added a versionIdList in addition to the versionList to capture the sys_id of the version record along with the version number. The updated version of the syncVersions function now looks like this:

syncVersions: function(targetGR, thisApplication, thisInstance, remoteAppId) {
	var versionList = [];
	var versionIdList = [];
	var versionGR = new GlideRecord('x_11556_col_store_member_application_version');
	versionGR.addQuery('member_application.name', thisApplication);
	versionGR.addQuery('member_application.provider.instance', thisInstance);
	versionGR.query();
	while (versionGR.next()) {
		versionList.push(versionGR.getDisplayValue('version'));
		versionIdList.push(versionGR.getUniqueValue());
	}
	if (versionList.length > 0) {
		var request  = new sn_ws.RESTMessageV2();
		request.setHttpMethod('get');
		request.setBasicAuth(this.CSU.WORKER_ROOT + targetGR.getDisplayValue('instance'), targetGR.getDisplayValue('token'));
		request.setRequestHeader("Accept", "application/json");
		request.setEndpoint('https://' + targetGR.getDisplayValue('instance') + '.service-now.com/api/now/table/x_11556_col_store_member_application_version?sysparm_fields=version%2Csys_id&sysparm_query=member_application%3D' + remoteAppId);
		var response = request.execute();
		if (response.haveError()) {
			gs.error('InstanceSyncUtils.syncVersions - Error returned from attempt to fetch version list: ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
		} else if (response.getStatusCode() == '200') {
			var jsonString = response.getBody();
			var jsonObject = {};
			try {
				jsonObject = JSON.parse(jsonString);
			} catch (e) {
				gs.error('InstanceSyncUtils.syncVersions - Unparsable JSON string returned from attempt to fetch version list: ' + jsonString);
			}
			if (Array.isArray(jsonObject.result)) {
				for (var i=0; i<versionList.length; i++) {
					var thisVersion = versionList[i];
					var thisVersionId = versionIdList[i];
					var remoteVerId = '';
					for (var j=0; j<jsonObject.result.length && remoteVerId == ''; j++) {
						if (jsonObject.result[j].version == thisVersion) {
							remoteVerId = jsonObject.result[j].sys_id;
						}
					}
					if (remoteVerId == '') {
						remoteVerId = this.sendVersion(targetGR, thisVersion, thisApplication, thisInstance, remoteAppId);
					}
					this.syncAttachments(targetGR, thisVersionId, thisVersion, thisApplication, thisInstance, remoteVerId);
				}
			} else {
				gs.error('InstanceSyncUtils.syncVersions - Invalid response body returned from attempt to fetch version list: ' + response.getBody());
			}
		} else {
			gs.error('InstanceSyncUtils.syncVersions - Invalid HTTP response code returned from attempt to fetch version list: ' + response.getStatusCode());
		}
	} else {
		gs.info('InstanceSyncUtils.syncVersions - No versions to sync for application ' + thisApplication + ' on instance ' + thisInstance);
	}
}

With the sys_id in hand, we could now build a simple query to fetch the attachment record.

var attachmentList = [];
var attachmentGR = new GlideRecord('sys_attachment');
attachmentGR.addQuery('table_name', 'x_11556_col_store_member_application_version');
attachmentGR.addQuery('table_sys_id', thisVersionId);
attachmentGR.addQuery('content_type', 'CONTAINS', 'xml');
attachmentGR.query();
while (attachmentGR.next()) {
	attachmentList.push(attachmentGR.getUniqueValue());
}

I’m still building up a list here, even though there should only be one XML attachment for each version record, but since I copied most of the code from the version function, I just left that in place. In practice, this should always be a list of one item. Assuming we have an attachment (which we always should!), we need to check the target instance to see if they already have it as well. For that, of course, we resort to making another REST API call.

var request  = new sn_ws.RESTMessageV2();
request.setHttpMethod('get');
request.setBasicAuth(this.CSU.WORKER_ROOT + targetGR.getDisplayValue('instance'), targetGR.getDisplayValue('token'));
request.setRequestHeader("Accept", "application/json");
request.setEndpoint('https://' + targetGR.getDisplayValue('instance') + '.service-now.com/api/now/table/sys_attachment?sysparm_fields=sys_id&sysparm_query=table_name%3Dx_11556_col_store_member_application_version%5Etable_sys_id%3D' + remoteVerId + '%5Econtent_typeCONTAINSxml');

Once we build the request, we execute it and check the results.

var response = request.execute();
if (response.haveError()) {
	gs.error('InstanceSyncUtils.syncAttachments - Error returned from attempt to fetch attachment list: ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
} else if (response.getStatusCode() == '200') {
	var jsonString = response.getBody();
	var jsonObject = {};
	try {
		jsonObject = JSON.parse(jsonString);
	} catch (e) {
		gs.error('InstanceSyncUtils.syncAttachments - Unparsable JSON string returned from attempt to fetch attachment list: ' + jsonString);
	}
	if (Array.isArray(jsonObject.result)) {
		if (jsonObject.result.length == 0) {
			this.sendAttachment(targetGR, attachmentList[0], remoteVerId, thisVersion, thisApplication);
		}
	} else {
		gs.error('InstanceSyncUtils.syncAttachments - Invalid response body returned from attempt to fetch attachment list: ' + response.getBody());
	}
} else {
	gs.error('InstanceSyncUtils.syncAttachments - Invalid HTTP response code returned from attempt to fetch attachment list: ' + response.getStatusCode());
}

All together, the new syncAttachments function looks like this:

syncAttachments: function(targetGR, thisVersionId, thisVersion, thisApplication, thisInstance, remoteVerId) {
	var attachmentList = [];
	var attachmentGR = new GlideRecord('sys_attachment');
	attachmentGR.addQuery('table_name', 'x_11556_col_store_member_application_version');
	attachmentGR.addQuery('table_sys_id', thisVersionId);
	attachmentGR.addQuery('content_type', 'CONTAINS', 'xml');
	attachmentGR.query();
	while (attachmentGR.next()) {
		attachmentList.push(attachmentGR.getUniqueValue());
	}
	if (attachmentList.length > 0) {
		var request  = new sn_ws.RESTMessageV2();
		request.setHttpMethod('get');
		request.setBasicAuth(this.CSU.WORKER_ROOT + targetGR.getDisplayValue('instance'), targetGR.getDisplayValue('token'));
		request.setRequestHeader("Accept", "application/json");
		request.setEndpoint('https://' + targetGR.getDisplayValue('instance') + '.service-now.com/api/now/table/sys_attachment?sysparm_fields=sys_id&sysparm_query=table_name%3Dx_11556_col_store_member_application_version%5Etable_sys_id%3D' + remoteVerId + '%5Econtent_typeCONTAINSxml');
		var response = request.execute();
		if (response.haveError()) {
			gs.error('InstanceSyncUtils.syncAttachments - Error returned from attempt to fetch attachment list: ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
		} else if (response.getStatusCode() == '200') {
			var jsonString = response.getBody();
			var jsonObject = {};
			try {
				jsonObject = JSON.parse(jsonString);
			} catch (e) {
				gs.error('InstanceSyncUtils.syncAttachments - Unparsable JSON string returned from attempt to fetch attachment list: ' + jsonString);
			}
			if (Array.isArray(jsonObject.result)) {
				if (jsonObject.result.length == 0) {
					this.sendAttachment(targetGR, attachmentList[0], remoteVerId, thisVersion, thisApplication);
				}
			} else {
				gs.error('InstanceSyncUtils.syncAttachments - Invalid response body returned from attempt to fetch attachment list: ' + response.getBody());
			}
		} else {
			gs.error('InstanceSyncUtils.syncAttachments - Invalid HTTP response code returned from attempt to fetch attachment list: ' + response.getStatusCode());
		}
	} else {
		gs.info('InstanceSyncUtils.syncAttachments - No attachments to sync for version ' + thisVersionId + ' of application ' + thisApplication + ' on instance ' + thisInstance);
	}
}

If there was no matching attachment on the target instance, we need to send it over, which we accomplish with a new sendAttachment function modeled after the earlier sendVersion function.

sendAttachment: function(targetGR, attachmentId, remoteVerId, thisVersion, thisApplication) {
	var attachmentGR = new GlideRecord('sys_attachment');
	if (attachmentGR.get(attachmentId)) {
		gs.info('InstanceSyncUtils.sendAttachment - Sending attachment ' + attachmentGR.getDisplayValue('file_name') + ' from version ' + thisVersion + ' of application ' + thisApplication + ' to target instance ' + targetGR.getDisplayValue('instance'));
		var result = this.CSU.pushAttachment(attachmentGR, targetGR, remoteVerId);
		if (result.error) {
			gs.error('InstanceSyncUtils.sendAttachment - Error occurred attempting to push attachment ' + attachmentGR.getDisplayValue('file_name') + ' from version ' + thisVersion + ' of application ' + thisApplication + ' to target instance ' + targetGR.getDisplayValue('instance') + ': ' + result.errorCode + '; ' + result.errorMessage);
		} else if (result.status != 200 && result.status != 201) {
			gs.error('InstanceSyncUtils.sendAttachment - Invalid HTTP response code returned from attempt to push attachment ' + attachmentGR.getDisplayValue('file_name') + ' from version ' + thisVersion + ' of application ' + thisApplication + ' to target instance ' + targetGR.getDisplayValue('instance') + ': ' + result.status);
		} else if (!result.obj) {
			gs.error('InstanceSyncUtils.sendAttachment - Invalid response body returned from attempt to push attachment ' + attachmentGR.getDisplayValue('file_name') + ' from version ' + thisVersion + ' of application ' + thisApplication + ' to target instance ' + targetGR.getDisplayValue('instance') + ': ' + result.body);
		}
	}
}

That completes all of the coding for the process of syncing all of the Client instances with the Host instance. If all is working correctly, it should ensure that all member instances have the same list of instances, the same list of applications from each contributing instance, the same list of versions for each application, and the associated Update Set XML file attachment for each version. Of course, there is still quite a bit of testing to do to make sure that all of this works as intended, so I will put together yet another Update Set and talk about how folks can help test all of this out in our next installment.

Collaboration Store, Part XLIV

“I am so clever that sometimes I don’t understand a single word of what I am saying.”
Oscar Wilde

Last time, we completed the shared functions in the CollaborationStoreUtils Script Include that will push the various record types from one instance to another. These functions will work to push data from a Client instance to the Host as well as push data from the Host to any of the Clients. Now that we have those in place, we can get back to the business of syncing up the Client instances with the Host. We have already built the code to sync up the list of instances, as well as the code to sync up the applications for each instance. Now we need to create the code to sync up the versions for each application. We will use the same approach that we used for all of the others, and much of the code will look very similar to that which we have already put into place.

Once again, we will start by gathering up all of the records associated with the active higher-level record, fetching the versions associated with the current application just as we previously gathered up all of the applications associated with the providing instance. Other than the table and query parameters, this should look quite familiar to those of you who have been following along at home.

var versionList = [];
var versionGR = new GlideRecord('x_11556_col_store_member_application_version');
versionGR.addQuery('member_application.name', thisApplication);
versionGR.addQuery('member_application.provider.instance', thisInstance);
versionGR.query();
while (versionGR.next()) {
	versionList.push(versionGR.getDisplayValue('version'));
}

If there are version records present on the Host, then we need to use the REST API to go gather up the same list from the target Client. As usual, we start out by building our RESTMessageV2 request object.

var request  = new sn_ws.RESTMessageV2();
request.setHttpMethod('get');
request.setBasicAuth(this.CSU.WORKER_ROOT + targetGR.getDisplayValue('instance'), targetGR.getDisplayValue('token'));
request.setRequestHeader("Accept", "application/json");
request.setEndpoint('https://' + targetGR.getDisplayValue('instance') + '.service-now.com/api/now/table/x_11556_col_store_member_application_version?sysparm_fields=version%2Csys_id&sysparm_query=member_application%3D' + remoteAppId);

Once we have the request constructed, we execute it and check the response.

var response = request.execute();
if (response.haveError()) {
	gs.error('InstanceSyncUtils.syncVersions - Error returned from attempt to fetch version list: ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
} else if (response.getStatusCode() == '200') {
	var jsonString = response.getBody();
	var jsonObject = {};
	try {
		jsonObject = JSON.parse(jsonString);
	} catch (e) {
		gs.error('InstanceSyncUtils.syncVersions - Unparsable JSON string returned from attempt to fetch version list: ' + jsonString);
	}
	if (Array.isArray(jsonObject.result)) {
		...
	} else {
		gs.error('InstanceSyncUtils.syncVersions - Invalid response body returned from attempt to fetch version list: ' + response.getBody());
	}
} else {
	gs.error('InstanceSyncUtils.syncVersions - Invalid HTTP response code returned from attempt to fetch version list: ' + response.getStatusCode());
}

Assuming all went well, the last thing that we need to do is to loop through all of the versions present on the Host and see if they are also present on the Client. If not, then we need to send them over, which we can now do with our new shared functions in CollaborationStoreUtils. And of course, whether we had to push the version over or not, we will need to check to see if the version record has the attached Update Set XML file.

for (var i=0; i<versionList.length; i++) {
	var thisVersion = versionList[i];
	var remoteVerId = '';
	for (var j=0; j<jsonObject.result.length && remoteVerId == ''; j++) {
		if (jsonObject.result[j].version == thisVersion) {
			remoteVerId = jsonObject.result[j].sys_id;
		}
	}
	if (remoteVerId == '') {
		remoteVerId = this.sendVersion(targetGR, thisApplication, thisInstance, thisVersion, remoteAppId);
	}
	this.syncAttachments(targetGR, thisVersion, remoteVerId);
}

Putting it all together, the entire function looks like this:

syncVersions: function(targetGR, thisApplication, thisInstance, remoteAppId) {
	var versionList = [];
	var versionGR = new GlideRecord('x_11556_col_store_member_application_version');
	versionGR.addQuery('member_application.name', thisApplication);
	versionGR.addQuery('member_application.provider.instance', thisInstance);
	versionGR.query();
	while (versionGR.next()) {
		versionList.push(versionGR.getDisplayValue('version'));
	}
	if (versionList.length > 0) {
		var request  = new sn_ws.RESTMessageV2();
		request.setHttpMethod('get');
		request.setBasicAuth(this.CSU.WORKER_ROOT + targetGR.getDisplayValue('instance'), targetGR.getDisplayValue('token'));
		request.setRequestHeader("Accept", "application/json");
		request.setEndpoint('https://' + targetGR.getDisplayValue('instance') + '.service-now.com/api/now/table/x_11556_col_store_member_application_version?sysparm_fields=version%2Csys_id&sysparm_query=member_application%3D' + remoteAppId);
		var response = request.execute();
		if (response.haveError()) {
			gs.error('InstanceSyncUtils.syncVersions - Error returned from attempt to fetch version list: ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
		} else if (response.getStatusCode() == '200') {
			var jsonString = response.getBody();
			var jsonObject = {};
			try {
				jsonObject = JSON.parse(jsonString);
			} catch (e) {
				gs.error('InstanceSyncUtils.syncVersions - Unparsable JSON string returned from attempt to fetch version list: ' + jsonString);
			}
			if (Array.isArray(jsonObject.result)) {
				for (var i=0; i<versionList.length; i++) {
					var thisVersion = versionList[i];
					var remoteVerId = '';
					for (var j=0; j<jsonObject.result.length && remoteVerId == ''; j++) {
						if (jsonObject.result[j].version == thisVersion) {
							remoteVerId = jsonObject.result[j].sys_id;
						}
					}
					if (remoteVerId == '') {
						remoteVerId = this.sendVersion(targetGR, thisApplication, thisInstance, thisVersion, remoteAppId);
					}
					this.syncAttachments(targetGR, thisVersion, remoteVerId);
				}
			} else {
				gs.error('InstanceSyncUtils.syncVersions - Invalid response body returned from attempt to fetch version list: ' + response.getBody());
			}
		} else {
			gs.error('InstanceSyncUtils.syncVersions - Invalid HTTP response code returned from attempt to fetch version list: ' + response.getStatusCode());
		}
	} else {
		gs.info('InstanceSyncUtils.syncVersions - No versions to sync for application ' + thisApplication + ' on instance ' + thisInstance);
	}
}

Now we have referenced two new functions that will need to built out as well, sendVersion and syncAttachments. The sendVersion function will be pretty much a clone of the sendApplication function that we already built, and it will call our new pushVersion function in CollaborationStoreUtils.

sendVersion: function(targetGR, thisApplication, thisInstance, thisVersion, remoteAppId) {
	var sysId = '';

	var versionGR = new GlideRecord('x_11556_col_store_member_application_version');
	versionGR.addQuery('member_application.name', thisApplication);
	versionGR.addQuery('member_application.provider.instance', thisInstance);
	versionGR.addQuery('version', thisVersion);
	versionGR.query();
	if (versionGR.next()) {
		gs.info('InstanceSyncUtils.sendApplication - Sending version ' + thisVersion + ' of application ' + versionGR.getDisplayValue('member_application') + ' to target instance ' + targetGR.getDisplayValue('instance'));
		var result = this.CSU.pushVersion(versionGR, targetGR, remoteAppId);
		if (result.error) {
			gs.error('InstanceSyncUtils.sendVersion - Error occurred attempting to push version ' + thisVersion + ' of application ' + remoteAppId + ' : ' + result.errorCode + '; ' + result.errorMessage);
		} else if (result.status != 200 && result.status != 201) {
			gs.error('InstanceSyncUtils.sendVersion - Invalid HTTP response code returned from attempt to push version ' + thisVersion + ' of application ' + remoteAppId + ' : ' + result.status);
		} else if (!result.obj) {
			gs.error('InstanceSyncUtils.sendVersion - Invalid response body returned from attempt to push version ' + thisVersion + ' of application ' + remoteAppId + ' : ' + result.body);
		} else {
			sysId = result.obj.sys_id;
		}
	}

	return sysId;
}

The other missing referenced function is syncAttachments. This one will be similar to the syncVersions function above, but because we will be working with attached files, there will be some minor differences in the tools that we will use to make all of that happen. We will get into all of that next time out.

Collaboration Store, Part XXXIII

“The only impossible journey is the one you never begin.”
Tony Robbins

Last time, we completed the construction of the process that will send out a new version of an application out to all of the instances in the community. Now we need to come up with a mechanism to kick this process off whenever a new version of an application is published to the Host instance. The simplest way to do that would be to create a new Business Rule on one of the tables housing the artifacts of the new application. To make sure that all of the application artifacts have made their way to the Host, the safest table to target would be the Attachment table, since that is the last artifact to be saved or sent to the Host. Unfortunately, that table is used for all kinds of things, so we will need to make sure that only records attached to the application version table trigger the rule.

To create a new Business Rule, navigate to System Definition > Business Rules and click the New button to bring up the data entry form.

New Business Rule input form

We’ll call our new rule Distribute New Version, associate it with the sys_attachment table, run it async on insert, and check the Advanced checkbox so that we can include a script to run when the rule is triggered.

New Distribute New Version Business Rule

To limit the impact of the rule to just the inserts that we want, we need to add a couple of Filter Conditions, one for the Table and another for the Content Type.

Business Rule filter conditions

On the Advanced tab, we will want to add a Condition that will ensure that this rule will only run on a Host instance. The easiest way to do that is to simply compare the stock instance_name property with the scoped host_instance property. Those two values will only be the same on a Host instance.

gs.getProperty('instance_name') == gs.getProperty('x_11556_col_store.host_instance')

The other thing that we need to do on the Advanced tab is to enter the script that will launch our Subflow. The original way to do that was to utilize the FlowAPI, which is the way that I had coded it initially.

sn_fd.FlowAPI.startSubflow('Distribute_New_Application_Version', {new_version: current.getDisplayValue('table_sys_id'), attachment_id: current.getUniqueValue()});

However, later I decided that it was high time to start embracing the new, preferred approach, which is to use the ScriptableFlowRunner. The code for that is a little more complicated, but at some point the older way of doing this will undoubtedly go away, so we might as well start transitioning things in that direction. Here is the script for the Business Rule using this newer technique:

(function executeRule(current, previous) {
	try {
		var result = sn_fd.FlowAPI.getRunner()
			.subflow('x_11556_col_store.Distribute_New_Application_Version')
			.inBackground()
			.withInputs({new_version: current.getDisplayValue('table_sys_id'), attachment_id: current.getUniqueValue()})
			.run();
	} catch (e) {
		gs.error('Business Rule Distribute New Version - Error running subflow Distribute_New_Application_Version: ' + e.getMessage());
	}
})(current, previous);

That completes the Business Rule, so all we need to do at this point is to Save it and now the next time that an XML attachment is created for a version record, our new Subflow will kick off, distributing the new version of the application to all instances in the community (except for the Host and the instance that produced the application). This also completes the application distribution process, so now it is time to release yet another Update Set so that those of you who have been nice enough to do a little testing can give it a go.

If you have just jumped into this and have not been following along, here is what you will need in order to install the app and try it out:

  • If you have not done so already, you will need to install the latest version of snh-form-fields, which you can find here.
  • You will also need to install this additional global component, which is not included in the scoped application.
  • Once you have successfully installed the prerequisites, you will then need to install the scoped application from this Update Set.
  • After the scoped application has been installed, using an admin account, navigate to Collaboration Store -> Collaboration Store Set-up and complete the set-up process for either a Host or Client instance. The Host instance will need to be set up first, as the set-up for a Client instance requires successful contact with a functioning Host.

And again, as I mentioned last time, in order to test out the distribution process, you will need three or more instances in your community so that one of the Client instances can publish the application to the Host instance and then the Host instance can distribute the new version out to all of the other Client instances. As always, all feedback is welcome and appreciated, and if you are able to get everything to actually work, please let me know in the comments. Thanks!

Next time, we will take a look at where we go from here. Thanks again to all of you who have taken the time to help me out and have let me know what you have found.

Collaboration Store, Part XXXI

“Someone’s sitting in the shade today because someone planted a tree a long time ago.”
Warren Buffett

While we wait for additional test results to come in from the corrected Update Set for our latest iteration, it would be a good time to take a look at what we will need to do to complete the last remaining step in the application publishing process. What we have completed so far pushes all of the new application version artifacts to the Host instance. Now we need to build out the process for the Host instance to send out all of those artifacts to all of the other Client instances. Fortunately, we already have in hand most of the code to do that; we just need to clone a few things and make a number of modifications to fit the new purpose.

We have already built a Subflow to push new Client instance information out to all of the existing Client instances during the new instance registration process. Pushing out a new version of an application to those same instances is essentially the same process, so we should be able to clone that guy, and the associated Action, to create our new process. We have also created functions to push over the application record, the version record, and the attached Update Set XML file, and we should be able to copy over those guys as well to complete the process. In fact, we should probably start with the JavaScript functions, as we will need to call the root function in the Action, which will need to be created before it can be referenced in the Subflow.

The functions as written are hard-coded to send everything over to the Host instance. We will want to do basically the same thing, but this time we will be sending things out to various Client instances. Ideally, I should just refactor the code to have the destination instance and credentials passed in so that one function would work in both circumstances, but at this point it was less complicated to just make copies of each function and hack them up for their new purpose. I don’t really like having two copies of essentially the same thing, though, so one day I am going to have to go back and rework the entire mess.

But for now, this is what I came up with when I copied what was once called processPhase5 to create the new publishNewVersion function:

publishNewVersion: function(newVersion, targetInstance, attachmentId) {
	var targetGR = new GlideRecord('x_11556_col_store_member_organization');
	if (targetGR.get('instance', targetInstance)) {
		var token = targetGR.getDisplayValue('token');
		var versionGR = new GlideRecord('x_11556_col_store_member_application_version');
		if (versionGR.get(newVersion)) {
			var canContinue = true;
			var targetAppId = '';
			var mbrAppGR = versionGR.member_application.getRefRecord();
			var request  = new sn_ws.RESTMessageV2();
			request.setHttpMethod('get');
			request.setBasicAuth(this.WORKER_ROOT + targetInstance, token);
			request.setRequestHeader("Accept", "application/json");
			request.setEndpoint('https://' + targetInstance + '.service-now.com/api/now/table/x_11556_col_store_member_application?sysparm_fields=sys_id&sysparm_query=provider.instance%3D' + mbrAppGR.getDisplayValue('provider.instance') + '%5Ename%3D' + encodeURIComponent(mbrAppGR.getDisplayValue('name')));
			var response = request.execute();
			if (response.haveError()) {
				gs.error('CollaborationStoreUtils.publishNewVersion: Error returned from Target instance ' + targetInstance + ': ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
				canContinue = false;
			} else if (response.getStatusCode() == '200') {
				var jsonString = response.getBody();
				var jsonObject = {};
				try {
					jsonObject = JSON.parse(jsonString);
				} catch (e) {
					gs.error('CollaborationStoreUtils.publishNewVersion: Unparsable JSON string returned from Target instance ' + targetInstance + ': ' + jsonString);
					canContinue = false;
				}
				if (canContinue) {
					var payload = {};
					payload.name = mbrAppGR.getDisplayValue('name');
					payload.description = mbrAppGR.getDisplayValue('description');
					payload.current_version = mbrAppGR.getDisplayValue('current_version');
					payload.active = 'true';
					request  = new sn_ws.RESTMessageV2();
					request.setBasicAuth(this.WORKER_ROOT + targetInstance, token);
					request.setRequestHeader("Accept", "application/json");
					if (jsonObject.result && jsonObject.result.length > 0) {
						targetAppId = jsonObject.result[0].sys_id;
						request.setHttpMethod('put');
						request.setEndpoint('https://' + targetInstance + '.service-now.com/api/now/table/x_11556_col_store_member_application/' + targetAppId);
					} else {
						request.setHttpMethod('post');
						request.setEndpoint('https://' + targetInstance + '.service-now.com/api/now/table/x_11556_col_store_member_application');
						payload.provider = mbrAppGR.getDisplayValue('provider.instance');
					}
					request.setRequestBody(JSON.stringify(payload, null, '\t'));
					response = request.execute();
					if (response.haveError()) {
						gs.error('CollaborationStoreUtils.publishNewVersion: Error returned from Target instance ' + targetInstance + ': ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
						canContinue = false;
					} else if (response.getStatusCode() != 200 && response.getStatusCode() != 201) {
						gs.error('CollaborationStoreUtils.publishNewVersion: Invalid HTTP Response Code returned from Target instance ' + targetInstance + ': ' + response.getStatusCode());
						canContinue = false;
					} else {
						jsonString = response.getBody();
						jsonObject = {};
						try {
							jsonObject = JSON.parse(jsonString);
						} catch (e) {
							gs.error('CollaborationStoreUtils.publishNewVersion: Unparsable JSON string returned from Target instance ' + targetInstance + ': ' + jsonString);
							canContinue = false;
						}
						if (canContinue) {
							targetAppId = jsonObject.result.sys_id;
						}
					}
				}
			} else {
				gs.error('CollaborationStoreUtils.publishNewVersion: Invalid HTTP Response Code returned from Target instance ' + targetInstance + ': ' + response.getStatusCode());
			}
			if (canContinue) {
				this.publishVersionRecord(targetInstance, token, versionGR, targetAppId, attachmentId);
			}
		} else {
			gs.error('CollaborationStoreUtils.publishNewVersion: Version record not found: ' + newVersion);
		}
	} else {
		gs.error('CollaborationStoreUtils.publishNewVersion: Target instance record not found: ' + targetInstance);
	}
},

Unlike the original application publication process, we are not bouncing back and forth between the client side and the server side, so with the successful completion of one artifact, we can simply move on to pushing over the next artifact. To create the next step in the process, the publishVersionRecord function, I started out with a copy of the original processPhase6 function. Here is the end result:

publishVersionRecord: function(targetInstance, token, versionGR, targetAppId, attachmentId) {
	var canContinue = true;
	var payload = {};
	payload.member_application = targetAppId;
	payload.version = versionGR.getDisplayValue('version');
	var request  = new sn_ws.RESTMessageV2();
	request.setBasicAuth(this.WORKER_ROOT + targetInstance, token);
	request.setRequestHeader("Accept", "application/json");
	request.setHttpMethod('post');
	request.setEndpoint('https://' + targetInstance + '.service-now.com/api/now/table/x_11556_col_store_member_application_version');
	request.setRequestBody(JSON.stringify(payload, null, '\t'));
	response = request.execute();
	if (response.haveError()) {
		gs.error('CollaborationStoreUtils.publishVersionRecord: Error returned from Target instance ' + targetInstance + ': ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
		canContinue = false;
	} else if (response.getStatusCode() != 201) {
		gs.error('CollaborationStoreUtils.publishVersionRecord: Invalid HTTP Response Code returned from Target instance ' + targetInstance + ': ' + response.getStatusCode());
		canContinue = false;
	} else {
		jsonString = response.getBody();
		jsonObject = {};
		try {
			jsonObject = JSON.parse(jsonString);
		} catch (e) {
			gs.error('CollaborationStoreUtils.publishVersionRecord: Unparsable JSON string returned from Target instance ' + targetInstance + ': ' + jsonString);
			canContinue = false;
		}
		if (canContinue) {
			targetVerId = jsonObject.result.sys_id;
			this.publishVersionAttachment(targetInstance, token, targetVerId, attachmentId);
		}
	}
},

The last step in the process, then, is to send over the Update Set XML file attachment. For that one, I started out with the processPhase7 function and ended up with this:

publishVersionAttachment: function(targetInstance, token, targetVerId, attachmentId) {
	var gsa = new GlideSysAttachment();
	var sysAttGR = new GlideRecord('sys_attachment');
	if (sysAttGR.get(attachmentId)) {
		var url = 'https://';
		url += targetInstance;
		url += '.service-now.com/api/now/attachment/file?table_name=x_11556_col_store_member_application_version&table_sys_id=';
		url += targetVerId;
		url += '&file_name=';
		url += sysAttGR.getDisplayValue('file_name');
		var request  = new sn_ws.RESTMessageV2();
		request.setBasicAuth(this.WORKER_ROOT + targetInstance, token);
		request.setRequestHeader('Content-Type', sysAttGR.getDisplayValue('content_type'));
		request.setRequestHeader('Accept', 'application/json');
		request.setHttpMethod('post');
		request.setEndpoint(url);
		request.setRequestBody(gsa.getContent(sysAttGR));
		response = request.execute();
		if (response.haveError()) {
			gs.error('CollaborationStoreUtils.publishVersionAttachment: Error returned from Target instance ' + targetInstance + ': ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
		} else if (response.getStatusCode() != 201) {
			gs.error('CollaborationStoreUtils.publishVersionAttachment: Invalid HTTP Response Code returned from Target instance ' + targetInstance + ': ' + response.getStatusCode());
		}
	} else {
		gs.error('CollaborationStoreUtils.publishVersionAttachment: Invalid attachment record sys_id: ' + attachmentId);
	}
},

So that takes care of the functions. Now we have to create an Action in the Flow Designer that will call the function, and then produce a new Subflow that will leverage that action. That sounds like a good subject for our next installment.

Special Note to Testers

If you want to test the set-up process, you can do that with a single instance. You just have to select the Host option during the set-up process. If you want to test the Client set-up process, you will need at least two instances, one to serve as the Host, which you will need to reference when you set up the Client (you cannot be a Client of your own Host). But if you really want to test out the full registration process, including the Subflow that informs all existing instances of the new instance, you will need at least three participating instances: 1) the Host instance, 2) the new Client instance, and 3) an existing Client instance that will be notified of the new member of the community.

The same holds true for the application publishing process. You can test a portion of the publishing process by publishing an app on a single Host instance, but if you want to test out all of the parts and pieces, you will want to publish the app from a Client instance and make sure that it makes its way all the way back to the Host. Once we complete the application distribution process, full testing will require at least three instances to verify that the Host distributes the new version of the application to other Clients.

Collaboration Store, Part XXVIII

“The best way out is always through.”
Robert Frost

Now that we have completed the functions that send the application record and version record over to the Host instance, the last thing that we need to do is to send over the Update Set XML file attached to the version record. For sending over an attachment, we will need to use the attachment REST API instead of the standard table REST API. Since the contents of the file that we are sending over is plain text, we can use a little hackery to bypass the need to send over an actual file and just place the text content in the body of the request. But first, as usual, we need to grab the GlideRecord that we want to send before we do anything else.

var gsa = new GlideSysAttachment();
var sysAttGR = new GlideRecord('sys_attachment');
if (sysAttGR.get(answer.attachmentId)) {
	...
} else {
	answer = this.processError(answer, 'Invalid attachment record sys_id: ' + answer.attachmentId);
}

Once we have obtained the record, we will need to grab the usual suspects from the System Properties and then construct the appropriate URL for the request.

var host = gs.getProperty('x_11556_col_store.host_instance');
var token = gs.getProperty('x_11556_col_store.active_token');
var url = 'https://';
url += host;
url += '.service-now.com/api/now/attachment/file?table_name=x_11556_col_store_member_application_version&table_sys_id=';
url += answer.hostVerId;
url += '&file_name=';
url += sysAttGR.getDisplayValue('file_name');

Next, we will need to construct and configure our sn_ws.RESTMessageV2 object.

var request  = new sn_ws.RESTMessageV2();
request.setBasicAuth(this.WORKER_ROOT + host, token);
request.setRequestHeader('Content-Type', sysAttGR.getDisplayValue('content_type'));
request.setRequestHeader('Accept', 'application/json');
request.setHttpMethod('post');
request.setEndpoint(url);
request.setRequestBody(gsa.getContent(sysAttGR));

Two things to note on this one: 1) the Content-Type header sets the value of the content_type property of the target instance sys_attachment record, and 2) we use our old friend, the GlideSysAttachment utility to obtain the actual XML of the attachment in lieu of a real file (the file contents are not actually a part of the GlideRecord for the sys_attachment table, hence the need to utilize GlideSysAttachment).

Now all we need to do is to execute the request and check the response.

var response = request.execute();
if (response.haveError()) {
	answer = this.processError(answer, 'Error returned from Host instance: ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
} else if (response.getStatusCode() != 201) {
	answer = this.processError(answer, 'Invalid HTTP Response Code returned from Host instance: ' + response.getStatusCode());
}

Unlike the application and version records, we have no need for any information from the returned JSON string, so there is no need to attempt to parse it and pull out any data. As long as there are no errors, we are good to go.

Putting it all together, the entire function looks like this:

processPhase7: function(answer) {
	var gsa = new GlideSysAttachment();
	var sysAttGR = new GlideRecord('sys_attachment');
	if (sysAttGR.get(answer.attachmentId)) {
		var host = gs.getProperty('x_11556_col_store.host_instance');
		var token = gs.getProperty('x_11556_col_store.active_token');
		var url = 'https://';
		url += host;
		url += '.service-now.com/api/now/attachment/file?table_name=x_11556_col_store_member_application_version&table_sys_id=';
		url += answer.hostVerId;
		url += '&file_name=';
		url += sysAttGR.getDisplayValue('file_name');
		var request  = new sn_ws.RESTMessageV2();
		request.setBasicAuth(this.WORKER_ROOT + host, token);
		request.setRequestHeader('Content-Type', sysAttGR.getDisplayValue('content_type'));
		request.setRequestHeader('Accept', 'application/json');
		request.setHttpMethod('post');
		request.setEndpoint(url);
		request.setRequestBody(gsa.getContent(sysAttGR));
		var response = request.execute();
		if (response.haveError()) {
			answer = this.processError(answer, 'Error returned from Host instance: ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
		} else if (response.getStatusCode() != 201) {
			answer = this.processError(answer, 'Invalid HTTP Response Code returned from Host instance: ' + response.getStatusCode());
		}
	} else {
		answer = this.processError(answer, 'Invalid attachment record sys_id: ' + answer.attachmentId);
	}

	return answer;
},

Unfortunately, when I took it out for a spin, it didn’t work. When I was nosing around to see what the source of the problem was, I figured out that the sys_id value that I was using for the attachment GlideRecord was not a string, but an array of comma-separated sys_id pairs, one for the original attachment and one for the copied attachment. This value came out of the fourth step, where we copied the attachment from the original scoped application record to the version record. Once I realized the actual format of the data returned from the GlideSysAttachment copy function, I did a little rewriting of the processPhase4 function to accommodate the actual structure of the returned data.

processPhase4: function(answer) {
	var gsa = new GlideSysAttachment();
	var values = gsa.copy('sys_app', answer.appSysId, 'x_11556_col_store_member_application_version', answer.versionId);
	gsa.deleteAttachment(answer.attachmentId);
	if (values[0]) {
		var ids = values[0].split(',');
		if (ids[1]) {
			answer.attachmentId = ids[1];
		} else {
			answer = this.processError(answer, 'Unrecognizable response from attachment copy: ' + values);
		}
	} else {
		answer = this.processError(answer, 'Unrecognizable response from attachment copy: ' + values);
	}
		
	return answer;
},

While I was at it, I went ahead and did a little work on the processError function to add a few diagnostic breadcrumbs to the system log whenever there is an error. That function now looks like this:

processError: function(answer, message) {
	gs.info('ApplicationPublisher.processError: ' + message);
	gs.info('ApplicationPublisher.processError: ' + JSON.stringify(answer));
	gs.addErrorMessage(message);
	answer.error = message;
	return answer;
},

Once I straightened all of that out, everything finally worked as intended. This is the last step in the process, so this essentially completes the code for publishing an application to the Collaboration Store. At this point, I should probably cut another Update Set so that the folks who would like to participate in the testing can take things out for a little test drive. I still need to address the issues with the set-up process uncovered by the last round of testing, so I think I will take that on next time out and then release a new Update Set for those of you who are willing to put things through their paces and report your results. As always, all feedback is very much appreciated.

Collaboration Store, Part XXVII

“It does not matter how slowly you go as long as you do not stop.”
Confucius

Last time we pushed the application record to the Host instance and now we have to do basically the same thing with the version record. The only difference really, other than the table and fields, is that a version record will always be a new record, so there is no need to determine if the record exists or not on the Host instance. This simplifies the code quite a bit. We still need to fetch the GlideRecord that will be sent over, so as we did with the application record, this will be the first order of business.

var versionGR = new GlideRecord('x_11556_col_store_member_application_version');
if (versionGR.get(answer.versionId)) {
	...
} else {
	answer = this.processError(answer, 'Invalid version record sys_id: ' + answer.versionId);
}

Once we have the record, we can gather up our System Properties and build the payload for the REST API call.

var host = gs.getProperty('x_11556_col_store.host_instance');
var token = gs.getProperty('x_11556_col_store.active_token');
var payload = {};
payload.member_application = answer.hostAppId;
payload.version = versionGR.getDisplayValue('version');

With our payload in hand, we can now create and configure our sn_ws.RESTMessageV2 object.

var request  = new sn_ws.RESTMessageV2();
request.setBasicAuth(this.WORKER_ROOT + host, token);
request.setRequestHeader("Accept", "application/json");
request.setHttpMethod('post');
request.setEndpoint('https://' + host + '.service-now.com/api/now/table/x_11556_col_store_member_application_version');
request.setRequestBody(JSON.stringify(payload, null, '\t'));

Now all that is left to do is to execute the request and check the response.

response = request.execute();
if (response.haveError()) {
	answer = this.processError(answer, 'Error returned from Host instance: ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
} else if (response.getStatusCode() != 201) {
	answer = this.processError(answer, 'Invalid HTTP Response Code returned from Host instance: ' + response.getStatusCode());
} else {
	var jsonString = response.getBody();
	var jsonObject = {};
	try {
		jsonObject = JSON.parse(jsonString);
	} catch (e) {
		answer = this.processError(answer, 'Unparsable JSON string returned from Host instance: ' + jsonString);
	}
	if (!answer.error) {
		answer.hostVerId = jsonObject.result.sys_id;
	}
}

Well, that was easy! Here’s the whole thing all put together.

processPhase6: function(answer) {
	var versionGR = new GlideRecord('x_11556_col_store_member_application_version');
	if (versionGR.get(answer.versionId)) {
		var host = gs.getProperty('x_11556_col_store.host_instance');
		var token = gs.getProperty('x_11556_col_store.active_token');
		var payload = {};
		payload.member_application = answer.hostAppId;
		payload.version = versionGR.getDisplayValue('version');
		var request  = new sn_ws.RESTMessageV2();
		request.setBasicAuth(this.WORKER_ROOT + host, token);
		request.setRequestHeader("Accept", "application/json");
		request.setHttpMethod('post');
		request.setEndpoint('https://' + host + '.service-now.com/api/now/table/x_11556_col_store_member_application_version');
		request.setRequestBody(JSON.stringify(payload, null, '\t'));
		response = request.execute();
		if (response.haveError()) {
			answer = this.processError(answer, 'Error returned from Host instance: ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
		} else if (response.getStatusCode() != 201) {
			answer = this.processError(answer, 'Invalid HTTP Response Code returned from Host instance: ' + response.getStatusCode());
		} else {
			var jsonString = response.getBody();
			var jsonObject = {};
			try {
				jsonObject = JSON.parse(jsonString);
			} catch (e) {
				answer = this.processError(answer, 'Unparsable JSON string returned from Host instance: ' + jsonString);
			}
			if (!answer.error) {
				answer.hostVerId = jsonObject.result.sys_id;
			}
		}
	} else {
		answer = this.processError(answer, 'Invalid version record sys_id: ' + answer.versionId);
	}

	return answer;
},

That takes care of 6 of the 7 steps. The last one that we will need to do will be to push the Update Set XML attachment over to the Host. That one may be a little more involved, so we will save that for next time out.

Hacking the REST Message API to Fetch a Remote File

“The way to get started is to quit talking and begin doing.”
Walt Disney

The other day I needed to fetch a file that was posted on another web site via its URL and process it. I have processed ServiceNow file attachments before, but I have never attempted to go out to the Internet and pull in a file using an HTTP GET of the URL. There are several ways to do that in Javascript such as XMLHttpRequest or fetch, but none of those seemed to work in server-side code in ServiceNow. But you can open up a URL using the ServiceNow RESTMessageV2 API, so I thought that maybe I would give that a shot. How hard could it be?

I decided to encapsulate everything into a Script Include, mainly so that if I ever needed to do this again, I could call the same function in some other context. My thought was to pass in the URL of the file that I wanted to fetch along with the table name and sys_id of the record to which I wanted the file to be attached, and then have the function fetch the file, attach it, and send back the attachment. Something like this:

var FileFetchUtils = Class.create();
FileFetchUtils.prototype = {
	initialize: function() {
	},

	fetchFileFromUrlAndAttach: function(table, sys_id, url) {
		...
	},

	type: 'FileFetchUtils'
};

That was idea, anyway. Let’s say that I wanted to attach this file, available on the ServiceNow web site, to some Incident:

https://www.servicenow.com/content/dam/servicenow-assets/public/en-us/doc-type/success/playbook/implementation.pdf

The code to do that would look something like this:

var fileFetcher = new FileFetchUtils();
fileFetcher.fetchFileFromUrlAndAttach('incident', incSysId, 'https://www.servicenow.com/content/dam/servicenow-assets/public/en-us/doc-type/success/playbook/implementation.pdf');

All we need to do now is come up with the code needed to do all of the work of fetching the file, attaching it to the specified record, and returning the attachment. To begin, we will need to extract the name of the file from the URL. Assuming that the file name is the last component of the path on the URL, we can do that by splitting the path into its component parts and grabbing the last part.

var parts = url.split('/');
var fileName = parts[parts.length-1];

Next, we will need to create and configure the request object.

var request  = new sn_ws.RESTMessageV2();
request.setHttpMethod('get');
request.setEndpoint(url);

The next thing to do would be to execute the request, but before we do that, we can take advantage of a nice built-in feature that will really simplify this whole operation. There is an available function of the RESTMessageV2 API that allows you to declare your intent to turn the retrieved file into an attachment, which will then handle all of the details of doing that on your behalf when the request is executed. You just need to invoke the function before you execute the request.

request.saveResponseBodyAsAttachment(table, sys_id, fileName);        
response = request.execute();

Although that really makes things super simple, it’s still a good idea to check the HTTP Response Code, just to make sure all went well. If not, it’s a good practice to relay that to the user.

if (response.getStatusCode() == '200') {
	...
} else {
	returnValue = 'Error: Invalid HTTP Response: ' + response.getStatusCode();
}

Now, assuming that things actually did go well and our new attachments was created, we still want to send that back to the calling script as a response to this function. The RESTMessageV2 API saveResponseBodyAsAttachment function does not return the attachment that is created, so we will have to use the table and sys_id to hunt it down. And if we cannot find it for any reason, we will want to report that as well.

var attachmentGR = new GlideRecord('sys_attachment');
attachmentGR.addQuery('table_name', table);
attachmentGR.addQuery('table_sys_id', sys_id);
attachmentGR.orderByDesc('sys_created_on');
attachmentGR.query();
if (attachmentGR.next()) {
	returnValue = attachmentGR.getUniqueValue();
} else {
	returnValue = 'Error: Unable to fetch attachment';
}

That should now be everything that we need to fetch the file, attach it, and send back the attachment. Putting it all together, the entire Script Include looks like this:

var FileFetchUtils = Class.create();
FileFetchUtils.prototype = {
	initialize: function() {
		this.REST_MESSAGE = '19bb0cde2fedd4101a75ad2ef699b6da';
	},

	fetchFileFromUrlAndAttach: function(table, sys_id, url) {
		var returnVlaue = '';
		var parts = url.split('/');
		var fileName = parts[parts.length-1];
		var request  = new sn_ws.RESTMessageV2();
		request.setHttpMethod('get');
		request.setEndpoint(url);
		request.saveResponseBodyAsAttachment(table, sys_id, fileName);        
		response = request.execute();
		if (response.getStatusCode() == '200') {
			var attachmentGR = new GlideRecord('sys_attachment');
			attachmentGR.addQuery('table_name', table);
			attachmentGR.addQuery('table_sys_id', sys_id);
			attachmentGR.orderByDesc('sys_created_on');
			attachmentGR.query();
			if (attachmentGR.next()) {
				returnValue = attachmentGR.getUniqueValue();
			} else {
				returnValue = 'Error: Unable to fetch attachment';
			}
		} else {
			returnValue = 'Error: Invalid HTTP Response: ' + response.getStatusCode();
		}
		return returnValue;
	},

	type: 'FileFetchUtils'
};

Now all we need to do is find an Incident to use for testing and use the background scripts feature to give it a try. First, we’ll need to pull up an Incident and then use the Copy sys_id option of the hamburger drop-down menu to snag the sys_id of the Incident.

Grabbing the sys_id of an Incident

Now we can pop over to the background script processor and enter our code, using the sys_id that we pulled from the selected Incident.

Testing using a background script

After running the script, we can return to the Incident to verify that the file from the specified URL is now attached to the Incident.

The selected test Incident with the remote file attached

Just to make sure that all went well, you can click on the download link to pull down the attachment and look it over, verifying that it came across complete and intact. That pretty much demonstrates that it all works as we had intended. If you would like a copy to play around with on your own, you can pick it up here.