Collaboration Store, Part XLVII

“Relinquish your attachment to the known, step into the unknown, and you will step into the field of all possibilities.”
Deepak Chopra

While we wait patiently for the testing results of the instance sync process to come trickling in, we should go ahead and get started on the application installation process. As I laid out earlier, there are a couple of different ways to go here, and I really do not like either one. However, as much as I would like to have come up with a preferable third alternative, nothing has really come to mind, despite the fact that I have been dragging out the instance sync process for much longer than I should have while I searched in vain for a better solution. We have to keep pushing forward, though, so enough waiting. Let’s get into it.

I have decided to go with the client-side approach, mainly because I don’t want to get involved with the security issues related to attempting to emulate a user session on the server side. Both approaches involve quite a bit more hackery than I really care to include in something that is supposed to be a viable product to be distributed and used by others, but at this point, I don’t see any other way. The client-side approach involves downloading the Update Set XML file from the server, and then presenting it back as if it were a local file uploaded to the import XML process. So, to start with, we will need a Script Include function that will deliver the XML to the client side. Since this will be a client callable Script Include, we will want to create a new one rather than add another function to our existing utilities. We will call this new Script Include ApplicationInstaller, and check the Client callable checkbox when we create it. Our lone function will be called getXML and it will accept a single parameter for the sys_id of the attachment record, and then use that sys_id to fetch the record and return both the file name and the file contents in a JSON string response.

var ApplicationInstaller = Class.create();
ApplicationInstaller.prototype = Object.extendsObject(global.AbstractAjaxProcessor, {

	getXML: function() {
		var answer = {};
		var sysId = this.getParameter("attachment_id");
		if (sysId) {
			var sysAttGR = new GlideRecord('sys_attachment');
			if (sysAttGR.get(sysId)) {
				var gsa = new GlideSysAttachment();
				answer.fileName = sysAttGR.getDisplayValue('file_name');
				answer.xml = gsa.getContent(sysAttGR);
			}
		}
		return JSON.stringify(answer);
	},

    type: 'ApplicationInstaller'
});

There’s nothing too extraordinary about the code here. We do leverage our old friend the GlideSysAttachment to grab the contents of the file, but that’s nothing that we have not already done before. Other than that, it’s pretty basic stuff.

Now that we have a way to fetch the name and contents of the attached file, we will need a way to launch the process, which will either be a clone of the existing upload.do page or the original upload.do page with a little creative hackery to bend it to our will. I hate to mess with the original, but I also don’t want to create an entire second copy if I don’t have to. But let’s not get too far ahead of ourselves just yet. Right now, we just need to build a launcher for the process, and we can easily change the page that we launch once we decide which way to go.

To launch the installation process, we can add a UI Action to the version form, which we can simply label Install. We will want to make this action conditional, since we would not want to see this option on a version that has already been installed. At this point, we don’t need to worry about the code behind the action; we can just create it and see how it renders out on the page.

New UI Action “Install”

After saving the new action, we can pull up a version page and see how it looks.

New UI Action rendered on the version page

Now we need to add some code to make the action actually do something. Basically, we just want to jump over to whatever version of the upload.do page we end up pursuing, but before we do that we need to include a parameter. The client callable Script Include that we just created takes an attachment record sys_id as a parameter, but out action is sitting on the version record, not the attachment record. We will need to hunt down the attachment record to pull out the sys_id and then we can pass that along to our destination so that it can be used to make the GlideAjax call to our Script Include function. Here is the UI Action script to make all of that happen.

var attachmentGR = new GlideRecord('sys_attachment');
attachmentGR.addQuery('table_name', 'x_11556_col_store_member_application_version');
attachmentGR.addQuery('table_sys_id', current.sys_id);
attachmentGR.addQuery('content_type', 'CONTAINS', 'xml');
attachmentGR.query();
if (attachmentGR.next()) {
	action.setRedirectURL('/upload.do?attachment_id=' + attachmentGR.getUniqueValue());
} else {
	gs.addErrorMessage('No Update Set XML file found attached to this version');
}

Once again, this is pretty basic stuff that doesn’t require a whole lot of explanation. We query the attachment table for an XML file attached to the current record, and if we find it, we construct a URL and jet off to that page. If not, we throw up an error message, but that really shouldn’t happen unless something has gone horribly wrong somewhere along the line.

So now we have built enough parts to get us to the page that will actually fetch the XML from the server and then push it back up again. I need to make a decision as to whether I should hack up the original upload.do page or just use it as a guide to make one of my own. I’ll need to give that some thought, so let’s pause for now and we’ll jump into that next time out.

Collaboration Store, Part XXXVI

“The definition of flexibility is being constantly open to the fact that you might be on the wrong track.”
Brian Tracy

Although it is long past time to start some serious work on the third major component of this little(?) project, the application installation process, I have been rather hesitant to officially kick that off. Last time, we addressed the last of the reported defects in the first two processes, the initial set-up and the application publishing process. Now, it would seem, would be the time to jump into wrestling with that last remaining primary function and put that to bed before turning our attentions to the list of secondary features that will complete the initial release of the full product. At least, that would appear to be the next logical step.

The reason for my reluctance, however, is that I have taken a cursory look at the various approaches available to accomplish my goal, and quite frankly, I don’t really like any of them. When we converted our Update Set to XML, we were able to fish out enough parts and pieces from the stock product to cobble together a reasonable solution with a minimal amount of questionable hackery. To go in the other direction, to convert the XML back to an Update Set, the only stock component that appears to provide this functionality is bound tightly with the process of uploading a local file from the user’s file system. The /upload.do page, or more specifically, the /sys_upload.do process to which the form on that page is posted, handles both the importing of the XML file and the conversion of that file to an Update Set. There is no way to extract the process that turns the XML into an Update Set, since everything is encapsulated into that one all-encompassing process. For our purposes, we do not have a local file on the user’s machine to upload; our file is an attachment already present on the server, so invoking this process, which seems the only way to go, involves much more than we really need.

To invoke the one and only process that I have found (so far!) to make this conversion, then, we will have to post a multi-part form to /sys_upload.do that includes our XML data along with all of the other fields, headers, and cookies expected by the server-side form processor. On the server side, we should be able to accomplish this with an outbound REST message, or on the client side, we should be able to do this by somehow sending our XML instead of a local file when submitting the form. Each approach has its own merits, but they also each have their own issues, and no matter which way you go, it’s a rather complicated mess.

The Server Side Approach

Posting a multi-part form on the server side is actually relatively simple as far as the form data goes. We can construct a valid body for a standard multipart/form-data POST using our XML data and related information and then send it out using an outbound REST message. That’s the easy part. We just need to add an appropriate Content-Type header, including some random boundary value:

Content-Type: multipart/form-data; boundary=somerandomvalue

Then we can build up the body by including all of the hidden fields on the form, and then add our XML in a file segment that would look something like this:

------------somerandomvalue
Content-Disposition: form-data; name="attachFile"; filename="app_name_v1.0.xml"
Content-Type: application/xml

<... insert XML data here ...>

------------somerandomvalue--

In addition to the XML file component, you would also need to send all of the other expected form field values, some of which are preloaded on the form when it is delivered. To obtain those values, you would have to first issue an HTTP GET request of the /upload.do page and pick those values out of the resulting HTML. This can be accomplished with a little regex magic and the Javascript string .match() method. Here is a simple function to which you can pass the HTML and a pattern to return the value found in the HTML based on the pattern:

function extractValue(html, pattern) {
	var response = '';
	var result = html.match(pattern);
	if (result.length > 1) {
		response = result[1];
	}
	return response;
}

For example, one of the form fields found on the /upload.do page is sysparm_ck. The INPUT element for this hidden field looks like this:

<input name="sysparm_ck" id="sysparm_ck" type="hidden" value="68fa4eee2fa401104425fcecf699b646939f52c6787c23fff22b124fcf58f713235b7478"></input>

To snag the value of that field, you would just pass the HTML for the page and the following pattern to our extractValue function:

id="sysparm_ck" type="hidden" value="(*.?)"

Once you have obtained the value, you can use it to build another “part” in the multi-part form body:

------------somerandomvalue
Content-Disposition: form-data; name="sysparm_ck"

68fa4eee2fa401104425fcecf699b646939f52c6787c23fff22b124fcf58f713235b7478

------------somerandomvalue

All of that is pretty easy to do, and would work great except for one thing: this POST would only be accepted as part of an authenticated session, and cannot just be sent in on its own. Theoretically, we could create an authenticated session by doing a GET of the /login.do page and then a POST of some authoritative user’s credentials, but that would mean knowing and sending the username and password of a powerful user, which is a dangerous thing with which to start getting involved. For that reason, and that reason alone, this does not seem to be a good way to go.

The Client Side Approach

On the client side, you are already involved in an authenticated session, so that’s not any kind of an issue at all. What you do not have is the XML, so to do anything on the client side, we will first need to create some kind of GlideAjax service that will deliver the XML over to the client. Once we have the XML that we would like to upload in place of the normal local file, we will have to perform some kind of magic trick to update the form on the page with our data in the place of a file from the local computer. To do that, we will have to either create our own copy of the /upload.do page or add a global script that will only run on that page, and only if there is some kind of URL parameter indicating that this is one of our processes and not just a normal user-initiated upload. We did this once before with a global script that only ran on the email client, so I know that I can run a conditional script on a stock page if I do not create a page of my own, but the trick will be getting the XML data to be sent back to the server with the rest of the input form.

After nosing around a bit for available options, it appears that you might be able to leverage the DataTransfer object to build a fake file, link it to the form, and then submit the form using something like this:

function uploadXML(xml, fileName) {
	var fileList = new DataTransfer();
	fileList.items.add(new File(xml, fileName));
	document.getElementById('attachFile').files = fileList.files;
	document.forms[0].submit();
}

Of course, there will be a lot more to it than that, as you will need to get a handle on the Update Set created and then try to Preview and Commit it programmatically as well, but this looks like a possibility. Still, you have to move the entire Update Set XML all the way down to the client just to push it all the way back up to the server again, which seems like quite a waste. Plus, with any client-side functionality, there is always the browser compatibility issues that would all need to be tested and resolved. Maybe this would work, but I still don’t like it. It seems like quite a bit of complexity and more than a few opportunities for things to go South. I’m still holding out hope that there is a better way.

Now what?

So … given that I don’t like any of the choices that I have come up with so far, I have decided to set that particular task aside for now in the hopes that a better alternative will come to me before I invest too much effort into a solution with which I am not all that thrilled. There is no shortage of things to do here, so my plan is to just focus on other issues and then circle back to this particular effort when a better idea reveals itself, or I run out of other things to do. Technically, once you have obtained the XML for a particular version from the Host, you can still manually install it by downloading the attachment yourself and importing like any other XML Update Set. That’s not really how I intend all of this to function, but it does work, so it should be OK to set this aside for a time.

Next time, then, instead of forging ahead with this third major component as I had originally planned, I will pick something else out of the pile and we will dig into that instead.

Collaboration Store, Part XXXI

“Someone’s sitting in the shade today because someone planted a tree a long time ago.”
Warren Buffett

While we wait for additional test results to come in from the corrected Update Set for our latest iteration, it would be a good time to take a look at what we will need to do to complete the last remaining step in the application publishing process. What we have completed so far pushes all of the new application version artifacts to the Host instance. Now we need to build out the process for the Host instance to send out all of those artifacts to all of the other Client instances. Fortunately, we already have in hand most of the code to do that; we just need to clone a few things and make a number of modifications to fit the new purpose.

We have already built a Subflow to push new Client instance information out to all of the existing Client instances during the new instance registration process. Pushing out a new version of an application to those same instances is essentially the same process, so we should be able to clone that guy, and the associated Action, to create our new process. We have also created functions to push over the application record, the version record, and the attached Update Set XML file, and we should be able to copy over those guys as well to complete the process. In fact, we should probably start with the JavaScript functions, as we will need to call the root function in the Action, which will need to be created before it can be referenced in the Subflow.

The functions as written are hard-coded to send everything over to the Host instance. We will want to do basically the same thing, but this time we will be sending things out to various Client instances. Ideally, I should just refactor the code to have the destination instance and credentials passed in so that one function would work in both circumstances, but at this point it was less complicated to just make copies of each function and hack them up for their new purpose. I don’t really like having two copies of essentially the same thing, though, so one day I am going to have to go back and rework the entire mess.

But for now, this is what I came up with when I copied what was once called processPhase5 to create the new publishNewVersion function:

publishNewVersion: function(newVersion, targetInstance, attachmentId) {
	var targetGR = new GlideRecord('x_11556_col_store_member_organization');
	if (targetGR.get('instance', targetInstance)) {
		var token = targetGR.getDisplayValue('token');
		var versionGR = new GlideRecord('x_11556_col_store_member_application_version');
		if (versionGR.get(newVersion)) {
			var canContinue = true;
			var targetAppId = '';
			var mbrAppGR = versionGR.member_application.getRefRecord();
			var request  = new sn_ws.RESTMessageV2();
			request.setHttpMethod('get');
			request.setBasicAuth(this.WORKER_ROOT + targetInstance, token);
			request.setRequestHeader("Accept", "application/json");
			request.setEndpoint('https://' + targetInstance + '.service-now.com/api/now/table/x_11556_col_store_member_application?sysparm_fields=sys_id&sysparm_query=provider.instance%3D' + mbrAppGR.getDisplayValue('provider.instance') + '%5Ename%3D' + encodeURIComponent(mbrAppGR.getDisplayValue('name')));
			var response = request.execute();
			if (response.haveError()) {
				gs.error('CollaborationStoreUtils.publishNewVersion: Error returned from Target instance ' + targetInstance + ': ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
				canContinue = false;
			} else if (response.getStatusCode() == '200') {
				var jsonString = response.getBody();
				var jsonObject = {};
				try {
					jsonObject = JSON.parse(jsonString);
				} catch (e) {
					gs.error('CollaborationStoreUtils.publishNewVersion: Unparsable JSON string returned from Target instance ' + targetInstance + ': ' + jsonString);
					canContinue = false;
				}
				if (canContinue) {
					var payload = {};
					payload.name = mbrAppGR.getDisplayValue('name');
					payload.description = mbrAppGR.getDisplayValue('description');
					payload.current_version = mbrAppGR.getDisplayValue('current_version');
					payload.active = 'true';
					request  = new sn_ws.RESTMessageV2();
					request.setBasicAuth(this.WORKER_ROOT + targetInstance, token);
					request.setRequestHeader("Accept", "application/json");
					if (jsonObject.result && jsonObject.result.length > 0) {
						targetAppId = jsonObject.result[0].sys_id;
						request.setHttpMethod('put');
						request.setEndpoint('https://' + targetInstance + '.service-now.com/api/now/table/x_11556_col_store_member_application/' + targetAppId);
					} else {
						request.setHttpMethod('post');
						request.setEndpoint('https://' + targetInstance + '.service-now.com/api/now/table/x_11556_col_store_member_application');
						payload.provider = mbrAppGR.getDisplayValue('provider.instance');
					}
					request.setRequestBody(JSON.stringify(payload, null, '\t'));
					response = request.execute();
					if (response.haveError()) {
						gs.error('CollaborationStoreUtils.publishNewVersion: Error returned from Target instance ' + targetInstance + ': ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
						canContinue = false;
					} else if (response.getStatusCode() != 200 && response.getStatusCode() != 201) {
						gs.error('CollaborationStoreUtils.publishNewVersion: Invalid HTTP Response Code returned from Target instance ' + targetInstance + ': ' + response.getStatusCode());
						canContinue = false;
					} else {
						jsonString = response.getBody();
						jsonObject = {};
						try {
							jsonObject = JSON.parse(jsonString);
						} catch (e) {
							gs.error('CollaborationStoreUtils.publishNewVersion: Unparsable JSON string returned from Target instance ' + targetInstance + ': ' + jsonString);
							canContinue = false;
						}
						if (canContinue) {
							targetAppId = jsonObject.result.sys_id;
						}
					}
				}
			} else {
				gs.error('CollaborationStoreUtils.publishNewVersion: Invalid HTTP Response Code returned from Target instance ' + targetInstance + ': ' + response.getStatusCode());
			}
			if (canContinue) {
				this.publishVersionRecord(targetInstance, token, versionGR, targetAppId, attachmentId);
			}
		} else {
			gs.error('CollaborationStoreUtils.publishNewVersion: Version record not found: ' + newVersion);
		}
	} else {
		gs.error('CollaborationStoreUtils.publishNewVersion: Target instance record not found: ' + targetInstance);
	}
},

Unlike the original application publication process, we are not bouncing back and forth between the client side and the server side, so with the successful completion of one artifact, we can simply move on to pushing over the next artifact. To create the next step in the process, the publishVersionRecord function, I started out with a copy of the original processPhase6 function. Here is the end result:

publishVersionRecord: function(targetInstance, token, versionGR, targetAppId, attachmentId) {
	var canContinue = true;
	var payload = {};
	payload.member_application = targetAppId;
	payload.version = versionGR.getDisplayValue('version');
	var request  = new sn_ws.RESTMessageV2();
	request.setBasicAuth(this.WORKER_ROOT + targetInstance, token);
	request.setRequestHeader("Accept", "application/json");
	request.setHttpMethod('post');
	request.setEndpoint('https://' + targetInstance + '.service-now.com/api/now/table/x_11556_col_store_member_application_version');
	request.setRequestBody(JSON.stringify(payload, null, '\t'));
	response = request.execute();
	if (response.haveError()) {
		gs.error('CollaborationStoreUtils.publishVersionRecord: Error returned from Target instance ' + targetInstance + ': ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
		canContinue = false;
	} else if (response.getStatusCode() != 201) {
		gs.error('CollaborationStoreUtils.publishVersionRecord: Invalid HTTP Response Code returned from Target instance ' + targetInstance + ': ' + response.getStatusCode());
		canContinue = false;
	} else {
		jsonString = response.getBody();
		jsonObject = {};
		try {
			jsonObject = JSON.parse(jsonString);
		} catch (e) {
			gs.error('CollaborationStoreUtils.publishVersionRecord: Unparsable JSON string returned from Target instance ' + targetInstance + ': ' + jsonString);
			canContinue = false;
		}
		if (canContinue) {
			targetVerId = jsonObject.result.sys_id;
			this.publishVersionAttachment(targetInstance, token, targetVerId, attachmentId);
		}
	}
},

The last step in the process, then, is to send over the Update Set XML file attachment. For that one, I started out with the processPhase7 function and ended up with this:

publishVersionAttachment: function(targetInstance, token, targetVerId, attachmentId) {
	var gsa = new GlideSysAttachment();
	var sysAttGR = new GlideRecord('sys_attachment');
	if (sysAttGR.get(attachmentId)) {
		var url = 'https://';
		url += targetInstance;
		url += '.service-now.com/api/now/attachment/file?table_name=x_11556_col_store_member_application_version&table_sys_id=';
		url += targetVerId;
		url += '&file_name=';
		url += sysAttGR.getDisplayValue('file_name');
		var request  = new sn_ws.RESTMessageV2();
		request.setBasicAuth(this.WORKER_ROOT + targetInstance, token);
		request.setRequestHeader('Content-Type', sysAttGR.getDisplayValue('content_type'));
		request.setRequestHeader('Accept', 'application/json');
		request.setHttpMethod('post');
		request.setEndpoint(url);
		request.setRequestBody(gsa.getContent(sysAttGR));
		response = request.execute();
		if (response.haveError()) {
			gs.error('CollaborationStoreUtils.publishVersionAttachment: Error returned from Target instance ' + targetInstance + ': ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
		} else if (response.getStatusCode() != 201) {
			gs.error('CollaborationStoreUtils.publishVersionAttachment: Invalid HTTP Response Code returned from Target instance ' + targetInstance + ': ' + response.getStatusCode());
		}
	} else {
		gs.error('CollaborationStoreUtils.publishVersionAttachment: Invalid attachment record sys_id: ' + attachmentId);
	}
},

So that takes care of the functions. Now we have to create an Action in the Flow Designer that will call the function, and then produce a new Subflow that will leverage that action. That sounds like a good subject for our next installment.

Special Note to Testers

If you want to test the set-up process, you can do that with a single instance. You just have to select the Host option during the set-up process. If you want to test the Client set-up process, you will need at least two instances, one to serve as the Host, which you will need to reference when you set up the Client (you cannot be a Client of your own Host). But if you really want to test out the full registration process, including the Subflow that informs all existing instances of the new instance, you will need at least three participating instances: 1) the Host instance, 2) the new Client instance, and 3) an existing Client instance that will be notified of the new member of the community.

The same holds true for the application publishing process. You can test a portion of the publishing process by publishing an app on a single Host instance, but if you want to test out all of the parts and pieces, you will want to publish the app from a Client instance and make sure that it makes its way all the way back to the Host. Once we complete the application distribution process, full testing will require at least three instances to verify that the Host distributes the new version of the application to other Clients.

Collaboration Store, Part XXVII

“It does not matter how slowly you go as long as you do not stop.”
Confucius

Last time we pushed the application record to the Host instance and now we have to do basically the same thing with the version record. The only difference really, other than the table and fields, is that a version record will always be a new record, so there is no need to determine if the record exists or not on the Host instance. This simplifies the code quite a bit. We still need to fetch the GlideRecord that will be sent over, so as we did with the application record, this will be the first order of business.

var versionGR = new GlideRecord('x_11556_col_store_member_application_version');
if (versionGR.get(answer.versionId)) {
	...
} else {
	answer = this.processError(answer, 'Invalid version record sys_id: ' + answer.versionId);
}

Once we have the record, we can gather up our System Properties and build the payload for the REST API call.

var host = gs.getProperty('x_11556_col_store.host_instance');
var token = gs.getProperty('x_11556_col_store.active_token');
var payload = {};
payload.member_application = answer.hostAppId;
payload.version = versionGR.getDisplayValue('version');

With our payload in hand, we can now create and configure our sn_ws.RESTMessageV2 object.

var request  = new sn_ws.RESTMessageV2();
request.setBasicAuth(this.WORKER_ROOT + host, token);
request.setRequestHeader("Accept", "application/json");
request.setHttpMethod('post');
request.setEndpoint('https://' + host + '.service-now.com/api/now/table/x_11556_col_store_member_application_version');
request.setRequestBody(JSON.stringify(payload, null, '\t'));

Now all that is left to do is to execute the request and check the response.

response = request.execute();
if (response.haveError()) {
	answer = this.processError(answer, 'Error returned from Host instance: ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
} else if (response.getStatusCode() != 201) {
	answer = this.processError(answer, 'Invalid HTTP Response Code returned from Host instance: ' + response.getStatusCode());
} else {
	var jsonString = response.getBody();
	var jsonObject = {};
	try {
		jsonObject = JSON.parse(jsonString);
	} catch (e) {
		answer = this.processError(answer, 'Unparsable JSON string returned from Host instance: ' + jsonString);
	}
	if (!answer.error) {
		answer.hostVerId = jsonObject.result.sys_id;
	}
}

Well, that was easy! Here’s the whole thing all put together.

processPhase6: function(answer) {
	var versionGR = new GlideRecord('x_11556_col_store_member_application_version');
	if (versionGR.get(answer.versionId)) {
		var host = gs.getProperty('x_11556_col_store.host_instance');
		var token = gs.getProperty('x_11556_col_store.active_token');
		var payload = {};
		payload.member_application = answer.hostAppId;
		payload.version = versionGR.getDisplayValue('version');
		var request  = new sn_ws.RESTMessageV2();
		request.setBasicAuth(this.WORKER_ROOT + host, token);
		request.setRequestHeader("Accept", "application/json");
		request.setHttpMethod('post');
		request.setEndpoint('https://' + host + '.service-now.com/api/now/table/x_11556_col_store_member_application_version');
		request.setRequestBody(JSON.stringify(payload, null, '\t'));
		response = request.execute();
		if (response.haveError()) {
			answer = this.processError(answer, 'Error returned from Host instance: ' + response.getErrorCode() + ' - ' + response.getErrorMessage());
		} else if (response.getStatusCode() != 201) {
			answer = this.processError(answer, 'Invalid HTTP Response Code returned from Host instance: ' + response.getStatusCode());
		} else {
			var jsonString = response.getBody();
			var jsonObject = {};
			try {
				jsonObject = JSON.parse(jsonString);
			} catch (e) {
				answer = this.processError(answer, 'Unparsable JSON string returned from Host instance: ' + jsonString);
			}
			if (!answer.error) {
				answer.hostVerId = jsonObject.result.sys_id;
			}
		}
	} else {
		answer = this.processError(answer, 'Invalid version record sys_id: ' + answer.versionId);
	}

	return answer;
},

That takes care of 6 of the 7 steps. The last one that we will need to do will be to push the Update Set XML attachment over to the Host. That one may be a little more involved, so we will save that for next time out.

Collaboration Store, Part XIX

“I like taking things apart and putting them back together. Tinkering. I’d be a professional tinkerer. Tinkerbell. I think that’s what they’re called.”
Chris Carmack

When we took a peek under the hood of the two UI Actions that produced an Update Set from a Scoped Application and turned it into an XML file, it looked like we could leverage most of the code to accomplish our own objective of publishing an app to the Host instance. Unfortunately, after experimenting a bit with various modifications, it appears that some things simply cannot be done within the scope of our Collaboration Store app. Two of the Script Includes involved can only be invoked from another global component (UpdateSetExport and ExportWithRelatedLists), and the delete statement that is executed after the XML file is produced is also something that can only be done in the global scope. I did not really want to create any components outside of our application scope, but in this instance, it doesn’t look like I have any other option.

Given this new tidbit of information, my new goal is to create a single global component and limit the contents of that component to just the things that are absolutely necessary to be in the global scope. I may even try to create this component in the global scope programmatically as part of the set-up process so that a separate global Update Set is not needed, but for now, let’s just see if we can get it to work. I called my new component CollaborationStoreGlobalUtils, and set the Accessible from to All application scopes so that it could be called from any of our scoped components.

var CollaborationStoreGlobalUtils = Class.create();
CollaborationStoreGlobalUtils.prototype = {
    initialize: function() {
    },

	type: 'CollaborationStoreGlobalUtils'
};

All of the things that needed to be in the global scope seemed to be centered around the process to convert an Update Set to an XML file, so I created a function called updateSetToXML to which I passed the GlideRecord for the Update Set. The purpose of the function was to turn the Update Set into XML and return the XML text. Most of the code I lifted straight out of the Publish to XML UI Action and the Processor that it called.

updateSetToXML: function(updateSetGR) {
	var updateSetExport = new UpdateSetExport();
	var rusSysId = updateSetExport.exportUpdateSet(updateSetGR);
	var remoteUpdateGR = new GlideRecord('sys_remote_update_set');
	remoteUpdateGR.get(rusSysId);
	var exporter = new ExportWithRelatedLists('sys_remote_update_set', rusSysId);
	exporter.addRelatedList('sys_update_xml', 'remote_update_set');
	var fakeResponse = this.responseObject();
	exporter.exportRecords(fakeResponse);
	remoteUpdateGR.deleteRecord();
	return fakeResponse.getOutputStreamText();
}

This is an abbreviated version of the code lifted from the stock components, as I removed things like role checking, the validity checking of various GlideRecords, and the value of passed parameters. My thinking is that those things can all be handled in our scoped components before we get to this point, so this function only needs to contain those things that are required to be in the global scope. Since the code that we are borrowing was intended to be used to send the XML file created to the servlet response output stream, I did have to introduce a little bit of creative hackery to replace the expected g_response object parameter in the exportRecords function with an object of my own design.

To construct an object that would be accepted as valid input to the exportRecords function, I dug through the code in that function looking for all of properties and methods that it was expecting to find. Most of those were basic setter functions used to define the HTTP response, and since we weren’t actually going to have an HTTP response, I was able to just include those in the object as empty functions that did nothing and returned nothing.

addHeader: function() {
},
setHeader: function() {
},
setContentType: function() {
},

The one method that did expect a return was the getOutputStream function, and it was expected to return a valid Java output stream object. Back in my Java developer days, whenever I needed to convert an output stream to a string, I always used the java.io.ByteArrayOutputStream class, so I set up a local variable that was an instance of that class and then added a getter to return that variable.

outputStream: new Packages.java.io.ByteArrayOutputStream(),
getOutputStream: function() {
	return this.outputStream;
}

Utilizing the global Packages object is another thing that cannot be done in a Scoped Application, but since we already made the commitment to build this global Script Include, that didn’t turn out to be a problem here. The last thing that I needed to do was to convert the output stream object to a string once the XML was assembled, so I created this additional nonstandard function to obtain the text:

getOutputStreamText: function() {
	var dataAsString = Packages.java.lang.String(this.outputStream);
	dataAsString = String(dataAsString);
	return dataAsString;
}

That’s the entire fake g_response object, and the function that provides the object looks like this:

responseObject: function() {
	return {
		outputStream: new Packages.java.io.ByteArrayOutputStream(),
		addHeader: function() {
		},
		setHeader: function() {
		},
		setContentType: function() {
		},
		getOutputStream: function() {
			return this.outputStream;
		},
		getOutputStreamText: function() {
			var dataAsString = Packages.java.lang.String(this.outputStream);
			dataAsString = String(dataAsString);
			return dataAsString;
		}
	};
}

So far, those are the only two functions that I have had to include in my global component. The entire thing at this stage of the process now looks like this:

var CollaborationStoreGlobalUtils = Class.create();
CollaborationStoreGlobalUtils.prototype = {
    initialize: function() {
    },

	updateSetToXML: function(updateSetGR) {
		var updateSetExport = new UpdateSetExport();
		var rusSysId = updateSetExport.exportUpdateSet(updateSetGR);
		var remoteUpdateGR = new GlideRecord('sys_remote_update_set');
		remoteUpdateGR.get(rusSysId);
		var exporter = new ExportWithRelatedLists('sys_remote_update_set', rusSysId);
		exporter.addRelatedList('sys_update_xml', 'remote_update_set');
		var fakeResponse = this.responseObject();
		exporter.exportRecords(fakeResponse);
		remoteUpdateGR.deleteRecord();
		return fakeResponse.getOutputStreamText();
	},

	responseObject: function() {
		return {
			outputStream: new Packages.java.io.ByteArrayOutputStream(),
			addHeader: function() {
			},
			setHeader: function() {
			},
			setContentType: function() {
			},
			getOutputStream: function() {
				return this.outputStream;
			},
			getOutputStreamText: function() {
				var dataAsString = Packages.java.lang.String(this.outputStream);
				dataAsString = String(dataAsString);
				return dataAsString;
			}
		};
	},

	type: 'CollaborationStoreGlobalUtils'
};

Now that we have that little piece out of the way, we can start building our Publish to Collaboration Store UI Action and the associated UI Page that will be launched in the modal pop-up when that action is selected. Unless we have some additional feedback from the set-up process testing, we’ll jump right into that next time out.

Collaboration Store, Part XVII

“I believe in a world where all these things can happen, even if I have to do them myself.”
Adrian Lamo

While we wait for the hordes of fellow developers currently running their extensive tests on the initial version of the Collaboration Store app to report their findings, I thought it might be a good time to give a little bit of thought as to where things should go from here. As I mentioned earlier, there are a number of other improvements that I would like to make to the set-up process once we are sure that everything is working as it should. That would definitely be one specific area towards which we could direct our attentions next. On the other hand, I am quite anxious to see if I can actually do some real sharing of artifacts between instances, and I am not quite sure how I am going to do that, so I would like to start focusing on that little project as well. I have managed to cobble together enough parts to get multiple instances aware of one another, so even though the initial set-up process is not quite ready for prime time, maybe it is good enough for now and I can start trying to work towards accomplishing the actual purpose of this application. Besides, we have not received any results from any of those selfless volunteers who have been doing the outside testing as yet, so we really don’t want to disturb that code at this point.

I would like to share all kinds of different artifacts at some point, but to start out, I am going to focus on sharing Scoped Applications in much the same way that applications are shared to internal app stores or the public ServiceNow App Store. There are two sides to the process of sharing: 1) publishing your app to the desired target, and 2) retrieving an app from the desired source and installing it in your instance. Since I can only do one thing at a time, and one obviously has to take place before you can do the other, my initial focus will be to see if I can publish an app to the Host instance. Once we clear that hurdle, then maybe we can see if we can do the flip side and pull something down from the Host and get it installed. But that’s nothing that we need to worry about today, particularly since we don’t even know yet if we can pull off putting something out there!

The first thing that we will need is a couple of tables, one to store the published applications and another to store each version of the application. These are just meta data tables containing information about the app and the versions. I envision that the application itself will be an XML Update Set file, which I plan to attach to the application version table record for each version published. The question is, How do we do that?

There are already built-in capabilities to turn an app into an Update Set and to turn an Update Set into an XML file. Both of those are UI Actions, so taking a peek under the hood of those artifacts would seem like a good place to start. Before you can turn an Update Set into an XML file, you first have to have an Update Set, so let’s first take a peek at the code under the UI Action that does that. The easiest way to do that would be to pull up a Scoped Application and then use the hamburger menu to pull up the list of UI Actions for that form.

Select Configure -> UI Actions from the context menu to pull up the list of UI Actions

The one that we are looking for is called Publish to Update Set… and what we are looking for is the code that is executed when the action is selected.

function publishIt() {
	var sysId = gel('sys_uniqueValue').value;	
	var dialogClass = window.GlideModal ? GlideModal : GlideDialogWindow;
	var dd = new dialogClass("publish_app_dialog");
	dd.setTitle(new GwtMessage().getMessage('Publish to Update Set'));
	dd.setPreference('sysparm_sys_id', sysId);
	dd.setWidth(500);
	dd.render();
}

Well, that didn’t tell us much. All this code does is launch a dialog box. We will need to look at the UI Page that will appear in the pop-up window, which we can see from the code is called publish_app_dialog. The key element there is the publishApp() function in the Client script of that UI Page.

function publishApp(updateSetId) {
	var dd = new GlideModal("hierarchical_progress_viewer");
	dd.on("beforeclose", function () {
               var notification = {"getAttribute": function(name) {return 'true';}};
               CustomEvent.fireTop(GlideUI.UI_NOTIFICATION + '.update_set_change', notification);
               window.location.href = "sys_update_set.do?sys_id=" + updateSetId;
	});

	dd.setTitle("Progress");
	dd.setPreference('sysparm_function', 'publishToUpdateSet');
	dd.setPreference('sysparm_update_set_id', updateSetId);
	dd.setPreference('sysparm_sys_id', this.appId);
	dd.setPreference('sysparm_name', this.inferredUsName);
	dd.setPreference('sysparm_version', this.versionField.value);
	dd.setPreference('sysparm_description', this.descriptionField.value);
	dd.setPreference('sysparm_include_data', this.includeDataField.checked);
	dd.setPreference('sysparm_progress_name', "Publishing application");
	dd.setPreference('sysparm_ajax_processor', 'com.snc.apps.AppsAjaxProcessor');
	dd.setPreference('sysparm_show_done_button', 'true');

	dd.render();
	GlideDialogWindow.get().destroy();

	return dd;
}

Basically, this code replaces the original pop-up with a progress bar driven off of the server side publishToUpdateSet function of the com.snc.apps.AppsAjaxProcessor. This is probably something that we still want to do. We just don’t want to be sent to the Update Set form when it all over. Instead, we want to go ahead and turn it into an XML file, and then we want to attach that file to a new version record for the application. Also, if the application has never been published before, we will need to create the master application record as well. But it looks like we can steal most of this code so far. We will need to clone both the UI Action and the UI Page, and then point our cloned action to our cloned page, which is where we will make the majority of the changes. So far, so good.

Now we need to turn the Update Set into XML. For that, we take a peek at the other UI Action for that purpose, found on the Update Set form. To hunt that guy down, we pull up any local Update Set and use the same hamburger menu options to pull up the UI Actions for that form and look for the one called Export to XML. Here is the relevant script:

var updateSetExport = new UpdateSetExport();
var sysid = updateSetExport.exportUpdateSet(current);

action.setRedirectURL("export_update_set.do?sysparm_sys_id=" + sysid + "&sysparm_delete_when_done=true");

The first couple of lines look important, but that last one looks like it pops up the XML file for downloading, which we definitely do not want to do in our adaptation. It turns out that export_update_set is a Processor, which also has code of its own, so we better dig into that guy and see if does anything important that we might need to retain. We can find that guy by entering the word processors in the left-hand navigation menu and then select the lone menu item that appears to bring up the list. Once we have the list, we just need to search for the one where the Path is export_update_set. Here is the relevant code:

(function process(g_request, g_response, g_processor) {
    var sysid = g_request.getParameter('sysparm_sys_id');
    var remoteUpdateGR = new GlideRecord('sys_remote_update_set');
    if (!remoteUpdateGR.get(sysid)) {
        gs.addErrorMessage(gs.getMessage('Update Set is invalid'));
        return;
    }
    if (!gs.hasRole('admin') && !sn_app_api.AppStoreAPI.canPublishToUpdateSet(remoteUpdateGR.application)) {
        gs.addErrorMessage(gs.getMessage('You do not have permission to perform this operation'));
        g_response.setStatus(403);
        return;
    }
    var exporter = new ExportWithRelatedLists('sys_remote_update_set', sysid);
    exporter.addRelatedList('sys_update_xml', 'remote_update_set');
    exporter.exportRecords(g_response);
    var del = g_request.getParameter('sysparm_delete_when_done');
    if (del == "true" && remoteUpdateGR.canDelete())
        remoteUpdateGR.deleteRecord();
})(g_request, g_response, g_processor);

So it looks like code in which we are interested is the stuff going on with the variable exporter. Unfortunately, there is no output from the exportRecords function of that object and it takes as input the global g_response object. That will send the output to the browser, which we definitely do not want to do. Maybe we can hack it by sending it a fake g_response object, and then pull our output from there. If you right click on the class name (ExportWithRelatedLists), you can use the resulting context menu to pull up the definition and see exactly what it does. There is quite a bit of code in there, but here is the relevant function:

exportRecords: function(response){
    this.setHeaders(response);
    var outputStream = response.getOutputStream();
    this.hd = this.beginExport(outputStream);
    var gr = new GlideRecord(this.parent_table);
    gr.get(this.sys_id);
    this.exportRecord(gr);
    this.exportChildren();
    this._exportQuerySets();
    this.endExport(outputStream);
}

It basically sends everything to the outputStream that it obtains from the response object that you send it. So, we could build our own response object with our own outputStream, use this component right out of the box, and then grab the outputStream from our phony g_response object. Or, we just steal the code from this guy and use it to build our own exporter and have our version of the exportRecords function simply return the XML. Either way, we should be able to leverage all of this code to do what it is that we want to do. That should give us our XML file, anyway.

Of course, generating the XML from the Scoped Application is just the start of the process. Once we have it in hand, we have to create a version record, attach the XML to the version record as a file, and then ship the thing over to the Host instance. Once it arrives at the Host, we will also have to send the version record and the attached XML file out to all of the other instances. That’s all quite a bit of work, but so was the set-up process, and we just trudged through that effort one piece at a time until we got to the end of the to-do list. Hopefully, we can do the same here. We may jump right into that next time, if we don’t yet have any feedback from the set-up testing.