“It is a myth that we can get systems ‘right the first time.’ Instead, we should implement only today’s stories, then refactor and expand the system to implement new stories tomorrow. This is the essence of iterative and incremental agility. Test-driven development, refactoring, and the clean code they produce make this work at the code level.” — Robert Cecil Martin (Uncle Bob)
In our last installment, we restructured the way in which the client-side code interacted with the server-side Script Include and then we went ahead and built out that first step. Now we just need to keep marching down through the list of steps until we have tackled them all. The next step in the list is to either update the existing member application record, or in the case of a first-time publication, create a brand new member application record. To build that record, we will need to grab the data from the original sys_app record, we should grab that guy right off of the bat.
processPhase2: function(answer) {
var sysAppGR = new GlideRecord('sys_app');
if (sysAppGR.get(answer.appSysId)) {
...
} else {
answer = this.processError(answer, 'Invalid Application sys_id: ' + answer.appSysId);
}
return answer;
},
As we did before, we throw in a simple check, just to make sure that we were actually able to fetch the record. The next thing that we will need to do is to see if this is a new record or not, and if so, create that initial entry.
var mbrAppGR = new GlideRecord('x_11556_col_store_member_application');
if (answer.mbrAppId == 'new') {
mbrAppGR.initialize();
mbrAppGR.provider.setDisplayValue(gs.getProperty('instance_name'));
mbrAppGR.setValue('application', answer.appSysId);
mbrAppGR.insert();
answer.mbrAppId = mbrAppGR.getUniqueValue();
mbrAppGR.initialize();
}
With a new record created if needed, the rest of the update logic can work for both new and existing records, since we just converted the new record to an existing record by inserting the bare minimum data to kick things off.
That’s all there is to that. Putting it all together, the entire processPhase2 function looks like this:
processPhase2: function(answer) {
var sysAppGR = new GlideRecord('sys_app');
if (sysAppGR.get(answer.appSysId)) {
var mbrAppGR = new GlideRecord('x_11556_col_store_member_application');
if (answer.mbrAppId == 'new') {
mbrAppGR.initialize();
mbrAppGR.provider.setDisplayValue(gs.getProperty('instance_name'));
mbrAppGR.setValue('application', answer.appSysId);
mbrAppGR.insert();
answer.mbrAppId = mbrAppGR.getUniqueValue();
mbrAppGR.initialize();
}
if (mbrAppGR.get(answer.mbrAppId)) {
mbrAppGR.setValue('name', sysAppGR.getValue('name'));
mbrAppGR.setValue('description', sysAppGR.getValue('short_description'));
mbrAppGR.setValue('current_version', sysAppGR.getValue('version'));
mbrAppGR.setValue('active', true);
mbrAppGR.update();
} else {
answer = this.processError(answer, 'Invalid Member Application sys_id: ' + answer.mbrAppId);
}
} else {
answer = this.processError(answer, 'Invalid Application sys_id: ' + answer.appSysId);
}
return answer;
},
That one was pretty easy compared to what we went through last time, so let’s go ahead and do another. The next step, which is to create the application version record, is even easier, as every version record is an insert, so we don’t have to check for new or existing.
processPhase3: function(answer) {
var sysAppGR = new GlideRecord('sys_app');
if (sysAppGR.get(answer.appSysId)) {
var versionGR = new GlideRecord('x_11556_col_store_member_application_version');
versionGR.setValue('member_application', answer.mbrAppId);
versionGR.setValue('version', sysAppGR.getValue('version'));
versionGR.setValue('installed', true);
versionGR.insert();
answer.versionId = versionGR.getUniqueValue();
} else {
answer = this.processError(answer, 'Invalid Application sys_id: ' + answer.appSysId);
}
return answer;
},
The next thing that we need to do is attach the XML to the version record now that it exists. This is actually something that was not on our original list of things to do, so before we jump into that, we are going to need to correct that oversight. That’s a little bit of rework to code that we have already had to rework a few times already, but that’s the way these things go sometimes. I’m not really feeling all that excited about reworking that one more time right at the moment, though, so let’s say we jump right into that next time out.
“Everything you can imagine is real.” — Pablo Picasso
Last time, we completed the work on the UI Page for our modal pop-up and tested it out with a stubbed-out version of our server-side Script Include. Now we need to go back into our Script Include and put in the actual code that will do all of the work of publishing an app to the Host instance. In our stubbed-out version, our processPhase function simply returned the value ‘success’ for every call. We need to add a little structure to that function so that each phase can have an exclusive function of its own to contain all of the logic for that particular phase. We can do that by examining the phase variable, and then we can create separate functions for each phase. At this point, we can even stub those out as we did the original, just to allow us to build and test one function at at time. Here is the modified portion of the script:
At this point, we can even run our test again, just to be sure that we did not break anything, and it still should step through all of the tasks and reveal the Done button at the end. So far, so good. Now we can tackle each step one at a time, and work our way through the various processes until we complete them all. We might as well take them all in order, so let’s start out with the processPhase1 function.
The purpose of this first step will be to turn the Update Set into XML. The first thing that we will need to do is to use the passed Update Setsys_id to get the GlideRecord for the Update Set, which we will need to pass to our global function that does all of the heavy lifting. Once we successfully produce the XML, we will have to do something with it temporarily, since we do not yet have the version record to which it will eventually be attached. The simplest thing to do would be to attach it to some record that we do have, and then transfer the attachment once we create the version record. This should work, but it would be nice to send back the sys_id of the new attachment record, just to make things easier in the future step. If we want to do that, though, we will have to change our strategy for the response from a simple string to a JSON string that can have multiple values. That’s not that much of a change, and it sounds like something that will be useful to have in this process, so let’s just go ahead and do that now.
If we create an object that contains all of the values that we have been passing around, that will actually simplify the function calls, as there will only be the one object to pass as an argument. This will change our onload function to this:
It will also change our processPhase function to accept the object as the lone argument, and to both send and receive a stringified version of the object with the GlideAjax call.
function processPhase(answer) {
var ga = new GlideAjax('ApplicationPublisher');
ga.addParam('sysparm_name', 'processPhaseClient');
ga.addParam('sysparm_json', JSON.stringify(answer));
ga.getXMLAnswer(function (jsonString) {
hideElement('loading_' + answer.phase);
answer = JSON.parse(jsonString);
if (answer.error) {
showElement('error_' + answer.phase);
showElement('done_button');
} else {
showElement('success_' + answer.phase);
answer.phase++;
if (answer.phase < 7) {
showElement('phase_' + answer.phase);
processPhase(answer);
} else {
showElement('done_button');
}
}
});
}
I actually like this much better, but now we are going to have to modify our Script Include to expect the JSON string coming in and also to send a JSON string back. The modified Script Include now looks like this:
I also went ahead and added a processError function to script to consolidate all of the code related to reporting an issue with any of the processes. Again, I think this is much better than the original, as it both simplifies the code and opens possibilities that did not exist with the original design. Before we get back to coding out that initial phase, we should run another test, just to make sure things are still working as they should.
Rerunning the concept test after modifying the UI Page and the Script Include
Well, at lease we did not break anything. Now let’s get back to work on that processPhase1 function. First, we need to fetch the GlideRecord for the Update Set.
var updateSetGR = new GlideRecord('sys_update_set');
if (updateSetGR.get(answer.updSetId)) {
...
} else {
answer = this.processError(answer, 'Invalid Update Set sys_id: ' + answer.updSetId);
}
There is no reason to expect that we would not retrieve the Update SetGlideRecord at this point, but just in case, we check for that anyway and report an error if something is amiss. With the GlideRecord in hand, we can now call on our global utility to turn the Update Set into XML.
var csgu = new global.CollaborationStoreGlobalUtils();
var xml = csgu.updateSetToXML(updateSetGR);
Our global utility does not report any kind of error, but we should probably examine the XML returned, just to make sure that we have a valid XML file. We should be able to do that by checking the first line for the standard XML header.
if (xml.startsWith('<?xml version="1.0" encoding="UTF-8"?>')) {
...
} else {
answer = this.processError(answer, 'Invalid XML file returned from subroutine');
}
Now that we have the XML, we need to stuff it somewhere until we need it in a future step. We should be able to go ahead and create the attachment record at this point, and then we can just move the attachment to its proper place once we reach that point. To do that, we can take advantage of the GlideSysAttachment API. One of the things that we will need in order to do that is a name for our new XML file, which we should be able to generate from some details in the original application record, so we will have to go fetch that guy first, and then build the file name from there.
var sysAppGR = new GlideRecord('sys_app');
if (sysAppGR.get(answer.appSysId)) {
var fileName = sysAppGR.getDisplayValue('name');
fileName = fileName.toLowerCase().replace(/ /g, '_');
fileName += '_v' + sysAppGR.getDisplayValue('version') + '.xml';
var gsa = new GlideSysAttachment();
...
} else {
answer = this.processError(answer, 'Invalid Application sys_id: ' + answer.appSysId);
}
Once again, there is no reason to expect that we would not retrieve the application GlideRecord at this point, but just in case, we check for that as well, and report any errors. Once we have the record, we build the file name from the app name and the app version. Now we have everything that we need to create the attachment.
answer.attachmentId = gsa.write(sysAppGR, fileName, 'application/xml', xml);
if (!answer.attachmentId) {
answer = this.processError(answer, 'Unable to create XML file attachment');
}
Now we have generated our XML and stuffed it into an attachment record for later use. All together, the entire function now looks like this:
processPhase1: function(answer) {
var updateSetGR = new GlideRecord('sys_update_set');
if (updateSetGR.get(answer.updSetId)) {
var csgu = new global.CollaborationStoreGlobalUtils();
var xml = csgu.updateSetToXML(updateSetGR);
if (xml.startsWith('<?xml version="1.0" encoding="UTF-8"?>')) {
var sysAppGR = new GlideRecord('sys_app');
if (sysAppGR.get(answer.appSysId)) {
var fileName = sysAppGR.getDisplayValue('name');
fileName = fileName.toLowerCase().replace(/ /g, '_');
fileName += '_v' + sysAppGR.getDisplayValue('version') + '.xml';
var gsa = new GlideSysAttachment();
answer.attachmentId = gsa.write(sysAppGR, fileName, 'application/xml', xml);
if (!answer.attachmentId) {
answer = this.processError(answer, 'Unable to create XML file attachment');
}
} else {
answer = this.processError(answer, 'Invalid Application sys_id: ' + answer.appSysId);
}
} else {
answer = this.processError(answer, 'Invalid XML file returned from subroutine');
}
} else {
answer = this.processError(answer, 'Invalid Update Set sys_id: ' + answer.updSetId);
}
return answer;
},
Well, that was a bit of work, but hopefully the remaining steps will all be a little easier. Next time out, we can start on the second step, which will be to create or update the Collaboration Store application record.
“Doubt is an uncomfortable condition, but certainty is a ridiculous one.” — Voltaire
Now that we have the face of our new pop-up window laid out, it’s time to build out the code underneath that will do all of the work. Since this is all server-side activity, we will need some kind of client accessible Script Include that we can call from the client side for each step of the process. Just to do a little testing on the structure and the process, I created a stubbed-out version of a Script Include that I can call from the client-side code to do a little testing before we get too far into the weeds.
var ApplicationPublisher = Class.create();
ApplicationPublisher.prototype = Object.extendsObject(global.AbstractAjaxProcessor, {
processPhaseClient: function() {
var phase = this.getParameter('sysparm_phase');
var mbrAppId = this.getParameter('sysparm_mbr_app_id');
var appSysId = this.getParameter('sysparm_app_sys_id');
var updSetId = this.getParameter('sysparm_upd_set_id');
var origAppId = this.getParameter('sysparm_orig_app_id');
return this.processPhase(phase, mbrAppId, appSysId, updSetId, origAppId);
},
processPhase: function(phase, mbrAppId, appSysId, updSetId, origAppId) {
return 'success';
},
type: 'ApplicationPublisher'
});
At this point, all it does is return the value ‘success’ each time that it is called. This will be enough to prove out all of the client-side code before we dig into the actual work on the server side. For now, we just want to prove that the design is functional before we invest too much time in the actual tasks ahead of us.
On the client side, we will want to create a generic function that will work essentially the same for every step of the process. To keep track of where we are, I created a variable called phase, which is basically just the number of the current step. I called this function processPhase.
This is a recursive function that calls itself for every step until all six steps have been completed or there was some kind of an error. If the response from the Ajax call is ‘success’, then the loading icon for that step is hidden and replaced with the success icon, and then the phase is incremented. If we haven’t reached the end of the steps, then the HTML block for the next step is revealed and the process is repeated. If we reach the end of the steps, or if the Ajax call returns anything other than success, then the process comes to an end and the Done button is revealed. There is no effort on the client side to communicate any error or completion message to the operator. The assumption is that all messaging will be handled on the server side.
For the hideElement and showElement functions, I just stole some old code from some other component where I needed to do the same thing. There’s not much exciting here, but just for the sake of including everything, here it is:
function showElement(elementName) {
var elem = gel(elementName);
elem.style.visibility = 'visible';
elem.style.display = '';
}
function hideElement(elementName) {
var elem = gel(elementName);
elem.style.visibility = 'hidden';
elem.style.display = 'none';
}
Even though the processPhase function calls itself once it gets going, something has to initially get the ball rolling to kicks things off. For that, we can use an onload function to initialize all of the values and make that first call.
function onLoad() {
var phase = 1;
var mbrAppId = gel('mbr_app_id').value;
var appSysId = gel('app_sys_id').value;
var updSetId = gel('upd_set_id').value;
var origAppId = gel('mbr_app_id').value;
processPhase(phase, mbrAppId, appSysId, updSetId, origAppId);
}
Although that seemed like a good plan, when I ran my first test run, nothing happened. After doing a little online research, I discovered that the onLoad function is not a natural thing for a UI Page, and if I wanted something to run on load, I needed to add just a bit more code.
addLoadEvent(function() {
onLoad();
});
That was much better! Now after calling the server side six times in succession, the final rendition of the pop-up screen looked like this:
Pop-up structure test results
That proves that the client-side process functions as we intended and the stubbed-out Script Include is getting called and is returning the hard-coded ‘success’ value. This should now complete the work on the UI Page itself, so all that is left is to finish out the server-side Script Include to perform all of the tasks itemized in our list of steps. Next time, we can get started on the fist step, which will involve calling the global function that we created earlier to convert the Update Set to XML.
“Slow, steady progress is better than daily excuses.” — Robin Sharma
Now that we have managed to create an Update Set from our UI Action, it is time to figure out how to best accomplish the rest of the tasks needed to complete the publication of our Scoped Application to the Host instance. We will need another pop-up window to control the rest of the activities and to report the status to the operator, as the original pop-up went away when the progress bar appeared, and the progress bar goes away once you click on the Done button (an unnecessary click for our purposes, which I would like to avoid at some point, but for now, we will leave it alone and focus on other, more important activities). The next pop-up window should be the last, as it will be a fully custom component and we should be able to control all of the events from this point forward.
Here is what I am thinking at this point: We can create a simple UI that lists all of the steps necessary to complete our objective, and reveal them one at a time as we progress through the work involved. We can have three different icons ahead of each item on the list, one for in progress, one for success, and another for failure, and then show/hide the icons based on the current status of each step in the process. To make that work, we would have to contact the server side for each step along the way, updating the UI on the client side with the status returned from each server call. Once we successfully complete the final step, we can reveal a Done button to close the pop-up.
To find some appropriate icons, I entered image_picker.do in the left-hand navigator to bring up the list of all of the images currently in the /images/ directory of the platform.
image_picker.do results
I selected the following icons to represent the associated stages for each step:
In progress:/images/loading_anim4.gif
Complete:/images/check32.gif
Error:/images/delete_row.gif
As with many things on the Now Platform, there are many ways in which we can develop a pop-up dialog. I am most comfortable with building Service Portal widgets, and I have used widgets for UI pop-ups before, so that was a tempting way to go. However, the initial pop-up that we customized for our UI Action was a straight UI Page, so I decided that, if I wanted to keep things consistent, I should build this new pop-up the same way. So I cloned our original publish_app_dialog_csUI Page to create publish_app_dialog_cs_2. Then I went in to our original UI Page and removed this code:
… which took you to the newly created Update Set when everything was finished, and replaced it with this:
var dialogClass = GlideModal ? GlideModal : GlideDialogWindow;
var dd2 = new dialogClass("x_11556_col_store_publish_app_dialog_cs_2");
dd2.setTitle(new GwtMessage().getMessage('Publish to Collaboration Store'));
dd2.setPreference('sysparm_mbr_app_id', this.mbrAppId);
dd2.setPreference('sysparm_app_sys_id', this.appId);
dd2.setPreference('sysparm_upd_set_id', updateSetId);
dd2.setWidth(500);
dd2.render();
… which now pops up our newly cloned UI Page in a new modal dialog. Now we actually have to build out the page. To begin, as I usually do, I started with the HTML of the presentation layer, which came out like this:
Basically, it is the same block of code repeated 6 times for the six remaining tasks needed to complete the process. Each item in the list contains a description of the step along with our three icons, only one of which should appear at any given time. Initially, the in progress icon is visible as soon as the step becomes visible, and my plan is to hide that one and reveal one of the other two, depending on how things came out once the step has been completed.
Here is how it looks when it first appears:
New pop-up dialog box
That takes care of the easy part. Now we have to put all of the code underneath of the UI to actually do all of the things that need to be done. That’s quite a bit of work, so let’s say we will get started on that next time out.
Now that we have the needed global Script Includeout of the way, we can turn our attention to building the Publish to Collaboration StoreUI Action. Before we do that, though, we are going to need a couple more tables, one for the published applications, and another for the application versions. I won’t waste a lot of time here on the process of building a new table; that’s pretty basic Now Platform stuff that you can find just about anywhere. I also did not spend a lot of time creating the tables. At this point, they are very basic and just enough to get things going. I am sure at some point I will be coming back around and adding more valuable fields, but for right now, I just included the bare necessities to create this initial publication process.
Here is the form for the base application table:
Base Member Application table input form
… and here is the form for the application versions:
Member Application Version table input form
There is an obvious one-to-many relationship between the application record and all of the various version records for an application. In practice, all of the versions would be listed under an application as a Related List on the application form. We can show examples of that once we build up some data.
Now that that is done, we can get back to our UI Action. Since our action is going to be very similar to the stock Publish to Update Set… action, the easiest way to create ours is to pull up the stock action and use the context menu to select Insert and Stay to clone the action.
Select Insert and Stay from the context menu to clone the UI Action
Now we just have to rename the action and update the script, which actually turned out to be more modifications that I had anticipated, primarily because our action is not in the global scope. First, we had to rename the function to avoid a conflict with the original action with which we will be sharing the page. Also, the gel function and the window object are not available to scoped actions, and of course, we needed to point to our own UI Page instead of the original and put in our own title.. Here is the modified script:
function publishToCollaborationStore() {
var sysId = g_form.getUniqueValue();
var dialogClass = GlideModal ? GlideModal : GlideDialogWindow;
var dd = new dialogClass("x_11556_col_store_publish_app_dialog_cs");
dd.setTitle(new GwtMessage().getMessage('Publish to Collaboration Store'));
dd.setPreference('sysparm_sys_id', sysId);
dd.setWidth(500);
dd.render();
}
I also removed all of the conditionals for now, since some of that code was not available to a scoped UI Action, and we will end up having our own conditions, anyway, with which we can deal later on. Right now, I am still just focused on seeing if I can get all of this to work.
The next thing that we need to do is to build our own UI Page, on which we can also get a head start by cloning the original using the same Insert and Stay method. The modifications here are a little more extensive, as we want to do much more than just create an Update Set. One thing that we will want to do is check to make sure that the version number is not one that has already been published. To support that, I added some code at the top to pull all of the current versions so that I can stick them into a hidden field.
var existingVersions = '';
var mbrAppId = 'new';
var appSysId = jelly.sysparm_sys_id;
var appGR = new GlideRecord('sys_app');
appGR.get(appSysId);
var appName = appGR.getDisplayValue();
var appVersion = appGR.version || '';
var appDescription = appGR.short_description;
var mbrAppGR = new GlideRecord('x_11556_col_store_member_application');
if (mbrAppGR.get('application', appSysId)) {
mbrAppId = mbrAppGR.getUniqueValue();
var versionGR = new GlideRecord('x_11556_col_store_member_application_version');
versionGR.addQuery();
versionGR.query('member_application', mbrAppId);
while (versionGR.next()) {
existingVersions += '|' + versionGR.getDisplayValue('version');
}
existingVersions += '|';
}
To pass this value to the client side for validation, I added a hidden field to the HTML for the page:
To check the validity on the client side, I retained the original version validity check and then added my own check of the existing versions right underneath using the same alert approach to delivering the bad news:
this.existingVersions = gel('existing_versions').value;
...
var vd = g_form.validators.version;
if (!vd)
console.log("WARNING. Cannot find validator for version field in parent form.");
else {
var answer = vd.call(g_form, this.versionField.value);
if (answer != true) {
alert(answer);
return null;
}
}
if (this.existingVersions.indexOf('|' + versionField.value + '|') != -1) {
alert('Version ' + versionField.value + ' has already been published to the Collaboration Store');
return null;
}
The last thing that we have to change is what happens once the Update Set has been created. In the original version that we cloned, once the Update Set has been produced, the operator is then taken to the Update Set form to see the newly created Update Set. We don’t want to do that, as we still have much work to do to, including the following:
Update or Create an Application record
Create a new Version record linked to the Application record
Convert the Update Set to XML
Attach the XML to the Version Record
Send all of the updated/created records over to the Host instance, including the attached XML
We will need some kind of vehicle in which to perform all of the these tasks and keep the operator updated as to the progress and success or failure of each one of the steps. That sounds like quite a bit of work, so I think that will be a good topic for our next installment.
“I like taking things apart and putting them back together. Tinkering. I’d be a professional tinkerer. Tinkerbell. I think that’s what they’re called.” — Chris Carmack
When we took a peek under the hood of the two UI Actions that produced an Update Set from a Scoped Application and turned it into an XML file, it looked like we could leverage most of the code to accomplish our own objective of publishing an app to the Host instance. Unfortunately, after experimenting a bit with various modifications, it appears that some things simply cannot be done within the scope of our Collaboration Store app. Two of the Script Includes involved can only be invoked from another global component (UpdateSetExport and ExportWithRelatedLists), and the delete statement that is executed after the XML file is produced is also something that can only be done in the global scope. I did not really want to create any components outside of our application scope, but in this instance, it doesn’t look like I have any other option.
Given this new tidbit of information, my new goal is to create a single global component and limit the contents of that component to just the things that are absolutely necessary to be in the global scope. I may even try to create this component in the global scope programmatically as part of the set-up process so that a separate global Update Set is not needed, but for now, let’s just see if we can get it to work. I called my new component CollaborationStoreGlobalUtils, and set the Accessible from to All application scopes so that it could be called from any of our scoped components.
All of the things that needed to be in the global scope seemed to be centered around the process to convert an Update Set to an XML file, so I created a function called updateSetToXML to which I passed the GlideRecord for the Update Set. The purpose of the function was to turn the Update Set into XML and return the XML text. Most of the code I lifted straight out of the Publish to XMLUI Action and the Processor that it called.
updateSetToXML: function(updateSetGR) {
var updateSetExport = new UpdateSetExport();
var rusSysId = updateSetExport.exportUpdateSet(updateSetGR);
var remoteUpdateGR = new GlideRecord('sys_remote_update_set');
remoteUpdateGR.get(rusSysId);
var exporter = new ExportWithRelatedLists('sys_remote_update_set', rusSysId);
exporter.addRelatedList('sys_update_xml', 'remote_update_set');
var fakeResponse = this.responseObject();
exporter.exportRecords(fakeResponse);
remoteUpdateGR.deleteRecord();
return fakeResponse.getOutputStreamText();
}
This is an abbreviated version of the code lifted from the stock components, as I removed things like role checking, the validity checking of various GlideRecords, and the value of passed parameters. My thinking is that those things can all be handled in our scoped components before we get to this point, so this function only needs to contain those things that are required to be in the global scope. Since the code that we are borrowing was intended to be used to send the XML file created to the servlet response output stream, I did have to introduce a little bit of creative hackery to replace the expected g_response object parameter in the exportRecords function with an object of my own design.
To construct an object that would be accepted as valid input to the exportRecords function, I dug through the code in that function looking for all of properties and methods that it was expecting to find. Most of those were basic setter functions used to define the HTTP response, and since we weren’t actually going to have an HTTP response, I was able to just include those in the object as empty functions that did nothing and returned nothing.
The one method that did expect a return was the getOutputStream function, and it was expected to return a valid Java output stream object. Back in my Java developer days, whenever I needed to convert an output stream to a string, I always used the java.io.ByteArrayOutputStream class, so I set up a local variable that was an instance of that class and then added a getter to return that variable.
outputStream: new Packages.java.io.ByteArrayOutputStream(),
getOutputStream: function() {
return this.outputStream;
}
Utilizing the global Packages object is another thing that cannot be done in a Scoped Application, but since we already made the commitment to build this global Script Include, that didn’t turn out to be a problem here. The last thing that I needed to do was to convert the output stream object to a string once the XML was assembled, so I created this additional nonstandard function to obtain the text:
So far, those are the only two functions that I have had to include in my global component. The entire thing at this stage of the process now looks like this:
var CollaborationStoreGlobalUtils = Class.create();
CollaborationStoreGlobalUtils.prototype = {
initialize: function() {
},
updateSetToXML: function(updateSetGR) {
var updateSetExport = new UpdateSetExport();
var rusSysId = updateSetExport.exportUpdateSet(updateSetGR);
var remoteUpdateGR = new GlideRecord('sys_remote_update_set');
remoteUpdateGR.get(rusSysId);
var exporter = new ExportWithRelatedLists('sys_remote_update_set', rusSysId);
exporter.addRelatedList('sys_update_xml', 'remote_update_set');
var fakeResponse = this.responseObject();
exporter.exportRecords(fakeResponse);
remoteUpdateGR.deleteRecord();
return fakeResponse.getOutputStreamText();
},
responseObject: function() {
return {
outputStream: new Packages.java.io.ByteArrayOutputStream(),
addHeader: function() {
},
setHeader: function() {
},
setContentType: function() {
},
getOutputStream: function() {
return this.outputStream;
},
getOutputStreamText: function() {
var dataAsString = Packages.java.lang.String(this.outputStream);
dataAsString = String(dataAsString);
return dataAsString;
}
};
},
type: 'CollaborationStoreGlobalUtils'
};
Now that we have that little piece out of the way, we can start building our Publish to Collaboration StoreUI Action and the associated UI Page that will be launched in the modal pop-up when that action is selected. Unless we have some additional feedback from the set-up process testing, we’ll jump right into that next time out.
“Every tester has the heart of a developer … in a jar on their desk.” — Unknown
Last time, we discussed jumping into the code that would allow us to share a local Scoped Application with the Host instance, but the results are starting to come in from some of the folks who have been testing the set-up process, so we should probably deal with those first. Here’s what we have so far:
Installation error: Table ‘sys_hub_action_status_metadata’ does not exist
Not allowing update of property: x_11556_col_store.store_name
Not allowing update of property: x_11556_col_store.host_instance
In the setup, the instance name field doesn’t inform you that you only need the instance prefix, not the full url
You can only collaborate with one host
Many thanks to all of those who have been participating in the testing process. Your efforts are much appreciated. For those of you who tried things out, but neglected to post anything, please feel free to leave a comment on your experience, even if you have no defects to report. All feedback is welcome … thanks!
Now let’s take a look at the issues that have been reported so far. one issue at a time.
Installation error: Table ‘sys_hub_action_status_metadata’ does not exist
It looks like the table ‘sys_hub_action_status_metadata’ is a table related to a version or plugin that I have in my instance, but is not present in the instance on which the test installation was being performed. The version of my instance is glide-rome-06-23-2021__patch0-07-07-2021 according to stats.do. The table in question looks relatively new, with a create date of 2021-07-31 16:24:07, and its name implies some sort of metadata, so I don’t think it is anything critical to the operation of the application. If there are no other issues with this installation, my opinion would be that this error could be safely ignored. To make it go away, I could probably just remove any references to this table from the Update Set. That sounds to me like the best way to go, just to avoid the potential of this error popping up, even though it seems as if it is fairly benign.
Not allowing update of property: x_11556_col_store.store_name Not allowing update of property: x_11556_col_store.host_instance
These two are basically the same problem for two different System Properties. This is an annoyance that really should be corrected somehow. The work-around that was used was to switch over to the application scope, but that should not be necessary. When you are installing an app for the first time, that scope has not even been established yet, so I need to do something to allow these properties to be modified from the global scope, or from any scope for that matter. I’m not exactly sure how to do that, so I will have to do a little research and see what I can come up with. But this is definitely an issue that needs to be addressed.
In the setup, the instance name field doesn’t inform you that you only need the instance prefix, not the full url
This is very true, and should probably be addressed as well. The snh-form-field tag does provide for a “help” attribute, which appears underneath the label, so that’s probably a good place to throw that onto the screen. I’ll make sure that gets added in there before I release the next version.
You can only collaborate with one host
This is also very true, but that’s the way this particular version was conceived. Back when I was thinking of doing something peer-to-peer without anyone designated as the Host, I was leaning more towards that kind of environment, but once I settled on the Host/Client approach, I was always thinking one Host and many Clients. I can see the benefit of being able to connect to more than one Host, but that’s a little more complicated that I was thinking of taking on at this point, so I think I will file that one in the maybe-I-will-do-that-one-day pile. Good idea, though.
All in all, the list so far is not bad, but I assume that there is more to come. It seems like the biggest issue at this point is the cross-scope updates of the application’s System Properties, but the missing table is also something that might give people pause for no reason. Hopefully, I can find a way to address those before I push out the next version.
Thanks again to everyone who took the time to pull this down and give it a whirl, particularly those of you who posted your findings. And if you did not run into any difficulties and were able to get to the point where every instance has the same list of member organizations, please post those results as well, including the number of instances involved. Any feedback is welcome, and as always, much appreciated. Looking forward to hearing more … thank you all!
Next time out, we’ll see if we can get back to building out the application publishing process and maybe start finding out if we can make that work.
“I believe in a world where all these things can happen, even if I have to do them myself.” — Adrian Lamo
While we wait for the hordes of fellow developers currently running their extensive tests on the initial version of the Collaboration Store app to report their findings, I thought it might be a good time to give a little bit of thought as to where things should go from here. As I mentioned earlier, there are a number of other improvements that I would like to make to the set-up process once we are sure that everything is working as it should. That would definitely be one specific area towards which we could direct our attentions next. On the other hand, I am quite anxious to see if I can actually do some real sharing of artifacts between instances, and I am not quite sure how I am going to do that, so I would like to start focusing on that little project as well. I have managed to cobble together enough parts to get multiple instances aware of one another, so even though the initial set-up process is not quite ready for prime time, maybe it is good enough for now and I can start trying to work towards accomplishing the actual purpose of this application. Besides, we have not received any results from any of those selfless volunteers who have been doing the outside testing as yet, so we really don’t want to disturb that code at this point.
I would like to share all kinds of different artifacts at some point, but to start out, I am going to focus on sharing Scoped Applications in much the same way that applications are shared to internal app stores or the public ServiceNow App Store. There are two sides to the process of sharing: 1) publishing your app to the desired target, and 2) retrieving an app from the desired source and installing it in your instance. Since I can only do one thing at a time, and one obviously has to take place before you can do the other, my initial focus will be to see if I can publish an app to the Host instance. Once we clear that hurdle, then maybe we can see if we can do the flip side and pull something down from the Host and get it installed. But that’s nothing that we need to worry about today, particularly since we don’t even know yet if we can pull off putting something out there!
The first thing that we will need is a couple of tables, one to store the published applications and another to store each version of the application. These are just meta data tables containing information about the app and the versions. I envision that the application itself will be an XML Update Set file, which I plan to attach to the application version table record for each version published. The question is, How do we do that?
There are already built-in capabilities to turn an app into an Update Set and to turn an Update Set into an XML file. Both of those are UI Actions, so taking a peek under the hood of those artifacts would seem like a good place to start. Before you can turn an Update Set into an XML file, you first have to have an Update Set, so let’s first take a peek at the code under the UI Action that does that. The easiest way to do that would be to pull up a Scoped Application and then use the hamburger menu to pull up the list of UI Actions for that form.
Select Configure -> UI Actions from the context menu to pull up the list of UI Actions
The one that we are looking for is called Publish to Update Set… and what we are looking for is the code that is executed when the action is selected.
function publishIt() {
var sysId = gel('sys_uniqueValue').value;
var dialogClass = window.GlideModal ? GlideModal : GlideDialogWindow;
var dd = new dialogClass("publish_app_dialog");
dd.setTitle(new GwtMessage().getMessage('Publish to Update Set'));
dd.setPreference('sysparm_sys_id', sysId);
dd.setWidth(500);
dd.render();
}
Well, that didn’t tell us much. All this code does is launch a dialog box. We will need to look at the UI Page that will appear in the pop-up window, which we can see from the code is called publish_app_dialog. The key element there is the publishApp() function in the Client script of that UI Page.
Basically, this code replaces the original pop-up with a progress bar driven off of the server side publishToUpdateSet function of the com.snc.apps.AppsAjaxProcessor. This is probably something that we still want to do. We just don’t want to be sent to the Update Set form when it all over. Instead, we want to go ahead and turn it into an XML file, and then we want to attach that file to a new version record for the application. Also, if the application has never been published before, we will need to create the master application record as well. But it looks like we can steal most of this code so far. We will need to clone both the UI Action and the UI Page, and then point our cloned action to our cloned page, which is where we will make the majority of the changes. So far, so good.
Now we need to turn the Update Set into XML. For that, we take a peek at the other UI Action for that purpose, found on the Update Set form. To hunt that guy down, we pull up any local Update Set and use the same hamburger menu options to pull up the UI Actions for that form and look for the one called Export to XML. Here is the relevant script:
var updateSetExport = new UpdateSetExport();
var sysid = updateSetExport.exportUpdateSet(current);
action.setRedirectURL("export_update_set.do?sysparm_sys_id=" + sysid + "&sysparm_delete_when_done=true");
The first couple of lines look important, but that last one looks like it pops up the XML file for downloading, which we definitely do not want to do in our adaptation. It turns out that export_update_set is a Processor, which also has code of its own, so we better dig into that guy and see if does anything important that we might need to retain. We can find that guy by entering the word processors in the left-hand navigation menu and then select the lone menu item that appears to bring up the list. Once we have the list, we just need to search for the one where the Path is export_update_set. Here is the relevant code:
(function process(g_request, g_response, g_processor) {
var sysid = g_request.getParameter('sysparm_sys_id');
var remoteUpdateGR = new GlideRecord('sys_remote_update_set');
if (!remoteUpdateGR.get(sysid)) {
gs.addErrorMessage(gs.getMessage('Update Set is invalid'));
return;
}
if (!gs.hasRole('admin') && !sn_app_api.AppStoreAPI.canPublishToUpdateSet(remoteUpdateGR.application)) {
gs.addErrorMessage(gs.getMessage('You do not have permission to perform this operation'));
g_response.setStatus(403);
return;
}
var exporter = new ExportWithRelatedLists('sys_remote_update_set', sysid);
exporter.addRelatedList('sys_update_xml', 'remote_update_set');
exporter.exportRecords(g_response);
var del = g_request.getParameter('sysparm_delete_when_done');
if (del == "true" && remoteUpdateGR.canDelete())
remoteUpdateGR.deleteRecord();
})(g_request, g_response, g_processor);
So it looks like code in which we are interested is the stuff going on with the variable exporter. Unfortunately, there is no output from the exportRecords function of that object and it takes as input the global g_response object. That will send the output to the browser, which we definitely do not want to do. Maybe we can hack it by sending it a fake g_response object, and then pull our output from there. If you right click on the class name (ExportWithRelatedLists), you can use the resulting context menu to pull up the definition and see exactly what it does. There is quite a bit of code in there, but here is the relevant function:
exportRecords: function(response){
this.setHeaders(response);
var outputStream = response.getOutputStream();
this.hd = this.beginExport(outputStream);
var gr = new GlideRecord(this.parent_table);
gr.get(this.sys_id);
this.exportRecord(gr);
this.exportChildren();
this._exportQuerySets();
this.endExport(outputStream);
}
It basically sends everything to the outputStream that it obtains from the response object that you send it. So, we could build our own response object with our own outputStream, use this component right out of the box, and then grab the outputStream from our phony g_response object. Or, we just steal the code from this guy and use it to build our own exporter and have our version of the exportRecords function simply return the XML. Either way, we should be able to leverage all of this code to do what it is that we want to do. That should give us our XML file, anyway.
Of course, generating the XML from the Scoped Application is just the start of the process. Once we have it in hand, we have to create a version record, attach the XML to the version record as a file, and then ship the thing over to the Host instance. Once it arrives at the Host, we will also have to send the version record and the attached XML file out to all of the other instances. That’s all quite a bit of work, but so was the set-up process, and we just trudged through that effort one piece at a time until we got to the end of the to-do list. Hopefully, we can do the same here. We may jump right into that next time, if we don’t yet have any feedback from the set-up testing.
“Every great mistake has a halfway moment, a split second when it can be recalled and perhaps remedied.” — Pearl S. Buck
A while back, I took a quick look at the new Flow Variables feature of the Flow Designer in the hopes that the new feature might eliminate the need for my earlier effort to develop a Flow DesignerScratchpad. Unfortunately, I wasn’t able to figure out how to make it work in the way that it was described in the Blog Entry that I was attempting to follow. I tried a few things on my own, but that did not work for me, either. Since I was not smart enough to figure out how to make it do what I was trying to do, I just gave up.
Since that time, though, I have upgraded my Personal Developer Instance to Rome, so I thought that it might be a good time to go back and see if anything had changed since the last time that I tried it. As it turns out, not much has changed, but during my second look I realized that if I had just kept reading the original Blog Entry that I had been following, I would have come across the way to actually pull this off. The secret is in this little icon that I never bothered to click on when I was looking at things the first time.
Flow variable transform function icon
I still cannot simply add the +1 after the Flow Variable pill as it shows earlier in the post, but by clicking on that little icon, I can bring up the transform options and select Math -> Add from the Suggested Transforms menu. That brings up a brief dialog box where I can enter the 1 as the number that I would like to add to the current value.
Transform pop-up dialog
Unlike my earlier attempt at manually scripting the math, this actually worked when I ran a test execution.
Sample Flow test results
This is great news. Now all that I have to do is hunt down all of the places where I have used Scratchpad variables and see if I can replace them with Flow Variables. One of the Actions that I created using the Scratchpad was the Array Iterator, which was built back when the For EachFlow Logic only supported GlideRecords. Now that For Each can be used on Arrays as well, there is no longer a need for the custom built Array Iterator. I also used the Scratchpad for a Counter, something else that could now be replaced with a Flow Variable (now that I know how to make it work!). I should really hunt down the entire lot and replace them all.
That sounds like a lot of work, though, so I will probably take that that on as I crack things open for other maintenance. Still, it is good to finally be able to use the features of the product rather than these homemade concoctions. They definitely served their purpose at the time, but now that the platform has caught up with my needs, it’s definitely time to move on.
“It’s hard enough to find an error in your code when you’re looking for it; it’s even harder when you’ve assumed your code is error-free.” — Steve McConnell
Now that we have completed all of the parts for the initial set-up process of our new Scoped Application, it’s time to take a step back and see where things stand. On the one hand, after 16 captivating installments, you would think that we would be much further along in this process beyond just the initial set-up. On the other hand, this is a fairly complex endeavor, and it’s good to get this necessary administrative function out of the way so that we can focus on the actual purpose of the application. But before we jump right into that, we should first have quick look at what we have and what we don’t have at this point.
What we have is an initial version of the set-up process for both the Host instance and the Client instances. Now all of this needs to be fully tested in multiple scenarios, but even if we manage to kill all of the bugs that are undoubtedly baked in there at this stage of the game, as it is written, it assumes for the most part that all will go well every time. What I mean by that is that there isn’t a whole lot of error recovery built into the process right now. Everything seems to work if all of the instances are up and running when contacted. That’s not really good enough for prime time, though, as it is always possible that one or more instances might be unavailable or off-line for some reason. At some point, we will have to build in some processes to monitor for that and to deal with it in some way. Right now, if you fail to get some kind of update from the Host, you just don’t get it. That’s not really good enough in the long run, but my approach is always to get things working first, and then add such features later in a future version. Maybe we will even handle that using Event Management, although not everyone has that feature activated, so maybe that’s not a good plan after all.
There are other features that I would like to add as well. For example, it would be nice if each participating instance had some form of logo or image that would visually identify them and all of the items that they have shared with the community. Things like that are nice-to-haves, though, so again, we’ll deal with that later. At this point, I just want to make sure that what we have put together so far actually works the way that it was intended before we go any further.
I also want all of the menu options hidden until set-up is complete, and then once set-up has been completed, I would like the set-up option to be hidden. I haven’t thrown that in there just yet, either, but that’s something that I don’t want to forget to do once I am sure that everything is working as it should be.
Not too long ago, I had an offer to assist with the testing of this particular project. Normally, I like to do all of my own testing, but they say that programmers are the worst testers of their own code, so I’m going to break with tradition and go ahead and put out an Update Set for this app that is clearly not finished and basically not good for anything of value at this point. If anyone want to participate in this effort, all that I ask is that you post any defects that you uncover to the comments section so that I can see if I can’t get them resolved and put out a new version with the corrections.
So, here’s the deal: gather up your friends and neighbors and come up with some strategy to see who draws the short straw and serves as the Host instance, set up the Host first, and then everyone else jump in and set up their Client instances by referencing the Host. This can work with just two instances, but to see the existing instance updates for any new instance, you will need at least three (one for the Host, one for the new Client, and at least one for an existing Client). Four our more would be even better, but three will at least test all of the current features. When all is said and done, everyone’s list of member instances should match, unless something went terribly wrong along the way. And if you really want to put yourself out there, you can set up a Host instance and put your instance ID in the comments so that other people that you don’t even know can attempt to connect to your instance. Your call.
To install this version of the Collaboration Store (we’ll call it version 0.1), you will need this Update Set, which contains the Scoped Application, and you will also need the latest version of snh-form-fields, which you can find here. Install the form fields Update Set first, and then install the Scoped Application. At that point, you should be good to go and should be able to click on the set-up menu option at any time. I’ll let this sit out here for a while and see if anything comes of it. Thanks in advance for helping a guy out. It’s very much appreciated.
