Aggregate List Columns, Part VI

“There are three qualities that make someone a true professional. These are the ability to work unsupervised, the ability to certify the completion of a job or task and, finally, the ability to behave with integrity at all times.”
Subroto Bagchi

Last time, we wrapped up the changes for the SNH Data Table from Instance Definition widget, and fixed a little issue with the core SNH Data Table widget in the process. Today, we need to do the same for the remaining wrapper widget, SNH Data Table from URL Definition. As with the previous widget, there isn’t that much code here that needs to be addressed, but I did come across these lines in the widget’s Server script:

copyParameters(data, ['p', 'o', 'd', 'filter', 'buttons', 'refpage', 'bulkactions']);
copyParameters(data, ['relationship_id', 'apply_to', 'apply_to_sys_id']);

It looks like I need to add the aggregates parameter to the list, so I threw that in and collapsed everything into a single line.

copyParameters(data, ['p', 'o', 'd', 'filter', 'aggregates', 'buttons', 'refpage',
	'bulkactions', 'relationship_id', 'apply_to', 'apply_to_sys_id']);

That was the only thing that I could find, so the next thing that I did was to clone the original test page once again and swap out the table widget for the one that I wanted to test. The problem with testing this one, though, is that all of the options that drive the process are URL parameters, and I did not want to type or paste in that huge URL every time I wanted to test. To eliminate that, I added an HTML widget above the table with a link that would contain the URL needed for testing. This way, all I needed to do to perform a test would be to click on that link.

Now I need to build a test URL using the same parameters that we used for the previous widget testing. Here is the list:

table: sys_user_group
filter: typeCONTAINS1cb8ab9bff500200158bffffffffff62
fields: name,manager
order_by: name
maximum_entries: 7
aggregates: [{ "label": "Members", "name": "members", "heading": "Members", "table": "sys_user_grmember", "field": "group", "filter": "user.active=true" },{ "label": "Incidents", "name": "incidents", "heading": "Incidents", "table": "incident", "field": "assignment_group", "filter": "active=true" },{ "label": "Catalog Tasks", "name": "sc_tasks", "heading": "Catalog Tasks", "table": "sc_task", "field": "assignment_group", "filter": "active=true" }]
refpage: {"sys_user": "user_profile"}

Using all of that to create a URL for the test page results in this lengthy string:

/sp?id=aggregate_test_4&table=sys_user_group&filter=typeCONTAINS1cb8ab9bff500200158bffffffffff62&fields=name,manager&order_by=name&maximum_entries=7&aggregates=[{"label":"Members","name":"members","heading":"Members","table":"sys_user_grmember","field":"group","filter":"user.active%3Dtrue"},{"label":"Incidents","name":"incidents","heading":"Incidents","table":"incident","field":"assignment_group","filter":"active%3Dtrue"},{"label":"CatalogTasks","name":"sc_tasks","heading":"CatalogTasks","table":"sc_task","field":"assignment_group","filter":"active%3Dtrue"}]&refpage={"sys_user":"user_profile"}

Once I constructed the URL, I added the following HTML to the HTML widget that I inserted on the page:

<div class="text-center" style="padding: 15px;">
  <a href="/sp?id=aggregate_test_4&amp;table=sys_user_group&amp;filter=typeCONTAINS1cb8ab9bff500200158bffffffffff62&amp;fields=name,manager&amp;order_by=name&amp;maximum_entries=7&amp;aggregates=[{&quot;label&quot;:&quot;Members&quot;,&quot;name&quot;:&quot;members&quot;,&quot;heading&quot;:&quot;Members&quot;,&quot;table&quot;:&quot;sys_user_grmember&quot;,&quot;field&quot;:&quot;group&quot;,&quot;filter&quot;:&quot;user.active%3Dtrue&quot;},{&quot;label&quot;:&quot;Incidents&quot;,&quot;name&quot;:&quot;incidents&quot;,&quot;heading&quot;:&quot;Incidents&quot;,&quot;table&quot;:&quot;incident&quot;,&quot;field&quot;:&quot;assignment_group&quot;,&quot;filter&quot;:&quot;active%3Dtrue&quot;},{&quot;label&quot;:&quot;CatalogTasks&quot;,&quot;name&quot;:&quot;sc_tasks&quot;,&quot;heading&quot;:&quot;CatalogTasks&quot;,&quot;table&quot;:&quot;sc_task&quot;,&quot;field&quot;:&quot;assignment_group&quot;,&quot;filter&quot;:&quot;active%3Dtrue&quot;}]&amp;refpage={&quot;sys_user&quot;:&quot;user_profile&quot;}">
    <button class="btn btn-primary">Click to Test</button>
  </a>
</div>

Now, all I needed to do was to bring up the Service Portal and pull up the page.

First look at the fourth test page

OK, that’s not all I needed to do. I also needed to click on the button to invoke that long URL with all of the parameters included.

Fourth test page after clicking on the Click to Test button

There, that’s better. So, it looks like everything seems to be in working order. That takes care of the last of the wrapper widgets. We still need to update the configuration file editor to support the new aggregate columns, which should be relatively straightforward, so maybe we can jump right into that next time out, wrap this whole thing up, and put out a new Update Set.

Aggregate List Columns, Part V

“The trouble you’re expecting never happens; it’s always something that sneaks up the other way.”
George R. Stewart

Last time, we took a little side trip on our journey to add this new functionality to the SNH Data Table Widget collection, but now we need to get back on track and finish up modifying the other wrapper widgets that still need to have some changes. Let’s start with the SNH Data Table from Instance Definition, which does not rely on an outside configuration object. When you configure this widget, you use the Edit feature of the Service Portal Page Designer. The edit dialog box that comes up can be customized for a widget using the Option schema field on the widget form, which already contains a number of customizations for the existing version of the widget. Here is how that looks right now:

[{"hint":"If enabled, show the list filter in the breadcrumbs of the data table",
"name":"enable_filter",
"default_value":"false",
"section":"Behavior",
"label":"Enable Filter",
"type":"boolean"},
{"hint":"A JSON object containing the specification for row-level buttons and action icons",
"name":"buttons",
"default_value":"",
"section":"Behavior",
"label":"Buttons",
"type":"String"},
{"hint":"A JSON object containing the page id for any reference column links",
"name":"refpage",
"default_value":"",
"section":"Behavior",
"label":"Reference Pages",
"type":"String"}]

In the current version, there is a string field for entering a JSON specification object for both Buttons/Icons and Reference Pages, so we will want to add one more just like that for our new aggregate columns. We can clean up the labels for all three while we are at it, just to make things a little clearer, so now it looks like this:

[{"hint":"If enabled, show the list filter in the breadcrumbs of the data table",
"name":"enable_filter",
"default_value":"false",
"section":"Behavior",
"label":"Enable Filter",
"type":"boolean"},
{"hint":"A JSON object containing the specifications for aggregate data columns",
"name":"aggregates",
"default_value":"",
"section":"Behavior",
"label":"Aggregate Column Specifications (JSON)",
"type":"String"},
{"hint":"A JSON object containing the specifications for row-level buttons and action icons",
"name":"buttons",
"default_value":"",
"section":"Behavior",
"label":"Button/Icon Specifications (JSON)",
"type":"String"},
{"hint":"A JSON object containing the page id for any reference column links",
"name":"refpage",
"default_value":"",
"section":"Behavior",
"label":"Reference Page Specifications (JSON)",
"type":"String"}]

Looking over the rest of the code, it doesn’t look as if there is anything else that needs to be done, so the next thing that we need to do is to set up a test so that we can try it out. Once again, we can clone the original test page and then make our modifications to the cloned page. The first thing that we will need to do is to swap out the SNH Data Table from JSON Configuration widget with our modified SNH Data Table from Instance Definition widget. Once we do that, we can click on the Edit pencil in the Page Designer and fill out the modified form.

Top half of the Page Designer widget option editor

For this test, I decided to use the Group table instead of the User table, and for our little test, I decided to limit the list to just ITIL groups. The type field can be more than one type, though, so I had to use a CONTAINS filter rather than an = filter, and the values are sys_ids, not names, so I had to look up the sys_id for the ITIL type. I just added a couple of fields, Name and Manager, and then ordered the list by Name. That pretty much took care of the top half of the widget option editor, so then I scrolled down to the bottom half, where our customizations appear.

Bottom half of the Page Designer widget option editor

Here I left the default color, selected an appropriate glyph icon for a group, and left the Link to this page and Enable Filter fields intentionally blank. In the Aggregate Column Specifications field, I entered a JSON String defining three different aggregate columns:

[{
 "label": "Members",
 "name": "members",
 "heading": "Members",
 "table": "sys_user_grmember",
 "field": "group",
 "filter": "user.active=true"
},{
 "label": "Incidents",
 "name": "incidents",
 "heading": "Incidents",
 "table": "incident",
 "field": "assignment_group",
 "filter": "active=true"
},{
 "label": "Catalog Tasks",
 "name": "sc_tasks",
 "heading": "Catalog Tasks",
 "table": "sc_task",
 "field": "assignment_group",
 "filter": "active=true"
}]

I did not define any buttons or icons, but I did set up a reference configuration to send the links on the managers to the User Profile page. All that was left to do at that point was to Save the options, jump out to the Service Portal, and give this baby a try.

First test of the new widget modifications

Now that’s disturbing. There is no data in any of the aggregate columns. I hate it when that happens! Now what? Well, I always try to keep this in mind whenever these things suddenly crop up.

After quite a few trials and tribulations, I finally tracked down the source of the problem. The ng-repeat for the aggregate columns is actually nested underneath another ng-repeat for the rows in the table. Apparently, ng-repeat has some issues with arrays of primitives or strings, and prefers to deal with objects. Why that did not present itself when I was working with an array length of 1 earlier, and only now caused a problem with an array length of 3 is still a mystery to me. Fortunately, I never need to know or understand the why; I just need to know what to do about it. The solution was to convert my integer value to an object containing an integer value. So, in the core SNH Data Table widget, I changed this:

return value;

… to this:

return {value: value};

Of course, that meant that I also had to change the HTML accordingly, so I changed this:

<td ng-repeat="aggValue in item.aggValue" class="text-right" ng-class="{selected: item.selected}" tabindex="0">
  {{aggValue}}
</td>

… to this:

<td ng-repeat="obj in item.aggValue" class="text-right" ng-class="{selected: item.selected}" tabindex="0">
  {{obj.value}}
</td>

Now let’s have another look at that new test page.

Second test of the new widget modifications

That’s better! OK, so that takes care of wrapper widget #2. Now we just need to handle the third wrapper widget, SNH Data Table from URL Definition. That sounds like a good project for our next installment.

Aggregate List Columns, Part IV

“Everything can be improved.”
Clarence W. Barron

Last time, I promised that we would get into providing aggregate column support to the remaining data table wrapper widgets and the Content Selector Configuration Editor, but before we do that, there is one little extra feature that I would like insert. As you may recall, when I set up the filter for the initial test page, I hard-coded the list of sys_ids for the members of the Hardware group. It would have been much easier to use the sys_user_grmember table and set the filter to group.name=Hardware, but the problem with that would be that the sys_id of the row would then be the sys_id of the sys_user_grmember record, and not the sys_id of the sys_user record. The reason that that is a problem is because there are no Incidents assigned to sys_user_grmember records, so the aggregate column values would all be 0.

There is, however, a way to make that work. We just need to add one more property to our aggregate column configuration object. If we add an optional source property, then we can allow developers to configure an alternate source for the sys_id rather than just relying on the sys_id of the primary record of the row. It could be optional, as we could always default to the sys_id of the row if you did not specify an alternate source, but if you did enter a value, then the contents of that field would be the sys_id that we would use in place of the standard behavior. Using that approach, we could develop a configuration that could look like this:

name: 'sys_user_grmember',
displayName: 'User',
all: {
	filter: 'group.name=Hardware',
	fields: 'group,user,user.title,user.email,user.department,user.location',
	aggarray: [{
		label: 'Incidents',
		name: 'incidents',
		heading: 'Incidents',
		table: 'incident',
		field: 'assigned_to',
		filter: 'active=true',
		source: 'user'
	}],
	btnarray: [],
	refmap: {},
	actarray: []
}

Of course, in order to do that, we would have to add some additional logic inside the for loop that puts together the data to be sent to the getAggregateValue function, but the changes should not be that significant. I simply created a new variable called sysId, set it initially to the sys_id of the record, then checked to see if there was a source defined, and if so, overrode the initial value with the value found in the source field of the current record.

for (var j=0; j<data.aggarray.length; j++) {
	var config = data.aggarray[j];
	var sysId = record.sys_id;
	if (config.source) {
		sysId = gr.getValue(config.source);
	}
	record.aggValue.push(getAggregateValue(sysId, config));
}

That’s all there is to that. Now we just need to put together a test case to see if it will actually work. Rather than muck up my original test page, I decided to clone the page and the configuration script so that I could run tests both with and without a source defined. I called my copy of the configuration script AggregateTestConfig2, and it turned out like this:

var AggregateTestConfig2 = Class.create();
AggregateTestConfig2.prototype = Object.extendsObject(ContentSelectorConfig, {
	initialize: function() {
	},

	perspective: [{
		name: 'all',
		label: 'All',
		roles: ''
	}],

	state: [{
		name: 'all',
		label: 'All',
	}],

	table: {
		all: [{
			name: 'sys_user_grmember',
			displayName: 'User',
			all: {
				filter: 'group.name=Hardware',
				fields: 'group,user,user.title,user.email,user.department,user.location',
				aggarray: [{
					label: 'Incidents',
					name: 'incidents',
					heading: 'Incidents',
					table: 'incident',
					field: 'assigned_to',
					filter: 'active=true',
					source: 'user'
				}],
				btnarray: [],
				refmap: {},
				actarray: []
			}
		}]
	},

	type: 'AggregateTestConfig2'
});

Now all we have to do is pull up the new test page and see how it looks.

Test page #2 with alternate sys_id source feature

So now you can define an alternate source for the sys_id that drives the aggregate, or you can choose not to and have the original functionality remain intact. Sweet! I like it. We still don’t have a way to click on the aggregate and pull up a list of the records represented there, but that’s something that we can always add in a future version. For now, we still have work to do involving the remainder of the core data table widget wrappers and the configuration file editor. I know that I said this last time, but this time for sure, we will jump right into that next time out.

Aggregate List Columns, Part III

“I want to try it to see what it’s like and see what my stuff looks like when I take it from inception to completion.”
Charlie Kaufman

Last time, we started the work of modifying the core widget of the data table widget collection. We took care of all of the code that imports the configuration data from the various wrapper widgets in the collection, and now we need to get down to the business of actually putting real data in the new aggregate list columns. To begin, we need to take a look at the code that pulls in the regular data for each row in the list.

	data.list = [];
	while (gr._next()) {
		var record = {};
		$sp.getRecordElements(record, gr, data.fields);
		if (typeof FilteredGlideRecord != 'undefined' && gr instanceof FilteredGlideRecord) {
			// FilteredGlideRecord doesn't do field-level
			// security, so take care of that here
			for (var f in data.fields_array) {
				var fld = data.fields_array[f];
				if (!gr.isValidField(fld))
					continue;

				if (!gr[fld].canRead()) {
					record[fld].value = null;
					record[fld].display_value = null;
				}
			}
		}
		for (var f in data.fields_array) {
			var fld = data.fields_array[f];
			if (record[fld].type == 'reference') {
				var refGr = gr;
				var refFld = fld;
				if (fld.indexOf('.') != -1) {
					var parts = fld.split('.');
					for (var x=0;x<parts.length-1;x++) {
						refGr = refGr[parts[x]].getRefRecord();
					}
					refFld = parts[parts.length-1];
				}
				if (refGr.isValidField(refFld)) {
					record[fld].table = refGr.getElement(refFld).getED().getReference();
					record[fld].record = {type: 'reference', sys_id: {value: record[fld].value, display_value: record[fld].value}, name: {value: record[fld].display_value, display_value: record[fld].display_value}};
				}
			}
		}
		record.sys_id = gr.getValue('sys_id');
		record.targetTable = gr.getRecordClassName();
		data.list.push(record);
	}

Basically, this code creates an array called data.list and then for each row creates an object called record, populates the record object from the GlideRecord, and then pushes the record object into the list. What we will want to do is add data to the record object before it gets pushed onto the list. The sys_id of the record will actually be useful to us, so it seems as if the best place to insert our code would be after the sys_id is established, but before the data.list.push(record) occurs. Here is what I came up with:

record.aggValue = [];
if (data.aggarray.length > 0) {
	for (var j=0; j<data.aggarray.length; j++) {
		record.aggValue.push(getAggregateValue(record.sys_id, data.aggarray[j]));
	}
}

To store the values, I added a value list to the record object. This is done whether or not any aggregate columns have been defined, just so there is something there regardless. Then we check to see if there actually are any aggregate columns defined, and if so, we loop through the definitions and then push a value onto the array for each definition. The value itself will be determined by a new function called getAggregateValue that takes the sys_id of the row and the aggregate column definition as arguments. We will need to build out that function, but for now, we can just return a hard-coded value, just to make sure that all is working before we dive into that.

function getAggregateValue(sys_id, config) {
	return 10;
}

Now that we have an array of values, we will need to go back into the HTML and modify the aggregate column section to pull data from this array. That section of the HTML now looks like this:

<td ng-repeat="aggValue in item.aggValue" class="text-right" ng-class="{selected: item.selected}" tabindex="0">
  {{aggValue}}
</td>

That should be enough to take it out for a spin and see if we broke anything. Let’s have a look.

Test page with hard-coded value array

Cool! So far, so good. Now we just need to go back into that getAggregateValue function definition and replace the hard-code value of 10 with some actual logic to pull the real data out of the database. For that, we will use a GlideAggregate on the configured table using the configured field and the sys_id of the current row. And if there is an optional filter present, we simply concatenate that to the end of the primary query.

function getAggregateValue(sys_id, config) {
	var value = 0;
	var ga = new GlideAggregate(config.table);
	ga.addAggregate('COUNT');
	var query = config.field + '=' + sys_id;
	if (config.filter) {
		query += '^' + config.filter;
	}
	ga.addEncodedQuery(query);
	ga.query();
	if (ga.next()) {
		value = parseInt(ga.getAggregate('COUNT'));
	}
	return value;
}

Now we can take another quick look and see if that finally gets us what we have been after all along.

Finally, the result we have been looking for

Voila! There it is — a column containing a count of selected related records. Now that wasn’t so hard, was it? Of course, we are not done quite yet. The SNH Data Table from JSON Configuration widget is not the only wrapper widget in the collection. We will need to add support for aggregate columns to the SNH Data Table from Instance Definition widget, as well as the SNH Data Table from URL Definition widget, the widget designed to work with the Configurable Data Table Widget Content Selector. Also, we have the Content Selector Configuration Editor, which allows you to create the JSON configuration files for both the Configurable Data Table Widget Content Selector and the SNH Data Table from JSON Configuration widget. That will have to be modified to support aggregate column specifications as well. None of that is super challenging now that we have things working, but it all needs to be done, so we will jump right into that next time out.

Aggregate List Columns, Part II

“If you have an apple and I have an apple and we exchange these apples then you and I will still each have one apple. But if you have an idea and I have an idea and we exchange these ideas, then each of us will have two ideas.”
George Bernard Shaw

Last time, we introduced the idea of a new column type for our modified version of the Service Portal Data Table Widget, built a sample configuration object, and then built a test page to try it all out. That was all relatively simple to do, but now we need to start digging around under the hood and see what coding changes we will need to make to the various table widgets in order to make this work. Our new test page uses the SNH Data Table from JSON Configuration widget, so that’s a good place to start looking for places that will need to be updated.

Virtually all of the heavy lifting for the data table widget collection is done in the core widget, SNH Data Table. All of the other widgets in the collection just provide a thin veneer over the top to gather up the configuration data from different sources. The SNH Data Table from JSON Configuration widget is no exception, and in digging through the code, it looks like there is really only one small place in the Server script that will need a slight modification.

// widget parameters
data.table_label = gr.getLabel();
data.filter = data.tableData.filter;
data.fields = data.tableData.fields;
data.btnarray = data.tableData.btnarray;
data.refmap = data.tableData.refmap;
data.actarray = data.tableData.actarray;

The above code moves each section of the configuration individually, and since we have added a new section to the configuration, we will need to insert a new line to include that new section. Copying the btnarray line to create a new aggarray line should take care of that.

// widget parameters
data.table_label = gr.getLabel();
data.filter = data.tableData.filter;
data.fields = data.tableData.fields;
data.aggarray = data.tableData.aggarray;
data.btnarray = data.tableData.btnarray;
data.refmap = data.tableData.refmap;
data.actarray = data.tableData.actarray;

One more easy part checked off of the To Do list. Now we need to take a look at that core widget, where we are going to find the bulk of the work to make this all happen. Still, we can start with the easy stuff, which will be found in the HTML portion of the widget. There are two areas on which we will need to focus, the column headings and the data rows. Once again, we can take a look at what is being done with the button array data and pretty much copy it verbatim to handle the new aggregate array data. Here is the relevant section for the column headings:

<th ng-repeat="button in data.btnarray" class="text-nowrap center" tabindex="0">
  {{button.heading || button.label}}
</th>

Inserting a copy of that code right above it and replacing the button references with the equivalent aggregate references yields the following new block of code:

<th ng-repeat="aggregate in data.aggarray" class="text-nowrap center" tabindex="0">
  {{aggregate.heading || aggregate.label}}
</th>
<th ng-repeat="button in data.btnarray" class="text-nowrap center" tabindex="0">
  {{button.heading || button.label}}
</th>

A little bit lower in the HTML is the code for the individual rows. Here again, we can take a look at the existing code dedicated to the buttons and icons.

<td ng-repeat="button in data.btnarray" class="text-nowrap center" ng-class="{selected: item.selected}" tabindex="0">
  <a ng-if="!button.icon" href="javascript:void(0)" role="button" class="btn-ref btn btn-{{button.color || 'default'}}" ng-click="buttonclick(button.name, item)" title="{{button.hint}}" data-original-title="{{button.hint}}">{{button.label}}</a>
  <a ng-if="button.icon" href="javascript:void(0)" role="button" class="btn-ref btn btn-{{button.color || 'default'}}" ng-click="buttonclick(button.name, item)" title="{{button.hint}}" data-original-title="{{button.hint}}">
    <span class="icon icon-{{button.icon}}" aria-hidden="true"></span>
    <span class="sr-only">{{button.hint}}</span>
  </a>
</td>

Since we have not done any work on gathering up any actual data at this point, we can skip the value portion of the cell for now and just throw in a hard-coded zero so that we can fire it up and take a look. Later, once we figure out how to insert the values, we can come back around and clean that up a bit, but for now, we just want to be able to pull up the page and see how the UI portion comes out on the screen. So our temporary code for the aggregate columns will start out looking like this:

<td ng-repeat="aggregate in data.aggarray" class="text-right" ng-class="{selected: item.selected}" tabindex="0">
  0
</td>

That should be enough to take a quick peek and see how things are working out. Here is the new test page with the column headings and columns, but no actual data.

Test page with column headings and mocked-up data

Beautiful! That takes care of another one of the easy parts. Now we are going to have to dig a little deeper and start figuring out how we can replace those zeroes with some real data. To begin, I got into the Server script section of the widget and did a Find on the word btnarray. My thought here was that anywhere there was code related to the buttons and icons, there should probably be similar code for the aggregates. The first thing that I came across was this comment:

* data.btnarray = the array of button specifications

So, I added a new comment right above that one.

* data.aggarray = the array of aggregate column specifications

The next thing I found was this line used to copy in all of the widget options:

	// copy to data[name] from input[name] || options[name]
	optCopy(['table', 'table_name', 'buttons', 'btns', 'refpage', 'bulkactions', 'btnarray', 'refmap', 'actarray',
		'p', 'o', 'd', 'filter', 'filterACLs', 'fields', 'field_list', 'keywords', 'view', 'relationship_id',
		'apply_to', 'apply_to_sys_id', 'window_size', 'show_breadcrumbs']);

… which I modified to be this:

	// copy to data[name] from input[name] || options[name]
	optCopy(['table', 'table_name', 'aggregates', 'buttons', 'btns', 'refpage',
		'bulkactions', 'aggarray', 'btnarray', 'refmap', 'actarray', 'p', 'o', 'd',
		'filter', 'filterACLs', 'fields', 'field_list', 'keywords', 'view',
		'relationship_id', 'apply_to', 'apply_to_sys_id', 'window_size',
		'show_breadcrumbs']);

Then I came across this block of code:

	if (data.buttons) {
		try {
			var buttoninfo = JSON.parse(data.buttons);
			if (Array.isArray(buttoninfo)) {
				data.btnarray = buttoninfo;
			} else if (typeof buttoninfo == 'object') {
				data.btnarray = [];
				data.btnarray[0] = buttoninfo;
			} else {
				gs.error('Invalid buttons option in SNH Data Table widget: ' + data.buttons);
				data.btnarray = [];
			}
		} catch (e) {
			gs.error('Unparsable buttons option in SNH Data Table widget: ' + data.buttons);
			data.btnarray = [];
		}
	} else {
		if (!data.btnarray) {
			data.btnarray = [];
		}
	}

So, I copied that and altered the copy to be this:

	if (data.aggregates) {
		try {
			var aggregateinfo = JSON.parse(data.aggregates);
			if (Array.isArray(aggregateinfo)) {
				data.aggarray = aggregateinfo;
			} else if (typeof aggregateinfo == 'object') {
				data.aggarray = [];
				data.aggarray[0] = aggregateinfo;
			} else {
				gs.error('Invalid aggregates option in SNH Data Table widget: ' + data.aggregates);
				data.aggarray = [];
			}
		} catch (e) {
			gs.error('Unparsable aggregates option in SNH Data Table widget: ' + data.aggregates);
			data.aggarray = [];
		}
	} else {
		if (!data.aggarray) {
			data.aggarray = [];
		}
	}

And that was it. None of that, of course, has anything to do with calculating the values for the new aggregate columns, but we won’t find that code by hunting for the btnarray variable. We are going to have to root around in the code that fetches the records to figure out where we need to add some logic to get the new values. That sounds like a good place to start in our next installment.

Aggregate List Columns

“Get a good idea and stay with it. Do it, and work at it until it’s done right.”
Walt Disney

We have had a lot of fun with the Service Portal Data Table Widget on this site. So much so, in fact, that we had to make our own copy to avoid extensive modifications to a core component of the Service Portal. So far, we have created a Configurable Data Table Widget Content Selector, allowed individual columns to be links to referenced records, added buttons and icons, added User Avatars to user columns, set up the Configurable Data Table Widget Content Selector to use a JSON object to configure the widget, created an Editor for the JSON configuration object, added check boxes to the rows, added an additional extension of the base Data Table Widget to use the JSON configuration object directly, and built a User Directory using all of these custom components. That’s quite a bit of customization, but there is at least one more thing that we could do to make this all even better.

The feature that I have in mind is to have one or more columns that would include counts of related records. For example, on a list of Assignment Groups, you might want to include columns for how many people are in the group or how many open Incidents are assigned to the group. These are not columns on the Group table; these are counts of related records. It seems as if we could borrow some of the concepts from our Buttons and Icons strategy to come up with a similar approach to configuring Aggregate List Columns that could be defined when setting up the configuration for the table. You know what I always like to ask myself: How hard could it be?

Let’s take a quick look at what we need to configure a button or icon using the current version of the tools. Then we can compare that to what we would need to configure an Aggregate List Column using a similar approach. Here is the data that we collect when defining a button/icon:

  • Label
  • Name
  • Heading
  • Icon
  • Color
  • Hint
  • Page

The first three would appear to apply to our new requirement as well, but the rest are specific to the button/icon configuration. Still, we could snag those first three, and copy all of the code that deals with those first three, and then ditch the rest and replace them with data points that will be useful to our new purpose. Our list, then, would end up looking something like this:

  • Label
  • Name
  • Heading
  • Table
  • Field
  • Filter

The first three would be treated exactly the same way that their counterparts are treated in the button/icon code. The rest would be unique to our purpose. Table would be the name of the table that contains the related records to be counted. Field would be the name of the reference field on that table that would contain the sys_id of the current row. Filter would be an additional query filter that would be used to further limit the records to be counted. The purpose for the Filter would be to provide for the ability to count only those records desired. For example, a Table of Incident and a Field of assigned_to would count all of the Incidents ever assigned to that person, which is of questionable value. With the ability to add a Filter of active=true, that would limit the count to just those Incidents that were currently open. That is actually a useful statistic to add to a list.

One other thing that would be useful would be the addition of a link URL that would allow you to pull up a list of the records represented in the count. Although I definitely see the value in this additional functionality, anyone who has followed this web site for any length of time knows that I don’t really like to get too wild and crazy right out of the gate. My intent is to just see if I can get the count to appear on the list before I worry too much about other features that would be nice to have, regardless of their obvious value.

So, it seems as if the place to start here would be to pull up a JSON configuration object and see if we can add a configuration for an Aggregate List Column. When we built the User Directory, we added a simple configuration file to support both the Location Roster and the Department Roster. This looks like a good candidate to use as a starting point to see if we can set up a test case. Here is the original configuration file used in the User Directory project:

var RosterConfig = Class.create();
RosterConfig.prototype = Object.extendsObject(ContentSelectorConfig, {
	initialize: function() {
	},

	perspective: [{
		name: 'all',
		label: 'All',
		roles: ''
	}],

	state: [{
		name: 'department',
		label: 'Department'
	},{
		name: 'location',
		label: 'Location'
	}],

	table: {
		all: [{
			name: 'sys_user',
			displayName: 'User',
			department: {
				filter: 'active=true^department={{sys_id}}',
				fields: 'name,title,email,location',
				btnarray: [],
				refmap: {
					cmn_location: 'location_roster'
				},
				actarray: []
			},
			location: {
				filter: 'active=true^location={{sys_id}}',
				fields: 'name,title,department,email',
				btnarray: [],
				refmap: {
					cmn_department: 'department_roster'
				},
				actarray: []
			}
		}]
	},

	type: 'RosterConfig'
});

For our purpose, we don’t really need two state options, so we can simplify this even further by reducing this down to just one that we can call all. Then we can add our example aggregate configuration just above the button configuration. Also, since this is just a test, we will want to limit our list of people to just members of a single Assignment Group, so we can update the filter accordingly to limit the number of rows. Here is the configuration that I came up with for an initial test.

var AggregateTestConfig = Class.create();
AggregateTestConfig.prototype = Object.extendsObject(ContentSelectorConfig, {
	initialize: function() {
	},

	perspective: [{
		name: 'all',
		label: 'All',
		roles: ''
	}],

	state: [{
		name: 'all',
		label: 'All',
	}],

	table: {
		all: [{
			name: 'sys_user',
			displayName: 'User',
			all: {
				filter: 'active=true^sys_idIN46c4aeb7a9fe1981002bbd372644a37b,46d44a23a9fe19810012d100cca80666,5137153cc611227c000bbd1bd8cd2005,5137153cc611227c000bbd1bd8cd2007,9ee1b13dc6112271007f9d0efdb69cd0,f298d2d2c611227b0106c6be7f154bc8',
				fields: 'name,title,email,department,location',
				aggarray: [{
					label: 'Incidents',
					name: 'incidents',
					heading: 'Incidents',
					table: 'incident',
					field: 'assigned_to',
					filter: 'active=true'
				}],
				btnarray: [],
				refmap: {},
				actarray: []
			}
		}]
	},

	type: 'AggregateTestConfig'
});

For now, I just created a simple filter using all of the sys_ids of all of the members of the Hardware group rather than bringing in the group member table and filtering on the group itself. This is not optimum, but this is just a test, and it will avoid other issues that we can discuss at a later time. The main focus at this point, though is the new aggarray property, which includes all of the configuration information that we listed earlier.

aggarray: [{
   label: 'Incidents',
   name: 'incidents',
   heading: 'Incidents',
   table: 'incident',
   field: 'assigned_to',
   filter: 'active=true'
}]

Now that we have a configuration script, we can create a page and try it out, even though we have not yet done any coding to support the new configuration attributes. At this point, we just want to see if the list comes up, and since nothing will be looking for our new data, that portion of the configuration object will just be ignored. I grabbed a copy of the Location Roster page from the User Directory project and then cloned it to create a page that I called Aggregate Test. Then I edited the configuration for the table to change the Configuration Script value to our new Script Include, AggregateTestConfig. I also removed the value that was present in the State field, as we only have one state, so no value is needed.

Updating the Configuration Script on the cloned portal page

With that saved, we can run out to the Service Portal and pull up the new page and see how it looks.

First look at the new test page using our new configuration script

Well, that’s not too bad for just a few minutes of effort. Of course, that was just the easy part. Now we have to tinker with the widgets to actually do something with this new configuration data that we are passing into the modules. That’s going to be a bit of work, of course, so we’ll start taking a look at how we want to do that next time out.

Collaboration Store, Part LVI

“Ideas don’t happen in isolation. You must embrace opportunities to broadcast and then refine your ideas through the energy of those around you.”
Scott Belsky

Now that we have released the latest version of the Collaboration Store Update Sets for testing purposes, we should take a quick look ahead to see where things might go from here. Obviously, the first order of business is to validate the existing build and work to put out that 1.0 version with the current feature set. However, it won’t hurt to take a moment and look a little bit beyond that to see what may lie ahead for this app. This first version accomplishes the three main goals of providing the ability to set-up an instance, publish an app, and install an app, but there are a lot things missing that would be helpful and/or really nice to have. So let’s take a look at that list.

Images

One of the things that has disturbed me for quite some time is the fact that the logo image for an app is not included in the Update Set when you select the Publish to Update Set… option on the Custom Application form. We use that very same function to move an app into the Store, so our process suffers from the same shortcoming. We should be able to add some extra processing to our workflow, however, that could work around this issue. One of the things that I thought might provide a solution would be to copy the image file attached to the app and attach the copy to the Collaboration Store application record. Then during the installation process, we could copy the image from the application record over to the installed app once the application installation process was completed. That would also provide the opportunity to display the application logo on the application form, which could be done in much the same way as the user photo is displayed on the User Profile form.

Speaking of images, I have also thought from the beginning that each instance in the community should have its own image as well, and that all of the applications shared by that instance should include the providing instance’s image as well as the specific image of the application. This would provide additional visual clues when searching through the apps in the store.

Conditionals and UI Policies

What you can and cannot do with certain records and fields needs to be tied to the original owner of the artifact. For example, the Install button should not appear on version records for your own applications. Conversely, the Publish to Collaboration Store link should not appear on applications that are not your own. And all of the form fields on records that did not originate in your instance should be protected, as you should not be allowed to make changes to records that are not your own. Also, there are certain fields that should never be updated under any circumstances (such as the version number), as these are determined by background processes and should never be editable. There is a lot of work that needs to be done in this area, and it probably should be done before the initial 1.0 version is released to the general public.

Shopping

Version 1.0 was all about getting the mechanics to work, making sure that a scoped app on one instance could be shared, stored, distributed, and installed on some other instance. Once testing establishes that all of those things work as they should, though, it will be time to start looking into setting up a much better experience for locating an app that looks interesting or meets a specific purpose. Images will help create a more visual, catalog shopping type of searching, but so will key words and categories and even marketing tools and programmatic hints such as “People who installed this app also installed these apps:”. User ratings, comments, error reports, and other feedback would also help improve the shopping experience. Also, statistics on how many instances have installed the app and which versions were in use would be useful information when looking at available options. So many possibilities; so little time.

Activity Tracking

From the onset of this project I have always thought that there should be some kind of running log of the activities within each instance, particularly the Host. I set up a table for this early on when I was first setting up the tables for the app, but there is no code in this version to write to this table for any reason. The next major release, if there is such a thing, should really complete this functionality and log every movement of data between the Host and the Clients. Maybe a simple logActivity function could be crafted to accept certain arguments and then all you would have to do to add logging would be to add a single line of code to call that shared function. That sounds like a project, though, but it needs to be done at some point.

Code Consolidation

A while back I created a series of shared function to replace a collection of cloned functions so that I would not clone them all for yet a third time. In my new use case, I utilized the shared functions that were designed to work in all three places, but I was too lazy to go back and refactor the original two sections of code where I had done the initial cloning. I really need to take the time to go back and rework the original and that first copy to have both of them use the redesigned functions that were designed to work for all of the places where I needed to ship instance data, application data, version data, and XML Update Set attachments from one instance to another. It works the way that it is right now, but for future maintenance (and just decent coding standards), that all needs to be addressed.

Additional Artifacts

This initial version allows you to share Scoped Applications between unrelated instances in the same community (Clients of the same Host). That is a good start, but it would also be nice to be able to share single artifacts or global Update Sets or any other component or collection developed on the Now Platform. That was a little more than I was willing to take on at the onset of this project, but now that the sharing of apps between instances seems to be a reality, it would be nice to branch out a little and start adding more stuff that could be shared. Once again, I seem to have more ideas than I do free time. Still, it would be nice to do more than just whole apps.

First Things First

Still, the original goal was to see if we could just make this work, and although it seems like everything is functioning as it should, every little thing needs to be tested out. Hopefully, a few brave souls are actually busy doing that very thing while I type this out, so maybe we will get some much needed feedback relatively soon and see if this thing actually works for anyone other than myself! As always, any feedback of any kind is always appreciated. Thanks in advance to those of you who have actually pulled this down and given things a go.

Collaboration Store, Part LV

“A person who tries has an advantage over the person who wishes.”
Utibe Samuel Mbom

Theoretically, all of the artifacts are complete for this initial version of the Collaboration Store, but now that we have crossed that threshold, we need to really shake everything out before we can build that final 1.0 version that will be solid enough for public consumption. Before we do that, though, we should probably back up just a bit and explain the general purpose of the app, and go through a little bit of a user guide for the folks that might be joining this party a little more recently.

The concept for the initial version of the app is pretty simple. It allows developers from one ServiceNow instance to share Scoped Applications with developers on a different instance. There are three primary functions included in the app: 1) the initial set-up process, 2) the application publishing (sharing) process, and 3) the application installation process. One instance in the community needs to be designated as the Host instance, and all other instances are considered Client instances. The Host instance needs to be set up first, as the set-up process for a Client instance requires communication with the identified Host. Once an application has been pushed to the Host, the Host instance then distributes that application to all of the Clients in the community. Once any instance has a version of a shared application, developers can then install that application on their own instance.

Initial Set-up

Once all of the artifacts have been installed, the first thing that you need to do on your instance is to run the set-up process. To enter the set-up process, select Collaboration Store Set-up from the Collaboration Store section of the left-hand nav. You will need to be in the Collaboration Store scope, and if you are not already in that scope, you will be prompted to make the switch.

Initial set-up screen

The form is fairly self-explanatory, but you need to make sure that you have selected the correct Installation Type, and if you are setting up a Client instance, you will want to make sure that you have entered the correct Host instance ID, which is the first element of the URL of the instance, the unique portion that precedes the .service-now.com portion of the instance address. Also, you will want to enter a valid email address to which you have access, as an email will be sent to that address with a verification code, which you will need to enter on the next screen.

Email verification screen

Once the email has been validated, the set-up process commences and the final screen of the set-up process appears when the set-up process is complete.

Set-up completion screen

Application Publishing

Once your instance has been set up, you should now be able to publish applications to the store. This is accomplished through a new link that the application has added to the Scoped Application form. A while ago I created a scoped app called Simple Webhook when I was playing around with outbound Webhooks, and we can use that as an example to give this thing a go. To publish the app to the store, pull up the application form and select the Publish to Collaboration Store link down at the bottom of the form.

Publishing a scoped application

Clicking on this link will pop up the Publish to Collaboration Store dialog where you will enter the details of this version of the application.

Publish to Collaboration Store dialog

Clicking on the Publish button will then bring up the Export to XML progress bar while the Update Set is converted to an XML file.

Export to XML progress bar

Clicking on the Done button will then bring up another dialog box where you can see the progress of all of the other steps involved in creating the Collaboration Store records, attaching the XML file, and sending all of the artifacts over to the Host instance.

Publish to Collaboration Store completion dialog

Clicking on the Done button here will close the dialog and reload the form, completing the process. Once the last artifact reaches the Host, that will trigger a background process that will distribute the app to all of the other instances in the community. You don’t need to worry about that, though; that all goes on behind the scenes and takes care of itself. If you are testing, however, and hopefully you are (the more, the merrier!), you will want to check all of the other instances in the community to ensure that all of the artifacts arrived safely and in good condition. That’s the whole point of the app, obviously, so we want to make sure that it all works as intended.

Now if things do take a wrong turn somewhere along the line, there is another background job that runs on the Host to check with all of the Clients each day and sends over any missing artifacts to keep everything in sync. That’s another thing that we will want to test out thoroughly, which may require introducing some intentional errors just to see if the daily sync process finds and corrects them.

Application Installation

Once another instance has shared an app, you should be able to install it on your own instance, which is accomplished in this version of the app by navigating to the version record for the version that you want to install and clicking on the Install button. This should kick off a series of events starting with the conversion of the XML file attachment back into an Update Set, Previewing that Update Set, correcting any errors detected in the Preview, then Committing the Update Set and updating the Collaboration Store data to reflect the installation of the app. Once again, using our Simple Webhook app as example, let’s pull up the version record on a different instance and click on that Install button.

Installing a shared application

After clicking on the Install button, you will first see the XML file being uploaded to the server.

Update Set XML file being uploaded to the server

Once the XML file has been uploaded and converted back into an Update Set, you will see the Preview progress bar.

Preview progress bar

Once the Preview has been completed, you will see the Commit progress bar, which looks very similar to the Preview progress bar.

Commit progress bar

Once the Commit has been completed, the Collaboration Store records will be updated and then you will be returned to the version record, where the Install button will no longer appear, since this version has now been installed.

Version record after installation

Also, if you preview the associated application record, you will see that the Collaboration Store application has been linked to the installed application and the installed application’s version matches the latest version of the app.

Application record linked to installed application

Let the Testing Begin!

OK, that’s it, the initial set-up process, the application publishing process, and the application installation process. Seems pretty simple for something that took 55 episodes to complete, but there is a lot going on under the hood behind all of those little pop-up screens. Of course, if you want to do any testing, you will need something to test, so here are the artifacts that you will need to install, in the order in which they need to be installed, if you want to take this out for a spin:

  • If you have not done so already, you will need to install the most recent version of the snh-form-field, tag, which is needed for the initial set-up widget. You can find that here.
  • Once you have the snh-form-field tag installed, you can install the newest Scoped Application Update Set.
  • And then once the application has been installed, you can install the Update Set for the additional global components that could not be included in the Scoped Application.

Once you have everything installed, you can go through the initial set-up process and then you should be good to publish and install shared applications. As usual, all feedback is welcome and encouraged. Please pull it down and give it a try, and please let me know what you find, good, bad, or indifferent. If we get any comments, we will take a look at those next time out.