Parts is Parts

“Parts is parts.”
Wendy’s TV commercial

I love making parts. One of the reasons that ServiceNow is such a powerful platform is that it is built on reusable parts, and the platform encourages the development and use of reusable parts. I have known a number of developers who are extraordinary coders that can lay down reliable code at an amazing rate of speed, but no one can code faster than they can pull an existing part down from a shelf. When you build a part in such a way that you can reuse it again in another context, you not only solve your current problem, but you also create the solution for problems that you haven’t even encountered yet. That doesn’t just improve productivity, it also increases reliability. Parts on the shelf are on the shelf because you have used them somewhere before, and if you’ve used them before, then you’ve tested them before, which means that you’ve already gone through the exercise of shaking out all of those initial bugs. Faster and better — it’s a win/win.

When I started my Highcharts experiment, my goal was to create a reusable component for displaying any Highcharts graph. I was able to do that with my Generic Chart widget, but in the process, I also ended up showcasing a number of the other little parts and pieces that have been developing on the Now Platform. For example, when I wanted to add the ability to click on the chart and drill down to the underlying data, I ended up linking to my enhanced Data Table widget. I actually built that to support my configurable content selector, but once created (and tested) and placed on the shelf, it was there to later pull down, dust off, and utilize for other, previously unforeseen purposes.

To make the various selections for the four different parameters used in the workload chart, I ended up using the angular provider that I put together to produce form fields on the Service Portal. I didn’t create that to support the workload chart, but once it was created, there is was on the shelf to pull down and put to use.

I built my dynamic Server Portal breadcrumbs because I really didn’t like the way that the out-of-the-box breadcrumbs widget required you to pre-define the trail of pages displayed by the widget. That philosophy assumes that each of your pages can only be reached through a single path. For someone who likes to build and leverage reusable parts, this just seemed a little too restrictive to me. This was yet another “part” that I had build with my content selector in mind, but which was equally useful once I started linking out from my workload chart. In fact, it was while working with my workload chart that I discovered a flaw in my breadcrumbs widget. That’s another nice thing about working with reusable parts: when you fix a problem, you don’t just fix it for your purposes; you fix it for everyone else who is using it for whatever other purpose. That’s the difference between cloning and reusing. If you clone a part and find a flaw, you fix your copy, but the original and any other clones are unaffected. If you reuse a part and fix a flaw, you fix it for everyone.

That’s why I love making parts.

More Fun with Highcharts

“Never give up on something that you can’t go a day without thinking about.”
Winston Churchill

Those of us who develop software for a living always like to blame the customer for the inevitable scope creep that works its way into an assignment or project. The real truth of the matter, though, is that a lot of that comes right from the developers themselves. Often, just when you think you are about to wrap something up and call it complete, you get that nagging you-know-it-would-be-even-better-if-we-added-this feeling that just won’t go away.

It was my intention to make my previous installment on the Highcharts Workload Chart my last and final offering on the subject. Wait … this sounds way too familiar. OK, fine … I have a bad habit of continuing to tinker with stuff long after it should have been put to bed. But, I did learn something new this time, so I think it was worth the return trip. All I really wanted to do was to add a couple more relevant charts to my workload status page:

Work Distribution and Aging Charts

I already had the Generic Chart widget in hand, so I just needed to drop it on the page a couple more times and run a few more queries to fetch the relevant data. How hard could it be? Well, experience gives us the answer to that question! As usual, things didn’t go quite as smoothly as I had anticipated. It turns out that the Generic Chart widget contains a fatal flaw that only allows it to be used once per page. In the HTML for the widget, I hard-coded the ID of the DIV that will contain the chart, which you have to pass to Highcharts so that the chart will be rendered in that specific DIV. Well, when you put more than one Generic Chart widget on a page, they all want to render their charts in the first instance of a DIV with that ID. That’s not going to work!

The fix wasn’t too bad, but it did require a fix. The revised HTML now looks like this:

<div class="panel panel-default">
  <div class="panel-body form-horizontal">
    <div class="col-sm-12">
      <div id="{{data.container}}" style="height: 400px;"></div>
    </div>
  </div>
</div>

To populate the new data.container variable, I set up yet another widget option, and then set up a default value for the option so that you would only need to mess with this if you were working on a multi-chart page. That, and the addition of the code to populate the two other charts, one a Pie Chart and one a Bar Chart, were really the only other additions. If you want to take a look at the whole thing in action, here is the most recent Update Set.

Fun with Highcharts, Part VII

“Time, which alone makes the reputation of men, ends by making their defects respectable.”
Voltaire

Once again, things were not quite as bad as I had originally envisioned. I definitely disliked returning to my workload chart page and having it revert back to all of the original, default settings. This was not good. But how best to fix it? I didn’t have that problem with my Data Table Content Selector, which also has multiple options, but that was built using a little bit different approach. Although there are two independent widgets on the page, they don’t really communicate with one another. The selector on the current page communicates with the Data Table widget on the next page, passing all of the information via URL parameters when it builds the URL for the next page. Reconfiguring the workload chart widget and underlying generic chart widget to adopt this philosophy seemed like a lot more work that I really wanted take on right at the moment.

Instead, I decided to leverage the same hack that I used in my dynamic breadcrumbs widget, leveraging the User Preferences feature to preserve the selections and then picking them back up each time the page loads. Here is the additional code I added to save the user’s selections in a User Preference that I called workload_selections:

var selections = {};
selections.group = data.group;
selections.type = data.type;
selections.frequency = data.frequency;
selections.ending = data.ending;
gs.getUser().setPreference('workload_selections', JSON.stringify(selections));

User Preferences are strings, so you have to convert the object to a string before you save it, and convert it back to an object when you fetch it back. Here is the code that I used to retrieve it once the page loads again:

selections = gs.getPreference('workload_selections');
if (selections) {
	selections = JSON.parse(selections);
	if (selections.group) {
		data.group = selections.group;
		data.type = getDefaultType();
		data.type = selections.type;
		data.frequency = selections.frequency;
		data.ending = selections.ending;
	}
}

You might have noticed that the value for the type selection was set twice, once to the default value and then again to the value saved in the User Preference. That’s because the getDefaultType function is actually a dual purpose function; not only does it return the default type value, but before it does so, it sets up the type options that are appropriate for the selected group (different groups are associated with different types of tasks). Since the group value preserved may be different than the default value, the appropriate list of type choices needs to be generated for that group before the preserved type value is established. Other than that, it’s pretty straightforward stuff.

That resolved the last known defect that I have been able to find so far, so it’s finally time to release another Update Set. If you happen to pull it down and try to use it, let me know if you find any others … thanks!

Update: There is a better (expanded) version here.

Fun with Highcharts, Part VI

“If you’re going through Hell, keep going.”
Someone other than Winston Churchill

Well, it turns out that it wasn’t as bad as I had originally imagined. I converted my server side GenericChartUtil Script Include into a client side UI Script, then created a Widget Dependency referencing the UI Script, and then finally associated the Widget Dependency to my Generic Chart widget. That pushed the chart object generation from the server side to client side (where I would no longer lose any functions in the chart object), but to make it all work, I needed to pass the chart data and chart type around rather than the completed chart object.

On my Generic Chart widget, I removed the chartObject option and replaced it with two new widget options, chartType and chartData. Then I added a line of code in the client side script to pass the chartData and chartType to the new client side UI Script to generate the chartObject. The client side code for the widget now looks like this:

function($scope, $rootScope, $location) {
	var c = this;
	if (c.data.chartData) {
		c.data.chartData.location = $location;
		$scope.chartOptions = genericChartUtil.getChartObject(c.data.chartData, c.data.chartType);
	}
	if (c.options.listen_for) {
		$rootScope.$on(c.options.listen_for, function (event, config) {
			if (config.chartData) {
				config.chartData.location = $location;
				$scope.chartOptions = genericChartUtil.getChartObject(config.chartData, config.chartType);
			}
		});
	}
}

On my Workload Chart widget, I removed all references to the deleted Script Include on the server side, and then modified the broadcast message on the client side to pass the chartData and chartType rather than the entire generated chartObject. That code now looks like this:

function($scope, $rootScope) {
	var c = this;
	$scope.updateChart = function() {
		c.server.update().then(function(response) {
			c.data.config = response.config;
			c.data.group = response.group;
			c.data.type = response.type;
			c.data.frequency = response.frequency;
			c.data.ending = response.ending;
			c.data.chartData = response.chartData;
			$rootScope.$broadcast('refresh-workload', {chartData: c.data.chartData, chartType: 'workload'});
		});
	}
}

That solved my earlier problem of losing the functions built into the chart objects that were generated on the server side. Now I could get back to what I was trying to do in the first place, which was to set things up so that you could click on any given data point on the chart and pull up a list of the records that were represented by that value. This code was now working as it should:

plotOptions: {
        series: {
            cursor: 'pointer',
            point: {
                events: {
                    click: function () {
                        alert('Category: ' + this.category + ', value: ' + this.y);
                    }
                }
            }
        }
    },

For my link URL to show the list of records represented by the chart item clicked, I was going to need the name of the table and the filter used in the GlideAggregate that calculated the value. Neither one of those was currently passed in the chart data, so the first thing that I needed to do was to modify the code that generated the chartData object to include those values. That code now looks like this:

function gatherChartData() {
	var task = new GlideAggregate(data.type);
	var periodData = getPeriodData();
	var chartData = {};
	chartData.table = data.type;
	chartData.filter = {Received: [], Completed: [], Backlog: []};
	var filter = '';
	chartData.title = task.getPlural() + ' assigned to ' + findOption(data.config.groupOptions, data.group).label;
	chartData.subtitle = periodData.frequencyInfo.label + ' through ' + periodData.endingDateInfo.label;
	chartData.labels = periodData.labels;
	chartData.received = [];
	chartData.completed = [];
	chartData.backlog = [];
	for (var i=1; i<periodData.endDate.length; i++) {
		// received
		filter = 'assignment_group=' + data.group + '^opened_at>' + periodData.endDate[i-1] + '^opened_at<=' + periodData.endDate[i];
		task.initialize();
		task.addAggregate('COUNT');
		task.addEncodedQuery(filter);
		task.query();
		task.next();
		chartData.received.push(task.getAggregate('COUNT') * 1);
		chartData.filter.Received.push(filter);
		// completed
		filter = 'assignment_group=' + data.group + '^closed_at>' + periodData.endDate[i-1] + '^closed_at<=' + periodData.endDate[i];
		task.initialize();
		task.addAggregate('COUNT');
		task.addEncodedQuery(filter);
		task.query();
		task.next();
		chartData.completed.push(task.getAggregate('COUNT') * 1);
		chartData.filter.Completed.push(filter);
		// backlog
		filter = 'assignment_group=' + data.group + '^opened_at<=' + periodData.endDate[i] + '^closed_at>' + periodData.endDate[i] + '^ORclosed_atISEMPTY';
		task.initialize();
		task.addAggregate('COUNT');
		task.addEncodedQuery(filter);
		task.query();
		task.next();
		chartData.backlog.push(task.getAggregate('COUNT') * 1);
		chartData.filter.Backlog.push(filter);
	}
	return chartData;
}

Since I need the filter value for multiple purposes, I first assigned that to a variable, and then used that variable wherever it was needed. Inside of Highcharts, my only reference to the series was the category name, which is why there is an odd capitalized property key for the chartData.filter properties (the category names are labels on the chart, so they are capitalized). These changes gave me the data to work with so that I could modify the onclick function to look like this:

plotOptions: {
	series: {
		cursor: 'pointer',
		point: {
			events: {
				click: function (evt) {
					var s = {id: 'snh_list', table: chartData.table, filter: chartData.filter[this.series.name][this.index]};
					var newURL = chartData.location.search(s);
					spAriaFocusManager.navigateToLink(newURL.url());
				}
			}
		}
	}
},

In order for that to work, I need to pass the $location object to Highcharts as well, so my client side GenericChart widget code ended up looking like this:

function($scope, $rootScope, $location) {
	var c = this;
	if (c.data.chartData) {
		c.data.chartData.location = $location;
		$scope.chartOptions = genericChartUtil.getChartObject(c.data.chartData, c.data.chartType);
	}
	if (c.options.listen_for) {
		$rootScope.$on(c.options.listen_for, function (event, config) {
			if (config.chartData) {
				config.chartData.location = $location;
				$scope.chartOptions = genericChartUtil.getChartObject(config.chartData, config.chartType);
			}
		});
	}
}

To get back to the chart itself, once you clicked on a data point to see the underlying records, I added my dynamic breadcrumbs widget to the the top of the chart page and to the top of a new list page that I created for this purpose. Now it was time to test things out …

Well, as often seems to be the case with these things, there is good news and bad news. The good news is that clicking on the data points on the chart actually does bring up the list of records, which is very cool. Even though I had to do a considerable amount of restructuring of my initial concept, everything now seems to work. And I like the feature. Clicking on a bar or point on the line now takes you to a list of the records that make up the value for that data point. And when you are done, you can click on the breadcrumb for the chart and get back to the chart itself. All of that works beautifully.

Unfortunately, when you get back to the chart page, it reverts to the original default settings for all of the options. If you used any of the four selections at the top of the page to get to a specific chart configuration, and then clicked on a data point to see the underlying records, when you returned to the chart, you were no longer looking at the chart that you had selected. You were back to the original, default values for all of the selections. The page basically restarts from the beginning. That, I do not like at all.

The primary reason for that behavior is that your chart option selections are not part of the URL, so they are not preserved when you click on the breadcrumb to return. Making them part of the URL would mean another complete reconfiguration of the way in which that chart works, but something is going to have to be done. I don’t like it the way that it works now. Originally, I was going to release a new Update Set with all of these changes, but I’m not happy with the way things are working right now. I’m going to have to do a little bit more work before I’m ready to release another version. Hopefully, I can do that next time out.

Fun with Highcharts, Part V

“It’s always something.”
Roseanne Roseannadanna

I decided to enhance my workload chart by adding the ability click on a bar or point on the backlog line and bring up a list of all of the tasks represented by that data on the chart. Highcharts makes that super simple to do, so I went out and grabbed some example code off of the Interwebs and added this to my workload chart template:

plotOptions: {
        series: {
            cursor: 'pointer',
            point: {
                events: {
                    click: function () {
                        alert('Category: ' + this.category + ', value: ' + this.y);
                    }
                }
            }
        }
    },

That was enough to prove that everything worked, after which I was going to modify the click function to navigate to a Data Table page with the appropriate filter to list out all of the tasks represented by the data point selected. Unfortunately, when I fired it up to test it out, nothing happened when I clicked on a bar or a point along the line. The cursor did change to a pointer, so that part was definitely working, but no alerts popped up no matter where I clicked. I hate it when that happens!

It took me a little while to figure this out, but after a little bit of digging around I finally reached the heart of the issue: I built the chart object on the server side, and when it was transferred to the client side where it was to be used, it was converted to a JSON string somewhere along the lines in the background Ajax process, and that conversion removed the function (JSON only preserves data; any functions are lost in translation). While it was nice to finally understand the root of the problem, the implication was that my whole way of going about this was pretty much invalidated. I can’t build a chart object on the server side and then pass it to the client side and retain any functions that might be a part of the object. The chart object will need be generated on the client side where it will be used. That means a total redesign of the entire concept.

Well, I guess that guy Charles Lauller knew what he was talking about. Time to start over with a blank sheet of paper

Fun with Highcharts, Part IV

“Lost wealth may be replaced by industry, lost knowledge by study, lost health by temperance or medicine, but lost time is gone forever.”
Samuel Smiles

Now that we have our generic chart widget, our generic chart object generator, and the choice lists and default selections for our example chart, we just need to come up with the code to gather up the data for the chart based on the selections. A number of elements are based on the period selected, so collecting that data would seem like an important first step. I organized all of that into a function that I called getPeriodData:

function getPeriodData() {
	var periodData = {};
	periodData.frequencyInfo = findOption(data.config.freqOptions, data.frequency);
	periodData.endingDateInfo = findOption(data.config.endingOptions[data.frequency], data.ending);
	periodData.labels = [];
	periodData.endDate = [];
	for (var i=0; i<=periodData.frequencyInfo.size; i++) {
		var dt = new Date(periodData.endingDateInfo.label);
		if (data.frequency == 'd') {
			dt.setDate(dt.getDate() - (periodData.frequencyInfo.size - i));
		} else if (data.frequency == 'w') {
			dt.setDate(dt.getDate() - (periodData.frequencyInfo.size - i) * 7);
		} else if (data.frequency == 'm') {
			dt.setDate(1);
			dt.setMonth(dt.getMonth() - (periodData.frequencyInfo.size - i));
			dt = getLastDayOfMonth(dt);
		} else if (data.frequency == 'q') {
			dt.setDate(1);
			dt.setMonth(dt.getMonth() - (periodData.frequencyInfo.size - i) * 3);
			dt = getLastDayOfMonth(dt);
		} else if (data.frequency == 'y') {
			dt.setFullYear(dt.getFullYear() - (periodData.frequencyInfo.size - i));
		}
		var dtInfo = getDateValues(dt);
		periodData.endDate.push(dtInfo.value);
		if (i > 0) {
			periodData.labels.push(dtInfo.label);
		}
	}			
		
	return periodData;
}

The function collects one more end date than it does labels because the end date of the previous period is used as the start of the current period. You need to be able to go back one extra period to get the end date of a period that you will not actually be using for the start date of the earliest period that you will.

Once you pull together all of the data for the selected frequency and period, you can then use that data to put together everything else needed for the chart. This takes care of the basics:

var chartData = {};
chartData.title = task.getPlural() + ' assigned to ' + findOption(data.config.groupOptions, data.group).label;
chartData.subtitle = periodData.frequencyInfo.label + ' through ' + periodData.endingDateInfo.label;
chartData.labels = periodData.labels;
chartData.received = [];
chartData.completed = [];
chartData.backlog = [];

… and then all that is left is to loop through all of the periods in the chart to run GlideAggregates to compile all of the actual data:

var task = new GlideAggregate(data.type);
for (var i=1; i<periodData.endDate.length; i++) {
	task.initialize();
	task.addAggregate('COUNT');
	task.addEncodedQuery('assignment_group=' + data.group + '^opened_at>' + periodData.endDate[i-1] + '^opened_at<=' + periodData.endDate[i]);
	task.query();
	task.next();
	chartData.received.push(task.getAggregate('COUNT') * 1);
	task.initialize();
	task.addAggregate('COUNT');
	task.addEncodedQuery('assignment_group=' + data.group + '^closed_at>' + periodData.endDate[i-1] + '^closed_at<=' + periodData.endDate[i]);
	task.query();
	task.next();
	chartData.completed.push(task.getAggregate('COUNT') * 1);
	task.initialize();
	task.addAggregate('COUNT');
	task.addEncodedQuery('assignment_group=' + data.group + '^opened_at<=' + periodData.endDate[i] + '^closed_at>' + periodData.endDate[i] + '^ORclosed_atISEMPTY');
	task.query();
	task.next();
	chartData.backlog.push(task.getAggregate('COUNT') * 1);
}

The last thing we need to deal with is the user making new selections from the four pick lists. That’s a client side issue, so we will need a client side script to detect the selections and call for a refresh of the chart.

function($scope, $rootScope) {
	var c = this;
	$scope.updateChart = function() {
		c.server.update().then(function(response) {
			c.data.config = response.config;
			c.data.group = response.group;
			c.data.type = response.type;
			c.data.frequency = response.frequency;
			c.data.ending = response.ending;
			$rootScope.$broadcast('refresh-workload', {chartObject: response.chartObject});
		});
	}
}

I still want to play around with a few more different and interesting chart types, but there are enough parts and pieces now to warrant the assembly of a version 1.0 Update Set. If I ever get a chance to do more, I can always put out a better one later on.

Fun with Highcharts, Part III

“It’s the little details that are vital. Little things make big things happen.”
John Wooden

So far, we have built a generic chart widget and a generic chart utility that produces chart objects from templates and dynamic data. Now it’s time to build the widget that will allow the user to pick and choose what they want to see, and then present the appropriate chart based on the user’s selections. Let’s start with a little HTML for the various pick lists that we will present to the user:

<link rel="stylesheet" type="text/css" href="/747e2219db213300f9699006db9619b9.cssdbx"/>
<link rel="stylesheet" type="text/css" href="/styles/retina_icons/retina_icons.css"/>
<div class="panel panel-default">
  <div class="panel-body form-horizontal">
    <div class="col-sm-12">
      <form id="form1" name="form1" novalidate>
        <div class="col-sm-3">
          <snh-form-field
            snh-model="c.data.group"
            snh-name="group"
            snh-label="Group"
            snh-type="choicelist"
            sn-value-field="value"
            sn-text-field="label"
            sn-items="c.data.config.groupOptions"
            ng-click="updateChart();"/>
        </div>
        <div class="col-sm-3">
          <snh-form-field
            snh-model="c.data.type"
            snh-name="type"
            snh-label="Task Type"
            snh-type="choicelist"
            sn-value-field="value"
            sn-text-field="label"
            sn-items="c.data.config.typeOptions"
            ng-click="updateChart();"/>
        </div>
        <div class="col-sm-3">
          <snh-form-field
            snh-model="c.data.frequency"
            snh-name="frequency"
            snh-label="Frequency"
            snh-type="choicelist"
            sn-value-field="value"
            sn-text-field="label"
            sn-items="c.data.config.freqOptions"
            ng-click="updateChart();"/>
        </div>
        <div class="col-sm-3">
          <snh-form-field
            snh-model="c.data.ending"
            snh-name="ending"
            snh-label="Period Ending"
            snh-type="choicelist"
            sn-value-field="value"
            sn-text-field="label"
            sn-items="c.data.config.endingOptions[c.data.frequency]"
            ng-click="updateChart();"/>
        </div>
      </form>
    </div>
    <div class="col-sm-12">
      <sp-widget widget="data.workloadWidget"></sp-widget>
    </div>
  </div>
</div>

This particular layout leverages our snh-form-field tag, but you could do basically the same thing with a simple sn-choice-list. There is nothing too exotic here, except maybe the lists of choices for the last element (Period Ending), which are contained in an object keyed by the value of the previous selection (Frequency). When you change the Frequency, the list of period ending dates changes to a list that is appropriate for the selected Frequency. Other than that one little oddity, it’s pretty vanilla stuff.

There are several reasons that I chose choicelist, which implements the sn-choice-list tag, for the snh-type of each pick list rather than reference, which implements the sn-record-picker tag. It would have been relatively easy to set up a filter on the sys_user_group table to create a record picker of active user groups, but I wanted to limit the choices to just those groups who had tasks assigned in the task table. That requires a GlideAggregate rather than a GlideRecord query, and I’m not sure how you would set that up in an sn-record-picker. For a choice list, you run the query yourself and then just set the value of the specified variable to an array created from your query results. For this widget, I created a config object to hold all of the choice list arrays, and used the following code to populate the array of choices for the group pick list:

cfg.max = 0;
cfg.maxGroup = '';
cfg.groupOptions = [];
var group = new GlideAggregate('task');
group.addAggregate('COUNT');
group.groupBy('assignment_group');
group.ordderBy('assignment_group');
group.query();
while (group.next()) {
	if (group.getDisplayValue('assignment_group')) {
		cfg.groupOptions.push({label: group.getDisplayValue('assignment_group'), value: group.getValue('assignment_group'), size: group.getAggregate('COUNT')});
		if (group.getAggregate('COUNT') > cfg.max) {
			cfg.max = group.getAggregate('COUNT');
			cfg.maxGroup = group.getValue('assignment_group');
		}
	}
}

For the choice list of task types, I wanted to wait until a group was selected, and then limit the choices to only those types that had been assigned to the selected group. This was another GlideAggregate, and that turned out to be very similar code:

var max = 0;
var defaultType = '';
data.config.typeOptions = [];
var type = new GlideAggregate('task');
type.addQuery('assignment_group', data.group);
type.addAggregate('COUNT');
type.groupBy('sys_class_name');
type.ordderBy('sys_class_name');
type.query();
while (type.next()) {
	if (type.getDisplayValue('sys_class_name')) {
		data.config.typeOptions.push({label: type.getDisplayValue('sys_class_name'), value: type.getValue('sys_class_name'), size: type.getAggregate('COUNT')});
		if (type.getAggregate('COUNT') > max) {
			max = type.getAggregate('COUNT') * 1;
			defaultType = type.getValue('sys_class_name');
		}
	}
}

The frequency choices, on the other hand, were just a hard-coded list that I came up with on my own. I wanted to be able to display the chart on a daily, weekly, monthly, quarterly, or yearly basis, so that’s the list of choices that I put together:

cfg.freqOptions = [
	{value: 'd', label: 'Daily', size: 7},
	{value: 'w', label: 'Weekly', size: 12},
	{value: 'm', label: 'Monthly', size: 12},
	{value: 'q', label: 'Quarterly', size: 8},
	{value: 'y', label: 'Yearly', size: 5}
];

The choices for period ending dates took a bit more code. For one thing, I needed a different list of choices for each frequency. For another, the methodology for determining the next date in the series was slightly different for each frequency. That meant that much of the code was not reusable, as it was unique to each use case. There is probably a way to clean this up a bit, but this is what I have working for now:

cfg.endingOptions = {d: [], w: [], m: [], q: [], y: []};
var todaysDateInfo = getDateValues(new Date());
var today = new Date(todaysDateInfo.label);
var nextSaturday = new Date(today.getTime());
nextSaturday.setDate(nextSaturday.getDate() + (6 - nextSaturday.getDay()));
dt = new Date(nextSaturday.getTime());
for (var i=0; i<52; i++) {
	cfg.endingOptions['d'].push(getDateValues(dt));
	cfg.endingOptions['w'].push(getDateValues(dt));
	dt.setDate(dt.getDate() - 7);
}
dt = new Date(today.getTime());
dt.setMonth(11);
dt = getLastDayOfMonth(dt);
cfg.endingOptions['y'].push(getDateValues(dt));
dt = new Date(today.getTime());
dt.setDate(1);
dt.setMonth([2,2,2,5,5,5,8,8,8,11,11,11][dt.getMonth()]);
dt = getLastDayOfMonth(dt);
cfg.endingOptions['q'].push(getDateValues(dt));
dt = new Date(today.getTime());
for (var i=0; i<36; i++) {
	dt = getLastDayOfMonth(dt);
	var mm = dt.getMonth();
	cfg.endingOptions['m'].push(getDateValues(dt));
	if (mm == 2 || mm == 5 || mm == 8 || mm == 11) {
		cfg.endingOptions['q'].push(getDateValues(dt));
	}
	if (mm == 11 && i != 0) {
		cfg.endingOptions['y'].push(getDateValues(dt));
	}
	dt.setDate(1);
	dt.setMonth(dt.getMonth() - 1);
}

That takes care of the four choice lists and the code to come up with the values for the four choice lists. We’ll want something selected when the page first loads, though, so we’ll need some additional code to come up with the initial values for each of the four selections. For the group, my thought was to start out with the group that had the most tasks, and if the user was a member of any groups, group of which the user was a member with the most tasks would be event better. Here’s what I came up with the handle that:

function getDefaultGroup() {
	var defaultGroup = '';

	var max = 0;
	var group = new GlideAggregate('task');
	if (data.usersGroups.size() > 0) {
		var usersGroups = '';
		var separator = '';
		for (var i=0; i<data.usersGroups.size(); i++) {
			usersGroups = separator + "'" + data.usersGroups.get(i) + "'";
			separator = ',';
		}
		group.addQuery('assignment_group', 'IN', usersGroups);
	}
	group.addAggregate('COUNT');
	group.groupBy('assignment_group');
	group.ordderBy('assignment_group');
	group.query();
	while (group.next()) {
		if (group.getDisplayValue('assignment_group')) {
			if (group.getAggregate('COUNT') > max) {
				max = group.getAggregate('COUNT') * 1;
				defaultGroup = group.getValue('sys_class_name');
			}
		}
	}
	if (!defaultGroup) {
		defaultGroup = data.config.maxGroup;
	}

	return defaultGroup;
}

Since the type choices are dependent on the group selected, that code is already built into the type selection list creation process (above). For the initial frequency, I just arbitrarily decided to start out with daily, and for the initial period ending date, I decided that the current period would be the best place to start as well. That code turned out to be pretty basic.

data.frequency = 'd';
data.ending = data.config.endingOptions[data.frequency][0].value;

With the initial choices made, we now need to work out the process of gathering up the data for the chart based on the choice list selections. That’s a bit of a task as well, so let’s make that our focus the next time out.

Fun with Highcharts, Part II

“The only thing that endures over time is the Law of the Farm. You must prepare the ground, plant the seed, cultivate, and water it if you expect to reap the harvest.”
Stephen Covey

Now that I have my functioning generic chart widget, it’s time to start working on my chart object generator. My intent, as usual, is to start out small and then build up a library of various chart types over time. Conceptually, I want to create a utility function to which I can pass some chart data and a chart type, and have it return a fully configured chart object that I can pass to my generic chart widget which will then render the chart. In use, it would look something like this:

var chartObject = generator.getChartObject(chartData, chartType);

Theoretically, you could make the chartType parameter optional if you set up a default, and then you would only have to pass that in when you wanted something other than the the established standard. For my first chart, though, I think I will do something slightly more sophisticated, mainly because this is the kind of thing that I like to utilize when I’m looking at the distribution of work within and across teams. I call it a Workload Chart, and it’s just your typical three-value tracking of work coming in, work getting done, and work still left in the needs-to-get-done pile. It doesn’t matter what the work is, or who is doing it, or over what period you are tracking things — the concept is pretty universal and the chart is the chart, regardless of the underlying data. Here’s one that allows you to select the Team, the Type of work, the Frequency, and the Period from a series of drop-down selections:

Typical workload chart

The chart data is the stuff that will change: title, subtitle, data values, and the labels across the bottom. The chart object includes all of the chart data, plus all of those structural elements that are constant and independent of the data. The utility function will take in the chart data, add it to a predefined model, and then return the entire object back to the caller. Here is the basic structure of the Script Include, which I called the GenericChartUtil:

var GenericChartUtil = Class.create();
GenericChartUtil.prototype = {
	initialize: function() {
	},

	getChartObject: function(chartData, chartType) {
		var chartObject = {};

		if (chartType == 'workload') {
			chartObject = this._getWorkloadChart(chartData);
		} else if (chartType == 'bar') {
			chartObject = this._getBarChart(chartData);
		} else {
			chartObject = this._getPieChart(chartData);
		}

		return chartObject;
	},

	_getPieChart: function(chartData) {
		return {
//			type-specific chart object w/data
		};
	},

	_getBarChart: function(chartData) {
		return {
//			type-specific chart object w/data
		};
	},

	_getWorkloadChart: function(chartData) {
		return {
//			type-specific chart object w/data
		};
	},

	type: 'GenericChartUtil'
};

For the workload chart, the dynamic data includes the title, the subtitle, an array of values for the received bars , an array of values for the completed bars , an array of values for the remaining work trend line (backlog), and an array of values for the time period labels. For the chart displayed in the previous image, the chartData object might look something like this:

{
   "title":"Incidents assigned to Hardware",
   "subtitle":"Quarterly through 9/30/2019",
   "labels":[
      "12/31/2017",
      "3/31/2018",
      "6/30/2018",
      "9/30/2018",
      "12/31/2018",
      "3/31/2019",
      "6/30/2019",
      "9/30/2019"
   ],
   "received":[0, 0, 7, 2, 0, 0, 0, 1],
   "completed":[0, 0, 6, 1, 0, 0, 0, 0],
   "backlog":[0, 0, 1, 2, 2, 2, 2, 3]
}

How you would compile that information is a subject for another time; today we just want to focus on how our chart object generator would take that information in and use it to create a complete chart object to be passed to Highcharts. It’s actually quite simple to do, and can be accomplished with a simple return statement that sends back a Javascript object that is a combination of hard-coded values and data pulled from the incoming chart data object. Here is the completed function for the workload chart:

	_getWorkloadChart: function(chartData) {
		return {
			chart: {
				zoomType: "xy"
			},
			title: {
				text: chartData.title
			},
			subtitle: {
				text: chartData.subtitle
			},
			xAxis: [
				{
					categories: chartData.labels,
					crosshair: true
				}
			],
			yAxis: [
				{
					labels: {
						format: "{value}",
						style: {}
					},
					title: {
						text: "Recieved/Completed",
						style: {}
					}
				},{
					labels: {
						format: "{value}",
						style: {}
					},
					title: {
						text: "Backlog",
						style: {}
					},
					opposite: true
				}
			],
			tooltip: {
				shared: true
			},
			legend: {
				layout: "vertical",
				align: "left",
				x: 120,
				verticalAlign: "top",
				y: 100,
				floating: true,
				backgroundColor: "rgba(255,255,255,0.25)"
			},
			series: [
				{
					name: "Received",
					type: "column",
					data: chartData.received,
					tooltip: {
						valueSuffix: " received"
					}
				},{
					name: "Completed",
					type: "column",
					data: chartData.completed,
					tooltip: {
						valueSuffix: " completed"
					}
				},{
					name: "Backlog",
					type: "spline",
					yAxis: 1,
					data: chartData.backlog,
					tooltip: {
						valueSuffix: " remaining"
					}
				}
			]
		};
	},

For every property in the returned chart object, there is either a hard-coded constant value that is an element of the chart template/design, or there is data pulled from the incoming chart data object. This idea can be repeated for any type or design of chart desired, and you can make independent decisions on a chart by chart basis as to what is a standard, fixed value and what has to be provided in the chart data object. As you get more sophisticated, you can even set up default values for certain things that can be overridden by values present in the chart data. If there is a value in the chart data, you can use that; otherwise, you can fall back to the predetermined default value. But before we start making more chart types, we’ll need to figure out how to obtain all of the values for the chart data object to support our workload chart example. That sounds like a bit of work though, so I think we should just call that a great topic for a future installment

Fun with Highcharts

“Most of the important things in the world have been accomplished by people who have kept on trying when there seemed to be no hope at all.”
Dale Carnegie

Long before I had ever heard of ServiceNow, I stumbled across a cool little product called Highcharts. In their own words, “Highcharts is a charting library written in pure JavaScript, offering an easy way of adding interactive charts to your web site or web application.” ServiceNow comes bundled with Highcharts included, which I thought was pretty nifty when I figured that out, because you can do quite a bit with Highcharts with very minimal effort. In fact, that’s one of the things that I really like about both products: both allow you to accomplish quite a bit with just a very small investment of time and energy.

To make things even easier, it always helps to have a few ready-made parts at your disposal to speed things along. Highcharts uses simple Javascript objects to configure their charts, and the contents of those objects can be broken down into two rather distinct categories: 1) elements that control the type, look, and feel of the chart and 2) elements that contain the data to be presented in the chart. Once you design a particular chart with the style, colors, and appearance that you find acceptable, you can save that off and reuse it again and again with different data or data from different periods. For the ServiceNow Service Portal, my thought was to create a generic chart widget that could take any chart configuration object as a passed parameter, and then also create a generic chart configuration object generator that would take a chart type and chart data as parameters and return a chart configuration object that could be then passed to the generic chart widget.

To start things off, I decided to build a simple generic chart widget and pass it a hard-coded chart configuration object, just to prove that everything about the widget was functional. There are a lot of sample Highcharts chart objects floating around the Internet, but for my test I just went out to their Your First Chart tutorial page and grabbed the configuration object for that simple example.

Rendered Highchart as displayed on their tutorial page

To grab just the chart object, I copied the highlighted code from the page.

Chart configuration object (highlighted)

Unfortunately, this is not a valid JSON string, which I would need if I was going to pass this around as a parameter, but I could paste it into simple Javascript routine as the value of a variable, and print out a JSON.stringify of that variable and then I ended up with a usable JSON string.

{
   "chart":{
      "type":"bar"
   },
   "title":{
      "text":"Fruit Consumption"
   },
   "xAxis":{
      "categories":[
         "Apples",
         "Bananas",
         "Oranges"
      ]
   },
   "yAxis":{
      "title":{
         "text":"Fruit eaten"
      }
   },
   "series":[
      {
         "name":"Jane",
         "data":[1, 0, 4]
      },{
         "name":"John",
         "data":[5, 7, 3]
      }
   ]
}

With my new hard-coded chart specification object in hand, I set out to create my generic widget. Every Highchart needs a place to live, so I started out with the HTML portion and created a simple chart DIV inside of a basic panel.

<div class="panel panel-default">
  <div class="panel-body form-horizontal">
    <div class="col-sm-12">
      <div id="chart-container" style="height: 400px;"></div>
    </div>
  </div>
</div>

On the server side, I wanted to be able to accept a chart configuration object from either input or options, so I ended up with this:

(function() {
	if (input && input.chartObject) {
		data.chartObject = input.chartObject;
	} else {
		if (options && options.chart_object) {
			try {
				data.chartObject = JSON.parse(options.chart_object);
			} catch (e) {
				gs.addErrorMessage("Unable to parse chart specification: " + e);
			}
		}
	}
})();

On the client side, I also wanted to be able to accept a chart configuration object from a broadcast message from another widget, and I also needed to add the code to actually render the chart, so that turned out to be this:

function ($scope) {
	var c = this;
	if (c.data.chartObject) {
		var myChart = new Highcharts.chart('chart-container', c.data.chartObject);
	}
	if (c.options.listen_for) {
		$scope.$on(c.options.listen_for, function (event, config) {
			if (config.chartObject) {
				var myChart = new Highcharts.chart('chart-container', config.chartObject);
			}
		});
	}
}

One more thing to do was to set up the Option schema for my two widget options, which is just another JSON string.

[{"hint":"An optional JSON object containing the Chart specifications and data",
"name":"chart_object",
"default_value":"",
"section":"Behavior",
"label":"Chart Object",
"type":"string"},
{"hint":"The name of an Event that will provide a new Chart specification object",
"name":"listen_for",
"default_value":"",
"section":"Behavior",
"label":"Listen for",
"type":"string"}]

Finally, I had to go down to the bottom of the widget page where the related records tabs are found and Edit the dependencies to add Highcharts as a dependency. This will bring in the Highcharts code and make all of the magic happen.

That pretty much completes the widget, so now I just needed to create a page on which to put the widget so that I could test it out. Once I created the page, I was able to pull up the widget options using the pencil icon in the page designer and enter in my JSON object for the chart configuration.

Adding the JSON chart object as a widget option

At that point, all that was left was for me to click on the Try it button and look at my beautiful new chart. Unfortunately, all I got to see was a nice big blank chunk of whitespace where my chart should have appeared. Digging into the console error messages, I came across the following:

Highcharts not defined

Well, either my dependency did not load, or my widget couldn’t see the code. I tried $rootScope.Highcharts and $window.Highcharts and $scope.Highcharts, but nothing worked. Then I thought that maybe it was a timing issue and so I put in a setTimeout to wait a few seconds for everything to load, but that didn’t work either. So then I added a script tag to the HTML thinking that maybe the dependency wasn’t pulling in the script after all. Nothing.

At that point, I decided to give up just trying different things on my own and see if I could hunt down some other widget that might already be working with Highcharts and see how those were set up. From the Dependencies list, I used the hamburger menu next to the Dependency column label to select Configure > Table, and then on the Table page, I scrolled down to where I could click on Show list to bring up the list of dependencies. Then I filtered the list for Highcharts to find that there were quite a few existing widgets that pulled in Highcharts as a dependency.

The secret, it appeared, was some additional code in the Link section of the widget, a section that I have always ignored, mainly because I have never understood its purpose or how it worked.

function(scope, element, attrs, controllers) {
	scope.$watch('chartOptions', function() {
		if (scope.chartOptions) {
			$('#chart-container').highcharts(scope.chartOptions);
			scope.chart = $('#chart-container').highcharts();
		}
	});
}

Now this is all AngularJS mystical nonsense sprinkled with Highcharts fairy dust as far as I was concerned, because I have no idea what the Link section of a widget is for or what all of this code actually does. However, it is attached to widgets that actually work, so I got busy cutting and pasting. The other thing that I had to do based on the examples that I was examining was to modify my client-side code a little, which now looked like this:

function ($scope) {
	var c = this;
	if (c.data.chartObject) {
		$scope.chartOptions = c.data.chartObject;
	}
	if (c.options.listen_for) {
		$scope.$on(c.options.listen_for, function (event, config) {
			if (config.chartObject) {
				$scope.chartOptions = config.chartObject;
			}
		});
	}
}

Yes, we are now deep into the realm of Cargo Cult Programming, but I’ve never been too proud to copy other people’s working code, even if I didn’t understand it. Now let’s hit that Try it button one more time …

That’s better!

Well, look at that! I’m not sure how all of that works, but it does work, so I’m good. Now that I have my functioning generic chart widget, I can start working on my chart object generator and then see if I can’t wire the two of them together somehow …