Overview

The JavaScript Transform allows you to manipulate data programmatically using JavaScript. The JavaScript Transform can return a new, possibly restructured, dataset.

Create a Javascript Transform

• Select the gear icon for a node.
• Select the “New” option from the radial menu.
• Select the “Transform” option.

The “Choose a Transform” dialog will appear, showing the available transforms in the system. Select the “JavaScript” option.

JavaScript Transform Wizard

The JavaScript transform wizard has five steps.

Step 1:  Configure

The following properties may be configured:

Properties	Description
Transform Name	Name of JavaScript Transform.
Description	This is where the administrator can enter notes for the transform.
Enable Server Subscription	If set, the server will subscribe to the transform, just as a client feed would. This means that the data and any resources that would otherwise be allocated “on-demand” for the first user to view a Visualization that leverages the data produced by this transform are allocated when the server is started and maintained as long as this transform is configured.

Step 2: Script References

This is an opportunity to explicitly declare any references the script makes to resources on the filesystem or otherwise, which would be required to be included in any partial backups made where this JavaScript transform is involved.

Properties	Description
Referenced Datasets	Choose one or more datasets that are needed for the JavaScript Code to access on the next wizard step. You must return to this step if you decide later to access another dataset in your JavaScript Code, as the editor will not allow you to access anything not chosen here. The unique alias of a given dataset can be used as a name index in your JavaScript Code.
Secured Variables	Secured Variables resolve to values based on the User that is logged into the system. If you have configured Secured Variables in the system, you can include them here to filter the data that will be returned based on what User is accessing this Transform.
Java Libraries	If the Transform’s script contains any references to Java library code which depends on any libraries installed in the [INSTALL_HOME]/lib directory, then they should be selected here.
JavaScript Libraries	If the Transform’s script contains any references to user-supplied JavaScript libraries/modules/files, they should be selected here. The supported locations within [INSTALL_HOME] are /data, /scripts, /static-web and /login. Any files with the .js extension within these directory trees will be made available for selection here.

Step 3:  Script

JavaScript Structure and API

The basic flow of a script will be to essentially process the incoming data, produce new data, and return it. In general, there will be one set of tabular data that is passed into the script, but if the JavaScript Transform is configured to have multiple upstream Producers, then there will be one data set for each upstream producer. Regardless, the JavaScript implementation may only return one result.
The script must provide an implementation of two functions: getAttributes(sourceAttrs, nodeVars, secVars, sourceRecords) and getRecords(sourceRecords, nodeVars, secVars, attributes).

Arguments

sourceAttrs: An array containing arrays of objects. Each array represents an upstream producer’s attribute list, and each object represents an individual attribute. Producer attribute lists can be referenced via either numerical indexing or named indexing (e.g. sourceAttrs[0] is the attribute list of the first producer; alternatively use sourceAttrs[aliasOfFirstProducer]). For each attribute, the following properties exist:

• [Read/Write Properties]:
  • name: required, should consist of alphanumerics and underscores only with no leading, trailing, or adjacent underscores.
  • type: valid types are ‘string’, ‘int’, ‘long’, ‘number’, ‘boolean’, and ‘date’. If not set, ‘string’ is used as the default type.
  • isId: true or false. If not set, false is the default value.
  • units: valid units are ‘millis’ or ‘seconds’. This property must be set if the type is ‘date’ and the data values consist of Unix timestamps in milliseconds or seconds, respectively.
  • format: valid formats consist of pattern strings such as those documented in the SimpleDateFormat Javadoc. This property should be set if the type is ‘date’ and the data values consist of parseable date strings. If not set, the server will attempt to identify the format, but the output may not be as predictable as if the user were to define the format ahead of time.
• [Read-Only Properties]:
  • producerName: the name of the producer (feed or transform) associated with this attribute.
  • sourceName: the original attribute name, not sanitized for database storage.
  • ordinalPosition: the attribute’s position in the attribute list.

sourceRecords: An array containing arrays of objects. Each array represents an upstream producer’s record list, and each object represents an individual record. Producer record lists can be referenced via either numerical indexing or named indexing (e.g. sourceRecords[0] is the record list of the first producer; alternatively use sourceRecords[aliasOfFirstProducer]). For any given record, the record’s value for a particular attribute can be accessed via record.attrName, e.g. record.foo = "bar".

Record values that are numeric (int/long/number), strings, or boolean can be compared using the standard JavaScript comparison operators. For date values, however, it is recommended that one use the java.util.Date API as the values are instances of the Date class. Alternatively, you can call dateValue.getTime() to get the timestamp value (in milliseconds) and use the JavaScript comparison operators.

nodeVars: An object whose properties correspond to any variables defined for this transform. To access a variable, use nodeVars.varName.

secVars: An object whose properties correspond to any secured variables defined for the current user. To access a variable, use secVars.varName.

attributes: An array of attribute objects, equivalent to the array returned from the getAttributes() function.

getAttributes(sourceAttrs, nodeVars, secVars, sourceRecords)

This function must return one of the following:

• jsAttributesSuccess(attributes, "Optional success message for debugging")
• jsAttributesFailure("Optional failure message")

In the success case, attributes is an array of objects, where each object represents an attribute definition. Attribute definition objects can be ported directly from the source attributes to the outgoing array, or new attribute definition objects can be created following the properties guidelines listed above.

getRecords(sourceRecords, nodeVars, secVars, attributes)

This function must return one of the following:

• jsRecordsSuccess(records, "Optional success message for debugging")
• jsRecordsFailure("Optional failure message")

In the success case, records is an array of objects, where each object represents a record. Records can be ported directly from the source records, with the option to edit existing record values or add new values to the record. Of course, new record objects can also be made from scratch and added to the outgoing array. Record property names should match attribute names defined in getAttributes(sourceAttrs, nodeVars, secVars, sourceRecords) (e.g. if a string attribute “foo” is one of the defined attributes, one can assign record.foo = "bar").

Nashorn allows the use of both java.util.Date objects as well as ES5 JavaScript Date objects. While similar, each type has its own API. When the edgeSuite server stores date values from the JavaScript Transform, it will do so using java.util.Date, with the conversion from JavaScript Date occurring behind the scenes. In other words, you can assign record.dateValue = new java.util.Date() or record.dateValue = new Date() and it should work just the same. As mentioned above, date values in the source records will be of the type java.util.Date. If using comparison methods from the java.util.Date API, be aware that JavaScript Date objects will not be considered as having the same class type and thus direct comparisons are likely to fail. You could, of course, create new instances of either type using the result of getTime() and then make your comparisons.

Step 4:  Attributes DB Options

Refer to Indexing for details.

Step 5:  Upstream Variables

Details about this step can found here: Satisfy Upstream Node Variables Wizard Step.

Step 6:  Data Preview

Use this step to preview the resulting data. You can use the Variables sidebar to tweak parameter values prior to saving the transform.

Additional Services

Some situations require fetching the results of other data producers within the pipeline. This can be achieved via the dataProducerService object.

dataProducerService API for edgeSuite 3.7.x and newer 

fetchRecords(producerName, nodeVars, secVars)

This function takes the name of the producer, as well as the nodeVars and secVars objects. Generally speaking, the nodeVars and secVars objects can be passed from the parent function (see above documentation for details). However, nodeVars can also be defined as an object with key:value pairs, which are then internally converted as required by the server.

The returned value is an array of objects (records) whose properties correspond to the attribute names of the resulting data set. The value of a record for a given attribute can then be accessed via record.attrName, e.g. record.foo = "bar".

dataProducerService API for edgeSuite 3.6.x and older

fetch(producerName, nodeVars, secVars)

This function behaves similarly to the one above, the main difference being that the returned value is a Java object of type DataResultsDO. Use of this function is no longer recommended, however versions as old as 3.4.x can leverage a simple workaround to achieve functionality similar to that of 3.7.x. Assuming the returned DataResultsDO object is defined, and its data is both defined and of type List<TabularRecordDO>, users may convert the data to an array of JavaScript objects by calling convertSourceRecordsForJs([returnedObject.getData()])[0]. This conversion call will generate an array of objects similar to the result of fetchRecords(producerName, nodeVars, secVars) in 3.7.x. 

In addition to getData(), the returned DataResultsDO object has a method getResultStatusCode(), which returns an enum, the value of which is one of CURRENT, ERROR_STALE, ERROR_MISSING, REQUIRE_CREDENTIALS, or BAD_CREDENTIALS.

Log information

Basic logger capability is provided to all runtime scripts via:

yellowstone/server/src/main/java/edge/server/script/ScriptEngineService.java

To get information into a log file, use the following syntax:

if ( logger.isFatalEnabled() ) {
    logger.fatal("Failed to calculate status level ...");
}
if ( logger.isErrorEnabled() ) {
    logger.error("Failed to calculate status level ...");
}
if ( logger.isWarnEnabled() ) {
    logger.warn("Warning: calculation of status level ...");
}
if ( logger.isInfoEnabled() ) {
    logger.info("calculation of status level: {}, completed in: {} seconds", level, instant);
}
if ( logger.isDebugEnabled() ) {
    logger.debug("calculation of status level: {}, completed in: {} seconds", level, instant);
}
if ( logger.isTraceEnabled() ) {
    logger.trace("calculation of status level: {}, completed in: {} seconds", level, instant);
}

More information on the Log4j API can be found at the link below.