In order to present data in Visualizations, edgeCore first needs to know where the data is coming from. Data Connections can be made to a variety of different data sources. Feeds off of these data connections are then used to pull in raw data.

Stock Connection Types

EdgeCore ships with a generalized set of adapters that allow you to connect a variety of different sources. Detailed descriptions about the configuration required to connect and pull in data from these sources can be found in the individual Connection Types listed below:

	Adapter	Supported Data Sources
	Database	H2, jSONAR, MySQL, Postgres, SQLServer, DB2, Oracle, Sybase
	Server Filesystem	CSV, Custom (JavaScript), JSON, XML XPath, XML XSLT
	Shell Exec	CSV, Custom (JavaScript), JSON, XML XPath, XML XSLT
	Web Data	CSV, Custom (JavaScript), JSON, XML XPath, XML XSLT
	JavaScript	JavaScript
	Push	CSV, Custom (JavaScript), JSON, XML XPath, XML XSLT
	System Metrics	System Metrics (Page Views, Jobs, Proxy Requests, Server Load, Entitlement, RPA Metrics)

Some database types (shown in italics above) are not available on a default installation until the required JDBC drivers have been installed. Refer to Installing Database (JDBC) Drivers for more information.

Caching and Polling

When the edgeCore server retrieves data, it always pushes it directly to the client without delay. However, there are still two settings that can control the freshness and latency of data in the client. Most data adapters provide settings for “Poll Interval” and “Enable Server Subscription”.

Poll Interval

This tells edgeCore how frequently you want it to go back to the original data source to request the latest data (in seconds). A low value for Poll Interval will give fresher data, but will produce more load on the server.

Poll interval is calculated based on the time the previous request was completed.  So, if the poll interval is 300 seconds, and the time it takes to run is 210 seconds, then roughly every 510 seconds the data will be updated.  This assumes there is still an active subscription that references that specific data job. If a subscription already exists for a job when a new request comes in for that same job (an existing or new browser session creates a subscription that ultimately needs that data job), then that data is immediately returned from the cache. Once any data job has retrieved data, it will never again have to wait to fulfill any new request for the same data unless either all subscriptions for it are terminated (all clients navigate away or end their sessions) or there is a remote error and the data update fails. This shows the power of the server subscription – any client desiring that data will never have to wait.

Enable Server Subscription

This option tells edgeCore to pre-load data immediately into a node in the Pipeline, even if the client (user) has not yet opened a Visualization which requested that data. This can make the user’s experience more responsive, but will produce more load on the server. This setting is not recommended during development, but you may want to turn it on when the system goes to production so the system will be more robust to the user.

Some consideration is required when deciding whether to enable server subscription for a particular node, especially in the case where the data may be changing dynamically by the real-time setting of Node Variables and Secured Variables. When Node Variables and Secured Variables are in use, the client plugs those values in and requests the data in real-time. The server subscription will behave exactly like a client is requesting the data, except it uses the default Node Variables and Secured Variable values. If these default values match what they might be in the client, then the server subscription will add value by caching the data. If these default values are different from what they will be in the client, then the server subscription would be a bad idea since it will only waste extra CPU cycles at login, caching data that will not be used.

On the data side, data can be shared based on unique keys. If user A and user B both happen to be accessing the same dataset with the same keys (same Node Variable and Secured Variable values), then it will resolve to the same thing. For efficiency, edgeData will attempt to cache and share data as much as possible, but this can be negated either intentionally or by accident depending on how Node Variables and Secured Variables are used. For example, if you have a node in the pipeline with a Secured Variable that resolves to the user’s username, then it is going to be a unique dataset for every user in the system. For edgeWeb proxied content, there is no sharing between users.

You can think of a server subscription as just a client whose browser session begins immediately at startup and never expires and is requesting that data producer with the default variable values. For any data producer, for any given set of variables (Node Variables and Secured Variables), if there is any subscription active for the data, the specific producer job (plus any upstream data jobs) will be run and data cached as long as there is any subscription that needs that data. A server subscription will guarantee that there will be a job for the producer with the default values for the variables (if there are any), which in turn will force all upstream jobs (if any) to be active and cached, with the updates for any sources always driven by the poll interval of whichever Feed(s) is(are) involved. Enabling a server subscription will ensure that one specific job for that producer (the one with the default variable values) will always exist, so that specific job will be permanent, but other jobs against the same producer (if variables are used with any non-default value) will only be maintained as long as any client session references those jobs.

edgeCore Pipeline Connection

To learn how to set up a remote pipeline, refer to this page.