Replication service

The replication service allows V4 nodes to replicate events and interact with each other.

Note

There can be only a single replication service instance per pub/sub server deployed on each EVA ICS node.

The nodes must be connected via either PSRT or MQTT pub/sub.

  • PSRT is an in-house pub/sub protocol, designed especially for industrial needs, which can efficiently replicate lots of events per seconds and deal with large payloads via slow channels.

  • MQTT is popular standard IoT protocol, widely used and provided by lots of free and commercial service applications, as well as hosted versions from various cloud providers.

v4 replication

To communicate, both local and remote node must share the same API key (a key service is required, e.g. Local user/key authentication service. In trusted environment it is possible to use the same “default” key for all nodes. In untrusted ones it is recommended to have a dedicated key for each node pair.

Warning

All key fields MUST contain API key ID, not the key value.

The API key specifies ACL(s) for the remote node to handle RPC-via-pub/sub calls.

To enable all remote bus calls, nodes must also share the same API key with admin privileges, which must be set in “admin_key_id” configuration field on the primary (manager) node. The security policy is the same as for regular keys.

When the “admin_key_id” is set, the remote node becomes “managed”.

Note

Remote bus calls are required for IaC and deployment if a deployment configuration contains sections for remote nodes.

If discovery is enabled, newly discovered nodes are automatically connected with the “default_key”. Discovery feature is recommended for trusted and semi-trusted environments.

One node can have multiple replication services deployed. However all of the must replicate with the own set of remote nodes. Having same remote nodes replicated by different services leads to abnormal system behavior.

See also: Pub/Sub RPC and data replication.

Pub/sub security

In production systems a pub/sub server can be securely shared between customers in case of the following conditions:

  • no node discovery (as node states can be forged)

  • bulk telemetry, bulk encryption

All RPC calls in EVA ICS are AES-256-encrypted by default.

Setup

Use the template EVA_DIR/share/svc-tpl/svc-tpl-replication.yml:

# EVA ICS v4-v4 replication service
command: svc/eva-repl
workers: 2
bus:
  path: var/bus.ipc
config:
  pubsub:
    # mqtt or psrt
    proto: psrt
    # path to CA certificate file. Enables SSL if set
    ca_certs: null
    # single or multiple hosts
    host:
      - 127.0.0.1:2873
    # if more than a single host is specified, shuffle the list before connecting
    cluster_hosts_randomize: false
    # user name / password auth
    username: null
    password: null
    ping_interval: 10
    # pub/sub queue size
    queue_size: 1024
    # pub/sub QoS (not required for PSRT)
    qos: 1
  # the local key service, required both to make and process API calls via PubSub
  key_svc: eva.aaa.localauth
  # set null to turn off announcements
  announce_interval: 10
  # respond to API calls
  api_enabled: true
  # enable discovery of secondary nodes
  discovery_enabled: true
  # key (id) to use for auto-discovered nodes
  default_key_id: default
  # subscription logic
  #
  # each: subscribe to individual topics for each item, plus bulks if defined
  # all: accept states for all items, move filtering from the Pub/Sub server
  #       to the local core
  # bulk_only: subscribe to bulk topics only
  subscribe: each
  #bulk:
    #send:
      #topic: all
      #compress: true # set to enable bulk compression
      #encryption_key: null # set (key id) to enable bulk encryption
      #buf_ttl_sec: 0.5
    #receive:
      #topics:
        #- all
  # subscribed OIDS
  oids:
    - "#"
user: nobody

Create the service using eva-shell:

eva svc create eva.repl.1 /opt/eva4/share/svc-tpl/svc-tpl-replication.yml

or using the bus CLI client:

cd /opt/eva4
cat DEPLOY.yml | ./bin/yml2mp | \
    ./sbin/bus ./var/bus.ipc rpc call eva.core svc.deploy -

(see eva.core::svc.deploy for more info)

EAPI methods

See EAPI commons for the common information about the bus, types, errors and RPC calls.

bus::<SVC_ID>::<METHOD>

Description

Calls any remote method (requires management access)

Parameters

Always required (provide empty Map if the target method has no ones)

Returns

RPC result as-is

node.append

Description

Appends remote node with the default configuration

Parameters

required

Returns

nothing

Parameters

Name

Type

Description

Required

i

String

Node name

yes

node.deploy

Description

Deploys remote nodes and their configuration

Parameters

required

Returns

nothing

Parameters

Name

Type

Description

Required

nodes

Vec<struct>

Node configurations (same as got in node.export)

no

node.export

Description

Exports node deployment configurations

Parameters

required

Returns

Node deploy config

Parameters

Name

Type

Description

Required

i

String

Node ID (accepts wildcards)

yes

Return payload example:

{
    "nodes": [
        {
            "admin_key_id": "admin",
            "compress": true,
            "enabled": true,
            "key_id": "default",
            "name": "rtest1",
            "ping_interval": 1.0,
            "reload_interval": 60.0,
            "timeout": 30.0
        }
    ]
}

node.get

Description

Gets state of a single remote node

Parameters

required

Returns

Node state (struct)

Parameters

Name

Type

Description

Required

i

String

Node name

yes

Return payload example:

{
    "build": 2022050902,
    "compress": true,
    "enabled": true,
    "link_uptime": 183.26962651,
    "managed": true,
    "name": "rtest1",
    "online": true,
    "ping_interval": 1.0,
    "reload_interval": 60.0,
    "static": true,
    "timeout": 30.0,
    "version": "4.0.0"
}

node.get_config

Description

Gets configuration of a single node

Parameters

required

Returns

Node config (struct)

Parameters

Name

Type

Description

Required

i

String

Node ID

yes

Return payload example:

{
    "admin_key_id": "admin",
    "compress": true,
    "enabled": true,
    "key_id": "default",
    "name": "rtest1",
    "ping_interval": 1.0,
    "reload_interval": 60.0,
    "timeout": 30.0
}

node.list

Description

Lists remote nodes

Parameters

none

Returns

List (struct)

Return payload example:

[
    {
        "build": 2022050902,
        "compress": true,
        "enabled": true,
        "link_uptime": 85.004101572,
        "managed": true,
        "name": "rtest1",
        "online": true,
        "ping_interval": 1.0,
        "reload_interval": 60.0,
        "static": true,
        "timeout": 30.0,
        "version": "4.0.0"
    },
    {
        "build": 2022050902,
        "compress": true,
        "enabled": true,
        "link_uptime": 122.0092,
        "managed": true,
        "name": "rtest2",
        "online": true,
        "ping_interval": 1.0,
        "reload_interval": 60.0,
        "static": true,
        "timeout": 30.0,
        "version": "4.0.0"
    }
]

node.mtest

Description

Tests management call to a remote node

Parameters

required

Returns

nothing

Parameters

Name

Type

Description

Required

i

String

Node ID

yes

node.reload

Description

Forces reload of a remote node

Parameters

required

Returns

nothing

Parameters

Name

Type

Description

Required

i

String

Node ID

yes

node.remove

Description

Removes a single node

Parameters

required

Returns

nothing

Parameters

Name

Type

Description

Required

i

String

Node ID

yes

node.test

Description

Tests connection to a remote node

Parameters

required

Returns

nothing

Parameters

Name

Type

Description

Required

i

String

Node ID

yes

node.undeploy

Description

Deploys remote nodes and their configuration

Parameters

required

Returns

nothing

Parameters

Name

Type

Description

Required

nodes

Vec<struct/String>

Node configurations or IDs

no