Know the Super User

Super User can manage functions which are not permissible to Admin or Normal users. Only Super User can manage Workspaces, default Data Pipelines, and Licenses.

Manage Workspaces

StreamAnalytix provides multitenancy support through Workspaces.

Super User can add a new workspace, alter the existing workspaces, assign users to a workspace, and assign multiple workspaces to a user.

Workspace Cluster Isolation

StreamAnalytix’s Cluster Isolation feature enables Superuser to allocate specific machines to Workspaces for better utilization of hardware resources.

System Pipelines

System Pipelines (or Superuser Pipelines) are the default system Pipelines that are created automatically when you deploy StreamAnalytix. These Pipelines cannot be modified. Superuser can only start or stop these Pipelines.

System Pipelines should be used (start/stop) to enable operations such as alert generation, error logs collection, monitoring, etc.

AlertPipeline

 

Alert Pipeline enables the Alert Processor to generate alerts on the UI and send alert emails. It stores alert- messages in the database and aggregate alerts for a given time window.

 

DataLineagePipeline

Datalineage Pipeline enables searching of lineage information. This Pipeline reads data from various kafka topics configured in Data Lineage sections of Pipelines and persists into Hbase by creating proper schema.

ErrorSearchPipeline

ErrorSearchPipeline sends a pipeline’s error logs and messages to an indexer so that error logs are available on the Application Errors tab in a Pipeline Summary View.

GraphiteWriterPipeline

GraphiteWriterPipeline takes the input (metrics) from the MonitorMetricPipeline and sends it to graphite. 

MonitorMetricPipeline

MonitorMetricPipeline pipeline sends the metrics to a server either Ambari or Graphite as per the configuration properties. It executes the aggregation rules based on the metrics type for a fixed time interval.

SparkDataLineagePipeline

SparkDatalineage Pipeline enables searching of Lineage information. This Pipeline reads data from various Kafka topics configured in Data Lineage sections of Pipelines.

SparkMonitorMetricPipeline

SparkMonitorMetricPipeline monitors the metrics of a Spark Pipeline. The data which consists of a set of message types is generated by the metrics scheduler which is read by SparkMonitorMetricPipeline pipeline from a RabbitMQ queue SaxSparkMonMetricQueue. The metrics data is processed in a window duration of 60 sec and the processed data is sent back to another RabbitMQ queue SaxSparkMetricsWriterQueue. The Pipeline generates different values for the incoming data by applying aggregation functions on the metrics data. The processed data helps in understanding the overall matrices of the Pipeline corresponding to the infrastructure, disk, network, jmx, and monitoring alert etc.

SparkMetricsWriterPipeline

SparkMetricsWriterPipeline writes the metrics generated by a Spark pipeline. The SparkMetricsWriterPipeline Pipeline reads data from the RabbitMQ queue SaxSparkMetricsWriterQueue and processes the data, which consists of a set of message types. The data is analysed based on monitoring criteria applied in the Manage Alerts section of the Monitoring widget.

Connections

Connections allow StreamAnalytix to connect to services like RabbitMQ, Elasticsearch etc.

Only Superuser can create connections. Admin and other users can use these connections.