For quite some time, people have been asking us how the Intelligence Hub relates to a Unified Namespace (UNS). Is it a specific part of a UNS architecture, a platform by which one might build a UNS, or a UNS architecture itself? Over time, our answers have developed alongside the capabilities of the Intelligence Hub. From the beginning, the Intelligence Hub could connect to third-party MQTT brokers as well as model the data going in and out of them. And recently, we added an embedded MQTT broker to the Intelligence Hub to address brokering and provide the ability for MQTT clients to connect to the Intelligence Hub directly. These provided the functionality needed for much of what might facilitate or be considered a UNS, but it was missing a critical part: a way to visualize the contents.
A UNS typically requires:
- Some sort of brokering technology such as MQTT.
- Well-defined topics and payloads that represent an organization and the way it consumes data. In practice, this requires a data engineering solution designed to pull data from other industrial systems, contextualize this data, and publish to the broker.
- A client to view the UNS’ topic namespace and its payloads.
HighByte Intelligence Hub version 3.2 also includes Pipelines enhancements and a new SQLite connector, delivering more powerful data engineering capabilities than ever.
UNS Client: Delivering visibility into your Unified Namespace
The UNS Client provides visualization of the UNS, allowing users to view the contents of any MQTT broker. Intelligence Hub users can visually browse, interrogate, and test their UNS without an external test client. When coupled with our embedded MQTT broker, the UNS Client enables the visualization of MQTT-based Industrial DataOps workloads in a single, unified engineering environment.
Figure 1: A view of the new UNS Client available in HighByte Intelligence Hub version 3.2, enabling users to visually browse, interrogate, and test their UNS without an external client.
The integrated UNS Client eliminates the need to configure additional client connections or security certification and credentials by leveraging the very same MQTT or Sparkplug connections used throughout the Intelligence Hub. Using the same underlying technology to construct and validate, the Intelligence Hub fosters a holistic approach to engineering a UNS, and best of all, it is simple to use. Just select a connection, and instantly visualize the namespace.
Finally, the UNS Client can automatically detect and visualize message payloads including JSON, Sparkplug, text, and raw binary. Sparkplug users will benefit from the UNS Client’s ability to decode Protobuf and make payloads human readable. In addition to topic and message inspection, the UNS Client can publish messages to topics.
Finally, the UNS Client can automatically detect and visualize message payloads including JSON, Sparkplug, text, and raw binary. Sparkplug users will benefit from the UNS Client’s ability to decode Protobuf and make payloads human readable. In addition to topic and message inspection, the UNS Client can publish messages to topics.
Data Engineering: Pipelines upgrades and a new SQLite connector
Though UNS users enjoy the benefits of standardized data, industrial interoperability requires much more than mere protocol conversion. Data must be collected, transformed, consolidated, contextualized, standardized, and curated to the needs of the consuming application or service. To make this possible, these steps must be defined and managed as pipelines of data that maintain state and allow for incoming data sets, decision points, branches, and writes, as well as acknowledgments. With the improvements in Intelligence Hub version 3.2, data pipelines can now sequentially read, filter, branch, maintain state, and conditionally write to outputs.
Pipelines in the Intelligence Hub have been enhanced to provide highly configurable, complex, and sequenced data processing. Pipelines can now:
This sequencing, along with new compression capabilities, can tackle “ETL” use cases, such as backfilling historical data from time-series databases to the cloud. Additionally, Pipelines can now maintain state from one run to the next, which is ideal for capturing long-running, conditional events used for tracking machine status and production.
This feature-packed release of Pipelines is topped off with an overhaul of the UI, complete with a new aesthetic, zoom functions, and panel hiding.
Pipelines in the Intelligence Hub have been enhanced to provide highly configurable, complex, and sequenced data processing. Pipelines can now:
- Seamlessly interface with complex systems hosting transactional and historical data.
- Sequence dependent transactions with MES and ERP systems.
- Handle acknowledgments and make systems aware of the progress or completion of an integration.
This sequencing, along with new compression capabilities, can tackle “ETL” use cases, such as backfilling historical data from time-series databases to the cloud. Additionally, Pipelines can now maintain state from one run to the next, which is ideal for capturing long-running, conditional events used for tracking machine status and production.
This feature-packed release of Pipelines is topped off with an overhaul of the UI, complete with a new aesthetic, zoom functions, and panel hiding.
Figure 2: A view of the improved Pipelines builder available in HighByte Intelligence Hub version 3.2. Enhanced with a new UI as well as several new stages and functions, Pipelines are now a must-have data engineering tool.
Also new for version 3.2, the Intelligence Hub now offers a SQLite connector to reach an embedded SQLite database within the Intelligence Hub’s runtime. Many Intelligence Hub users leverage Structured Query Language (SQL) as a tool to transform relational data. This new connector provides users with a simple, ready-to-use relational data store to employ in their Industrial DataOps workloads. The connection can read and write records stored in-memory or persisted in database files.
The SQLite database is an ideal tool for complex data transformations that require logging, constructing relationships between datasets, or cross-referencing values. The in-memory storage is optimal for performant, on-the-fly transformations, while the file-based storage is optimal for logging records and managing state locally within the Intelligence Hub. In many cases, this negates the need for an external database.
The SQLite database is an ideal tool for complex data transformations that require logging, constructing relationships between datasets, or cross-referencing values. The in-memory storage is optimal for performant, on-the-fly transformations, while the file-based storage is optimal for logging records and managing state locally within the Intelligence Hub. In many cases, this negates the need for an external database.
Wrap Up
Over the last five years, the UNS has become a prominent architectural pattern for integrating industrial organizations. But, implementing and managing a UNS has been hampered by its need for multiple software solutions from multiple vendors for even a single site.
Intelligence Hub version 3.2 delivers the full functionality of a Unified Namespace in an easily implemented package and a robust array of data engineering tools that empower users to further integrate their business and unlock potential. With this release, we hope to make industrial data more accessible and valuable than ever. To learn more, please check out these additional resources:
Intelligence Hub version 3.2 delivers the full functionality of a Unified Namespace in an easily implemented package and a robust array of data engineering tools that empower users to further integrate their business and unlock potential. With this release, we hope to make industrial data more accessible and valuable than ever. To learn more, please check out these additional resources:
- Read the release notes for details on all new features and fixes.
- Request a free trial or log in to your existing account to test and deploy the software in your unique environment.
