HighByte Blog
Read company updates and our technology viewpoints here.
|
Read company updates and our technology viewpoints here.
|
Elevate your industrial interoperability: A primer on data Pipelines in the Intelligence Hub3/8/2024
Time to read: 15 minutes In HighByte Intelligence Hub, the Pipelines feature was created to make modeled data consumable by a diverse range of applications and services. With the last few releases of the Intelligence Hub, Pipelines has undergone big changes to further that goal and more. From adding new functionality to refining the UX, Pipelines has swiftly evolved beyond its initial focus on “post-processing” payloads for advanced use cases. It has become a core data engineering capability to solve industrial interoperability problems within the Intelligence Hub. Time to read: 9 minutes I consistently hear that many manufacturers are drowning in data and struggling to make it useful. Why is that? A modern industrial facility can easily produce more than a terabyte of data each day. With a wave of new technologies for artificial intelligence and machine learning coupled with real-time dashboards and prescriptive insights, industrial companies should be seeing huge gains in productivity. Unplanned asset and production line maintenance should be a thing of the past. But we know that is not the case. Access to data does not make it useful. Industrial data is raw and must be made fit for purpose to extract its true value. Furthermore, the tools used to make the data fit for purpose must operate at the scale of an industrial enterprise. For many industrial companies, this is a daunting task requiring alignment of people, process, and technology across a global footprint and supply chain. At HighByte, we’re putting our best foot forward to solve this data architecture and contextualization problem from a technology perspective. But what about people and process? To pull it all together, we recently published a new guide, “Think Big, Start Small, Scale Fast: The Data Engineering Workbook.” The guide provides 10 steps to achieving a scalable data architecture based on the best practices we’ve learned from our customers over the last several years. Time to read: 7 minutes For the past several months, 55 beta testers in 13 countries have been kicking the tires on HighByte Intelligence Hub version 3.0 and generously providing their feedback. Today, I’m excited to announce this major release is now available. Version 3.0 is a step change for the Intelligence Hub and for the Industrial DataOps market. It raises the bar for what a DataOps solution can be at Enterprise scale. It introduces a powerful new Pipelines builder to curate complex data pipelines. It makes the often-vague concept of the Unified Namespace (UNS) tangible and achievable with an embedded MQTT broker—reducing additional software, cost, and administration overhead for our customers. I sat down with HighByte Chief Product Officer John Harrington to talk about some of these advancements available in Version 3.0, including Pipelines. His thoughts are below. I also provide insights from our partner Goodtech, a deep dive on the embedded broker, a review of new project management capabilities, and more. Time to read: 6 minutes Have you ever watched a press conference when a room full of reporters bark questions at the same time? Typically, the media event host will call on a particular reporter to repeat the question and then move on to the next person in the room. Without some ground rules, an actual conversation couldn’t take place. No one could hear the questions being asked, and few would get any answers. Unfortunately, this same scenario often occurs with industrial data. With so much operational technology (OT) data generated on any given day, manufacturers risk losing critical information in the sea of “data noise” coming from their systems or having to expend vast resources to clean that data in the cloud. Time to read: 6 minutes When it comes to data collection, who are you really serving? That objective often gets lost amid the OT/IT alignment discussions. Anyone who has embarked on a digital transformation project is likely familiar with the data silos that exist between their OT and IT departments. But we don’t spend enough time talking about how to make that data usable for the line of business. Our line of business colleagues (and their systems of record) are the ultimate customer. The use of IoT-enabled devices is increasing the availability of operational data. IDC has projected there will be 41.6 billion IoT devices in the field generating 79.4 zettabytes of data by 2025. These devices include machines, sensors, and cameras as well as industrial tools. To truly make that data usable, we need to merge this data with information from other systems and provide context for line of business users. In an industrial environment, these users include quality, maintenance, engineering, R&D, regulatory, and product management. Time to read: 8 minutes How much time do you spend cleaning data? If your factory is like most connected operations, you probably have tons of raw data streaming from connected devices to existing enterprise systems, bespoke databases, and a cloud data lake. This architecture often leads to inconsistent or even unusable data for several reasons. We know the Cloud is a key tool for digital transformation. It provides the scalability and storage capacity you need to collect and interpret vast amounts of data coming from the operations level. However, by nature, cloud platforms are IT-focused tools. They structure data differently than operational systems, which means IT must spend a lot of time cleaning the data before it can be used. And if the data moves directly to different enterprise systems, multiple teams across the organization will clean the data independently, leading to different versions of the truth.
Time to read: 10 minutes
Most manufacturing companies realize the benefits of leveraging industrial data to improve production and save costs, but they remain challenged as to how to scale-up their pilots and small-scale tests to the plant-wide, multi-plant, or enterprise level. There are many reasons for this including the time and cost of integration projects, the fear of exposing operational systems to cyber-threats, and a lack of skilled human resources.
At the root of all of these problems is the difficulty of integrating data streams across applications in a multi-system and multi-vendor environment, which has required some degree of custom coding and scripting. Standardizing data models, flows, and networks is hard work. Unlike an office environment with its handful of systems and databases, a typical factory can have hundreds of data sources distributed across machine controls, PLCs, sensors, servers, databases, SCADA systems, and historians—just to name a few. Industrial DataOps provides a new approach to data integration and management. It provides a software environment for data documentation, governance, and security from the most granular level of a machine in a factory, up to the line, plant, or enterprise level. Industrial DataOps offers a separate data abstraction layer, or hub, to securely collect data in standard data models for distribution across on-premises and cloud-based applications. These four use cases illustrate how Industrial DataOps can integrate your role-based operational systems with your business IT systems as well as those of outside vendors such as machine builders and service providers.
Time to read: 7 minutes
An executive for an industrial products company once told me even though his factories are full of similar equipment, he still struggled to access meaningful data from the machines. Each one of the plastic injection molding machines had a different way of presenting the data. That meant the company needed to customize coding for every piece of equipment to obtain meaningful insights.
It’s a common scenario in many industrial environments, where plants may have hundreds of PLCs and machine controllers on disparate machines generating operational data that is unintelligible to the data scientists who must make sense of it. This is where Industrial DataOps comes in. It provides a way to standardize data using common models, or object-oriented approaches, to integrate and manage information coming from multiple sources. Here’s a closer look at the top six signs it’s time to consider an Industrial DataOps architecture for your company.
Time to read: 6 minutes
A modern industrial facility can easily produce a terabyte of data each day. With the proliferation of sensors and the recent wave of real-time dash-boarding, artificial intelligence, and machine learning technologies, we should be seeing huge productivity gains. Unplanned maintenance of assets and production lines should be obsolete.
But this is not the case. Access to data does not mean it is useful. Industrial data is very raw and must be made “fit for purpose” in order to extract its true value. Furthermore, the tools used to make the data fit for purpose must operate at the scale of an industrial facility. With these realities in mind, I’ve written a practical, seven-step guide for manufacturers and other industrial companies to make their data fit for purpose. |
|