• Publicatie 17 April 2022



Industrial maintenance strategies increasingly rely on artificial intelligence to predict asset conditions and prescribe maintenance actions. The related maintenance software and human maintenance actors can form a hybrid-augmented intelligence system where each side benefits from and enhances the other side’s intelligence. This system requires optimized human-machine interfaces to help users express their knowledge and retrieve information from difficult-to-use software. Therefore, this article proposes a novel approach for maintenance experts and operators to interact with a predictive maintenance system through a digital intelligent assistant. This assistant is artificial intelligence (AI) that could help its users interact with the system via natural language and collect their feedback about the success of maintenance interventions.

Implementing hybrid-augmented intelligence in a predictive maintenance system faces several technical, social, economic, organizational, and legal challenges. The benefits, limitations, and risks of hybrid-augmented intelligence must be clear to all employees to advocate its use. AI-focused change management and employee training could be techniques to address these challenges. The success of the proposed approach also relies on the continuous improvement of natural language understanding. Such a process will need conversation-driven development where actual interactions with the assistant provide accurate training data for language and dialog models. Future research has to be interdisciplinary and may cover the integration of explainable AI, suitable AI laws, operationalized trustworthy AI, efficient design for human-computer interaction, and natural language processing adapted to predictive maintenance.

Authors:  StefanWellsandt, KonstantinKlein, KarlHribernik, MarcoLewandowsk,i AlexandrosBousdekis, GregorisMentzas and Klaus-DieterThoben Source: Article

Improving production’s reliability and efficiency while simultaneously taking sustainability and safety issues into account are pressing issues for all manufacturing industries. Maintenance has a significant impact on these issues. Its improvement can lead to a 10–20% cost reduction in related labor and materials, which together make up 15% of the total costs of a typical manufacturing company. This potential means that these organizations need to consider maintenance an essential operation function.

Organizations typically use reactive, preventive, or condition-based maintenance strategies. Reactive maintenance means that technicians carry out maintenance after a fault occurs, while preventive maintenance focuses on planners that rigidly schedule maintenance to prevent faults from occurring. Condition-based maintenance uses data-driven diagnoses or predictions about an asset’s condition. The increasing product complexity and customer demands for quality and reliability drive the trend towards predictive and prescriptive maintenance. Both approaches rely on sensors, programmable logic controllers, big data, artificial intelligence (AI), and cloud computing. Predictive maintenance aims at increasing the availability of the production equipment while avoiding unplanned downtimes with the use of condition monitoring. It takes advantage of vast real-time and historical data from the shop floor to detect early equipment behavior anomalies, predict the equipment’s future health, and formulate proactive maintenance plans to eliminate or mitigate the predicted failures’ impact.

The activities above form a complex business process that organizations manage with software forming a predictive maintenance system (PMS). The increasing share of AI-based tools for automated decision-making and decision support is central to future PMSs. While software takes over formalized, repetitive, and time-consuming tasks, humans focus on information-intensive activities, complex problem-solving, and supervision. If humans receive support from and contribute their knowledge to the AI-based tools in a PMS, the PMS becomes a hybrid-intelligence system. Such systems promote the idea that AI and humans possess non-interchangeable roles and mutually benefit from each other. Consequently, a related PMS is a socio-technical hybrid-intelligence system. The interaction between technology, information, and social subsystems enables such a system’s functional utility. Besides, the emergence of human-centered AI dictates an optimized human-machine collaboration capable of facilitating the combination of computational capabilities provided by AI and knowledge provided by humans.

1. Hybrid-augmented intelligence systems

The theory of hybrid-augmented intelligence systems suggests that human input to AI systems can prevent mistakes and shortcomings and help them learn what decision is best to take for a given problem. The rationale behind this is the combination of complementary, heterogeneous, and intelligent human and artificial agents to create a socio-technological system capable of overcoming the current limitations of AI. Human and artificial agents of such systems can then co-evolve by learning and optimizing decisions through intuitive interfaces. Besides, the AI enhances (augments) the human’s expertise, knowledge, and experience. Humans and AIs have complementary strengths to co-evolve by learning from each other and thus improving over time. Implementing the interaction between maintenance actors and the PMS is a critical challenge since demand pressure, short lifecycles, cognitively and physically demanding work, and high complexity of decisions are typical for maintenance tasks. 

Click to enlarge: Asset & Maintenanec Management-A time perspective by Jan Stoker 

DIAs are a tool to address these challenges and eventually remove them as obstacles to adopting PMS. Research articles already outline models describing how maintenance actors should be involved. The research gap covers interaction modes and the appropriate supporting technology. Interaction modes concern how the system responds to the user’s behavior, while supporting technologies are the means to realize the system’s modes.

2. What is a digital asistant

An assistant supports or entirely takes over time-consuming, stressful, or otherwise not desirable activities for the client. A digital assistant (DA) is software that achieves the same goals for one or more users. Its users interact with a DA via one or more user interfaces that can be, for instance, text-based, graphical, or intelligent. Intelligent interfaces include conversational interfaces enabling natural language interactions. An assistant’s various interaction modalities make it a “multiexperience” application. Such an application facilitates a consistent user experience.

When a DA supports its users, it serves as a combination of a conversational agent and a technology-based agent. A conversational agent is a dialog system embedded in personal technologies and devices. It interacts with one or more users via natural language. Differentiate voice-user-interface (VUI) agents, text-based agents (Chatbots), and embodied conversational agents (e.g., represented through a robot). A technology-based agent is a system that observes, interprets, decides, and learns to act upon its environment. Such an agent interacts with humans and technical systems to achieve mutual goals. Its autonomy and capabilities determine how it supports humans.



Intermezzo: The Asset Management BowTie

The recently published standard EN 17485 introduces methods and procedures about maintenance within physical asset management for all the levels and functions of the organizations’ management, including corporate planning management, plant management, technical management, production management, financial management, asset management, maintenance management, and quality management. Further and maybe even greater benefits are now being found through improved credibility in the eyes of customers, regulators and other stakeholders.

Click to enlarge

Physical asset management also results in much greater engagement and motivation of the workforce, and in more sustainable, continual improvement business processes. Physical asset management builds up the required link between maintenance management and the organizational strategic plan and gives direction to maintenance activities.The standards EN 16646 and EN 17485 build the bridge between ISO 5500x (Asset management system standards) and the EN maintenance standards. ISO 55001 states that organizations should determine e.g. the organizational context, requirements for the assets, decision criteria, strategic asset management plan and asset management plan (including maintenance).

However, it does not describe how to do it. Respectively, maintenance standards often introduce e.g. the concept of the required function or the concept of maintenance strategy, but do not explain how they have been determined. EN 17485 introduces a methodological framework which advises organizations to implement the requirements presented in ISO 55001. By doing this it creates the bridge between the several maintenance standards and ISO 5500x in order to give an applicable starting point to the more detailed documents for the specific sub-functions of maintenance (See AM-BowTie).

This insights, the Asset Management BowTie, can be added toward the Asset Management paradigm with the (Digital) Line of Sit. To add the BowTie principle, the body of thoughts of the EN-17485 can be incorporated in the Asset Management paradigm with the mentioned standards. Result is a deepened figure 1 of the ISO5500 completed with the Line(s) of Sights, the levels of information, Business, Risk Assessment and Asset & Maintenance going to decision making  in balancing Costs, Risks and Performance.

The Asset Management BowTie show how standards can be uses as a reference framework. 

Click to enlare Asset Management BowTie: See Body of thoughts AM-BowTie Click Here

3. How does a digital assistant work

The technological research behind voice-enabled DIA’s reaches back to the first interactive voice response (IVR) systems in the 1970s – some principles of dialog design are still valid since then. A voice-enabled DIA for the industry has five essential components. First, the speech-to-text (STT) component transcribes a user’s audio utterances. Second, the Natural language understanding (NLU) extracts semantic elements from this transcribed text, typically through data analytics processes built with machine learning. Essential semantic elements are the user’s intents and entities. Intents are what the user wants the assistant to do.

Entities are information items the assistant needs to know to respond meaningfully. Typical entities include persons, locations, items, dates, and activities. Third, the dialog management (DM) component analyses the extracted intents and entities and decides how the assistant should respond. Such decisions consider information stored or generated in external information systems – they are the fourth component. 

External systems can also trigger the DM proactively to notify users. Access to external systems is critical for the assistant’s capabilities and perceived intelligence. Finally, the text-to-speech (TTS) component turns the generated text into a computer-generated voice. Users can typically customize this voice by changing its speed and pitch.

Click to enlare

4. Human intervention in Pdm Proces

In Industry 4.0, human operators experience increased complexity in their working environment. Therefore, they need advanced technologies with high usability requirements. The interaction with a PMS often occurs via computer screens that are stationary or mobile. Visualized diagnosis data, prediction curves, and maintenance recommendations are often complex and difficult to quickly understand, especially for staff not experienced in digital technologies, particularly when applying to emerging and dynamic situations. In this context, complexity concerns:

  1. how much information the user receives (e.g., a table with many rows and columns, or a high-resolution graph)
  2. how heterogeneous the information is (e.g., dashboard with performance indicators, graphs, and notifications)
  3. how connected the information is (e.g., dashboard with dependent indicators)


The figure shows a five-phase, unified model of predictive maintenance synthesizing concepts of predictive maintenance, proactive computing, Gartner’s levels of industrial analytics maturity, and ISO 13,374/MIMOSA OSA-CBM. The following sections contain a brief description of each phase and an attempt to identify human intervention potentials.

  1. The sense phase
  2. The detect phase
  3. The predict phase
  4. The decide phase
  5. The act phase

Click to enlare


5. A Predictive Maintenance System with a digital intelligent assistant

The figure shows a simplified view of a generic, data-driven PMS. The human maintenance actors interact turn-wise with the DIA (conversation) through mobile or stationary devices. One or more production systems contain the maintained assets, comprising control systems and sensors, which provide real-time streaming data to the PMS.

Sensors and control systems send data streams to local hubs, implementing IoT gateways to access and manage the data. The hub pushes the streaming data to the PMS, e.g., via a streaming message broker, while the storage element stores all data processed or generated by the PMS. A time-series database contains the streaming sensor data and the intermediate results of diagnosis and prognosis.

The stream processing component includes functionalities responsible for analyzing stream data typically generated by sensors installed in the assets. It consists of a component for condition detection & diagnosis and one for condition and RUL prediction, providing feature value forecasts. 

Decision-making receives streams of predictions about future failure modes or RUL estimations and generates optimal maintenance prescriptions based on that and other maintenance planning factors. Enterprise systems such as Computerised Maintenance Management Systems (CMMS), Electronic Maintenance Modules (EMM), and MES contribute to managing and executing production and maintenance plans. The decision-making component sends them instructions to act according to the decisions. All components exchange data via a message bus (e.g., Kafka). The DIA connects with the overall system in the same way.

Click to enlare



Click for the full article


6.  React, Discuss & Article


Discuss or give your opinionFollow on FacebookFollow on Twitter


Related Articles 




Tags: , , , ,