+

CN112547352A - Automatic spraying monitoring teleoperation method and system based on digital twinning - Google Patents

Automatic spraying monitoring teleoperation method and system based on digital twinning Download PDF

Info

Publication number
CN112547352A
CN112547352A CN202011228212.7A CN202011228212A CN112547352A CN 112547352 A CN112547352 A CN 112547352A CN 202011228212 A CN202011228212 A CN 202011228212A CN 112547352 A CN112547352 A CN 112547352A
Authority
CN
China
Prior art keywords
fault
processing
real
knowledge base
remote
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011228212.7A
Other languages
Chinese (zh)
Other versions
CN112547352B (en
Inventor
周凯
张景淘
郭展赫
杨帅
龙晓军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong Agricultural University
Original Assignee
Shandong Agricultural University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong Agricultural University filed Critical Shandong Agricultural University
Priority to CN202011228212.7A priority Critical patent/CN112547352B/en
Publication of CN112547352A publication Critical patent/CN112547352A/en
Application granted granted Critical
Publication of CN112547352B publication Critical patent/CN112547352B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B05—SPRAYING OR ATOMISING IN GENERAL; APPLYING FLUENT MATERIALS TO SURFACES, IN GENERAL
    • B05B—SPRAYING APPARATUS; ATOMISING APPARATUS; NOZZLES
    • B05B12/00—Arrangements for controlling delivery; Arrangements for controlling the spray area
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/02—Knowledge representation; Symbolic representation
    • G06N5/022—Knowledge engineering; Knowledge acquisition
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y10/00—Economic sectors
    • G16Y10/25—Manufacturing
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y40/00—IoT characterised by the purpose of the information processing
    • G16Y40/10—Detection; Monitoring
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y40/00—IoT characterised by the purpose of the information processing
    • G16Y40/20—Analytics; Diagnosis
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y40/00—IoT characterised by the purpose of the information processing
    • G16Y40/30—Control

Landscapes

  • Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Business, Economics & Management (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Manufacturing & Machinery (AREA)
  • Data Mining & Analysis (AREA)
  • Accounting & Taxation (AREA)
  • Development Economics (AREA)
  • Economics (AREA)
  • General Business, Economics & Management (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Testing And Monitoring For Control Systems (AREA)

Abstract

本申请公开了一种基于数字孪生的自动喷涂监测遥操作方法及系统，所述方法包括：通过传感网络对自动喷漆现场进行实时监测并且建立现场实时增强模型；如果监测到自动喷漆过程发生异常，则根据知识库对故障进行诊断，确定故障的类型和发生的原因，获得解决方法；从监测模式切换为故障遥操作模式，通过遥操作控制器对故障进行处理。通过传感网络获得现场状态的实时数字信号，利用实时增强模型可以更加高效和准确地做出判断，以至于对机器进行更好的控制。操作者利用摇操作，启动应急处理系统，排除设备故障。通过知识库可以诊断出可能发生的故障类型，并根据实例中的故障处理方法，提出当前故障处理方法和建议实现故障准确快速的处理。

Figure 202011228212

The present application discloses a digital twin-based automatic spraying monitoring remote operation method and system. The method includes: monitoring the automatic spraying site in real time through a sensor network and establishing an on-site real-time enhanced model; if it is detected that the automatic spraying process is abnormal , then diagnose the fault according to the knowledge base, determine the type and cause of the fault, and obtain the solution; switch from the monitoring mode to the fault remote operation mode, and handle the fault through the remote operation controller. The real-time digital signal of the field status is obtained through the sensor network, and the real-time enhanced model can be used to make judgments more efficiently and accurately, so that the machine can be better controlled. The operator uses the shaking operation to start the emergency treatment system to eliminate equipment failures. The possible fault types can be diagnosed through the knowledge base, and according to the fault processing methods in the instance, the current fault processing methods and suggestions are put forward to achieve accurate and fast fault processing.

Figure 202011228212

Description

Automatic spraying monitoring teleoperation method and system based on digital twinning
Technical Field
The application relates to the technical field of automatic spraying, in particular to an automatic spraying monitoring teleoperation method and system based on digital twinning.
Background
A machine part is an essential element constituting a machine, and is a non-detachable single piece constituting both the machine and the machine. In the prior art, a part production side generally needs to paint parts in order to protect the parts and avoid surface oxidation or damage of the parts.
Mostly manual operation sprays paint to gyration type part among the conventional art, arranges the part in a single file and puts, and the spraying of spray gun straight reciprocating motion spraying. And after finishing spraying, waiting for drying the paint on the surface of the part, and boxing the part. Because manual operation exists inefficiency, and the toxic substance in the paint threatens staff's health, consequently adopts the automatic machine that sprays paint to the part spraying among the prior art.
The fault treatment in the automatic paint spraying process is an important link for ensuring the product quality in the actual production process, but the fault diagnosis result is inaccurate along with the occurrence of various complex data, and the normal spraying operation is influenced.
Disclosure of Invention
In order to solve the technical problems, the following technical scheme is provided:
in a first aspect, an embodiment of the present application provides an automatic spraying monitoring teleoperation method based on digital twinning, where the method includes: monitoring an automatic paint spraying site in real time through a sensing network and establishing a site real-time enhancement model; if the automatic paint spraying process is monitored to be abnormal, diagnosing the fault according to the knowledge base, determining the type and the occurrence reason of the fault, and obtaining a solution; after the solution is obtained, the monitoring mode is switched to a fault teleoperation mode, and the fault is processed through the teleoperation controller.
By adopting the implementation mode, the signals with different formats can be preprocessed by the multi-source heterogeneous data through different data acquisition interfaces of the sensor network, and the real-time digital signals of the field state are obtained. The digital twin and augmented reality technology is utilized to establish a field real-time augmented model of the spraying operation, so that an operator can make judgment more efficiently and accurately so as to control a machine better. The safety risk caused by human factors can be reduced by adopting a teleoperation technology, and an operator starts an emergency treatment system by using the remote operation to remove equipment faults. The possible fault types can be diagnosed through the knowledge base, and according to the fault processing method in the example, the current fault processing method and the suggestion are provided to realize accurate and rapid processing of the fault.
With reference to the first aspect, in a first possible implementation manner of the first aspect, the monitoring an automatic paint spraying site in real time through a sensing network and establishing a site real-time enhanced model includes: determining the composition and layout of an automatic spraying teleoperation assembly control device; and the establishment of the automatic spraying field augmented reality modeling is realized through multi-source data processing, multi-source information registration and virtual-real information fusion in sequence.
With reference to the first possible implementation manner of the first aspect, in a second possible implementation manner of the first aspect, the establishing of the automatic spraying field augmented reality modeling by sequentially performing the multi-source data processing, the multi-source information registration, and the virtual-real information fusion includes: under the support of a workshop characteristic data model and an information expression template, rapidly extracting, identifying and optimizing characteristic data and key information of multi-source data acquired on site, and further realizing uniform expression of heterogeneous information; registering the multi-source sensing information based on a least square rule and a maximum likelihood registration algorithm; and dynamically calibrating the field real-time information and the 3D virtual model in real time, and performing interaction of multi-source information on the model to obtain an automatic spraying field virtual-real fusion enhanced model.
With reference to the first aspect, in a third possible implementation manner of the first aspect, operation data, fault data, historical data, process data, resource data, safety processing data, and the like used in teleoperation control are reasonably stored in a knowledge base, where the knowledge base includes a fault instance knowledge base and a fault diagnosis rule knowledge base, the fault instance knowledge base stores fault phenomena, fault sources, and fault causes of faults, and the fault diagnosis rule knowledge base stores rules for diagnosing, reasoning, determining, and processing faults.
With reference to the third possible implementation manner of the first aspect, in a fourth possible implementation manner of the first aspect, if it is monitored that an automatic paint spraying process is abnormal, a fault is diagnosed according to a knowledge base, a type and a cause of the fault are determined, and a solution is obtained, where the method includes: firstly, loading a fault phenomenon, determining a search strategy, and searching an instance library by using the search strategy to obtain a similar fault instance set; obtaining an example meeting a similarity threshold value for reuse through example matching calculation; and (3) performing analog analysis on similar examples, diagnosing the type of the fault which possibly occurs, and proposing a current fault handling method proposal according to the fault handling method in the examples to provide a basis for teleoperation control.
With reference to the fourth possible implementation manner of the first aspect, in a fifth possible implementation manner of the first aspect, after the obtaining the solution, the monitoring mode is switched to a fault teleoperation mode, and the processing of the fault by the teleoperation controller includes: switching modes, namely switching from a normal automatic operation mode to a teleoperation safety processing mode; determining the fault type and proposing a safety processing mode suggestion based on a fault mode and processing method knowledge base, and guiding teleoperation personnel to carry out remote safety processing; and the teleoperation personnel operates the teleoperation controller to process the safety problem, the teleoperation controller generates a teleoperation signal, the teleoperation signal is transmitted to a PLC (programmable logic controller) of the paint spraying equipment through conversion, the field equipment is controlled to execute a teleoperation processing action, and after the safety processing is finished, the teleoperation processing process of the safety problem is ended.
In a second aspect, an embodiment of the present application provides a digital twin-based automatic spray monitoring teleoperation system, including: the monitoring module is used for monitoring an automatic paint spraying site in real time through a sensing network and establishing a site real-time enhancement model; the fault diagnosis and judgment module is used for diagnosing the fault according to the knowledge base if the abnormity of the automatic paint spraying process is monitored, determining the type and the occurrence reason of the fault and obtaining a solution; and the processing module is used for switching the monitoring mode into a fault teleoperation mode after the solution is obtained, and processing the fault through the teleoperation controller.
With reference to the second aspect, in a first possible implementation manner of the second aspect, the monitoring module includes: the determining unit is used for determining the composition and the layout of the automatic spraying teleoperation assembly control device; and the model establishing unit is used for realizing the establishment of the automatic spraying field augmented reality modeling by sequentially carrying out multi-source data processing, multi-source information registration and virtual-real information fusion.
With reference to the second aspect, in a second possible implementation manner of the second aspect, the fault diagnosis determining module includes: the first acquisition unit is used for loading the fault phenomenon, determining a search strategy and searching the instance library by using the search strategy to obtain a similar fault instance set; the second acquisition unit is used for acquiring the instances meeting the similarity threshold value for reuse through instance matching calculation; and the diagnosis and judgment unit is used for diagnosing the type of the fault which possibly occurs by carrying out analog analysis on the similar examples, and providing a current fault processing method suggestion according to the fault processing method in the examples so as to provide a basis for teleoperation control.
With reference to the second possible implementation manner of the second aspect, in a third possible implementation manner of the second aspect, the processing module includes: the mode switching unit is used for switching the mode from a normal automatic operation mode to a remote operation safety processing mode; the remote control unit is used for determining the fault type and proposing a safety processing mode suggestion based on the fault mode and the processing method knowledge base and guiding teleoperation personnel to carry out remote safety processing; and the processing unit is used for the teleoperation personnel to operate the teleoperation controller to process the safety problem, the teleoperation controller generates a teleoperation signal, the teleoperation signal is transmitted to the PLC of the paint spraying equipment through conversion, the field equipment is controlled to execute a teleoperation processing behavior, and after the safety processing is finished, the teleoperation processing process of the safety problem is ended.
Drawings
Fig. 1 is a schematic flow chart of an automatic spraying monitoring teleoperation method based on digital twinning according to an embodiment of the present disclosure;
FIG. 2 is a schematic diagram of the hardware components and layout of an automatic coating teleoperation assembly control system according to an embodiment of the present disclosure;
fig. 3 is a schematic diagram of spray painting site perception information processing and augmented reality modeling provided in an embodiment of the present application;
FIG. 4 is a schematic diagram of a database storage structure according to an embodiment of the present application;
FIG. 5 is a schematic diagram of a fault instance knowledge base and a fault diagnosis rule knowledge base provided by an embodiment of the present application;
FIG. 6 is a schematic diagram of an ontology structure of an automatic painting failure provided by an embodiment of the present application;
FIG. 7 is a schematic diagram of a knowledge-base-based automatic paint failure diagnosis process provided by an embodiment of the present application;
fig. 8 is a schematic view of an automatic spraying monitoring teleoperation system based on digital twinning according to an embodiment of the present disclosure.
Detailed Description
The present invention will be described with reference to the accompanying drawings and embodiments.
Fig. 1 is a schematic flow chart of an automatic spraying monitoring teleoperation method based on digital twinning provided in an embodiment of the present application, and referring to fig. 1, the method includes:
s101, monitoring an automatic paint spraying site in real time through a sensing network and establishing a site real-time enhancement model.
Firstly, considering the composition and layout of the automatic spraying teleoperation assembly control device, as shown in fig. 2, the field data sensed by the automatic spraying field sensor enters the data acquisition PLC through different data interfaces for rapid preprocessing and structured coding. When a safety problem occurs, a user transmits a control signal to the equipment control PLC through the optical fiber and the switch by remote processing, and then the control signal is transmitted to each on-site execution equipment, and safety processing is performed according to a control instruction.
In order to establish a real-time on-site augmented reality model, as shown in fig. 3, the automatic spraying on-site augmented reality modeling is divided into three steps including multi-source data processing, multi-source information registration and virtual-real information fusion.
Multi-source data processing: under the support of a workshop characteristic data model and an information expression template, rapidly extracting, identifying and optimizing characteristic data and key information of multi-source data acquired on site, and further realizing uniform expression of heterogeneous information;
multi-source information registration: registering the multi-source sensing information based on a least square rule and a maximum likelihood registration algorithm;
fusing virtual and actual information: and carrying out dynamic real-time calibration on the field real-time information and the 3D virtual model, and carrying out interaction on multi-source information on the model to finally obtain the automatic spraying field virtual-real fusion enhanced model.
S102, if the situation that the automatic paint spraying process is abnormal is monitored, diagnosing the fault according to the knowledge base, determining the type and the reason of the fault, and obtaining a solution.
In order to reasonably store various data in the knowledge base, as shown in fig. 4, a storage structure of the database is designed, and operation data, fault data, history data, process data, resource data, safety processing data, and the like used in the remote operation control are reasonably stored in the database. In the figure, rectangular boxes represent entities, connecting lines between the boxes represent relations between the entities, and attribute names in the boxes represent attributes of the entities.
In order to realize reasonable fault diagnosis, the knowledge base is divided into a fault example knowledge base and a fault diagnosis rule knowledge base as shown in fig. 5 according to the requirement of equipment fault maintenance support. The fault instance knowledge base stores fault phenomena, fault sources and fault reasons of the faults. The fault diagnosis rule knowledge base stores rules for diagnosing, reasoning, judging and processing faults.
And analyzing the hierarchical relationship according to the core concept and the 5 top classes of the painting fault extraction to obtain the knowledge ontology structure of the automatic painting fault shown in the figure 6. The method comprises 3 pieces of basic information of fault phenomena, fault sources and fault reasons: the failure phenomenon refers to an abnormal phenomenon or state shown by the paint spraying equipment and the environment when the failure occurs; the failure source refers to a failed device; the cause of the failure guides the specific factors that cause the failure to occur.
The specific steps of fault diagnosis are shown in fig. 7: firstly, loading the fault phenomenon, determining a search strategy, and searching an instance library by using the search strategy to obtain a similar fault instance set. And obtaining the instances meeting the similarity threshold value for reuse through instance matching calculation. And (3) performing analog analysis on similar examples, diagnosing the type of the fault which possibly occurs, and proposing a current fault handling method proposal according to the fault handling method in the examples to provide a basis for teleoperation control.
S103, after the solution is obtained, the monitoring mode is switched to a fault teleoperation mode, and the fault is processed through the teleoperation controller.
The working process of the teleoperation control module can be known, the real-time monitoring of the paint spraying process is carried out based on the virtual and real enhancement model in the paint spraying process, and once a safety fault occurs, the teleoperation control processing of the safety problem can be started immediately. When a safety fault occurs:
firstly, switching modes, namely switching from a normal automatic operation mode to a teleoperation safety processing mode;
secondly, determining the fault type and proposing a safety processing mode suggestion based on a fault mode and processing method knowledge base, and guiding teleoperation personnel to carry out remote safety processing;
and finally, the teleoperation personnel operates the teleoperation controller to process the safety problem, the teleoperation controller generates a teleoperation signal, the teleoperation signal is transmitted to a PLC of the paint spraying equipment through conversion, the field equipment is controlled to execute a teleoperation processing action, and after the safety processing is finished, the teleoperation processing process of the safety problem is ended.
According to the embodiment, the automatic spraying monitoring teleoperation method based on the digital twin is provided, signals with different formats can be preprocessed through different data acquisition interfaces of a sensor network, and real-time digital signals of a field state are obtained. The digital twin and augmented reality technology is utilized to establish a field real-time augmented model of the spraying operation, so that an operator can make judgment more efficiently and accurately so as to control a machine better. The safety risk caused by human factors can be reduced by adopting a teleoperation technology, and an operator starts an emergency treatment system by using the remote operation to remove equipment faults. The possible fault types can be diagnosed through the knowledge base, and according to the fault processing method in the example, the current fault processing method and the suggestion are provided to realize accurate and rapid processing of the fault.
Corresponding to the automatic spraying monitoring teleoperation method based on the digital twin provided by the above embodiment, the present application also provides an embodiment of an automatic spraying monitoring teleoperation system based on the digital twin, and referring to fig. 8, the automatic spraying monitoring teleoperation system 20 based on the digital twin includes: a monitoring module 201, a fault diagnosis judging module 202 and a processing module 203.
The monitoring module 201 is used for monitoring an automatic paint spraying site in real time through a sensing network and establishing a site real-time enhancement model. If the fault diagnosis and judgment module 202 monitors that the automatic paint spraying process is abnormal, the fault is diagnosed according to the knowledge base, the type and the occurrence reason of the fault are determined, and a solution is obtained. The processing module 203 is configured to switch the monitoring mode to the fault teleoperation mode after obtaining the solution, and process the fault through the teleoperation controller.
Further, the monitoring module 201 includes: a determining unit and a model building unit.
And the determining unit is used for determining the composition and the layout of the automatic spraying teleoperation assembly control device. The model establishing unit is used for realizing the establishment of the automatic spraying field augmented reality modeling through multi-source data processing, multi-source information registration and virtual-real information fusion in sequence.
Specifically, under the support of a workshop characteristic data model and an information expression template, the characteristic data and key information of multi-source data collected on site are quickly extracted, identified and optimized, and then the uniform expression of heterogeneous information is realized; registering the multi-source sensing information based on a least square rule and a maximum likelihood registration algorithm; and dynamically calibrating the field real-time information and the 3D virtual model in real time, and performing interaction of multi-source information on the model to obtain an automatic spraying field virtual-real fusion enhanced model.
The failure diagnosis determining module 200 includes: the device comprises a first acquisition unit, a second acquisition unit and a diagnosis judgment unit.
The method comprises the steps of reasonably storing operation data, fault data, historical data, process data, resource data, safety processing data and the like used in teleoperation control in a knowledge base, wherein the knowledge base comprises a fault instance knowledge base and a fault diagnosis rule knowledge base, the fault instance knowledge base stores fault phenomena, fault sources and fault reasons of faults, and the fault diagnosis rule knowledge base stores rules for diagnosing, reasoning, judging and processing the faults.
The first obtaining unit is used for loading the fault phenomenon, determining a search strategy, and searching the example base by using the search strategy to obtain a similar fault example set. And the second acquisition unit is used for acquiring the instances meeting the similarity threshold value for reuse through instance matching calculation. The diagnosis and judgment unit is used for diagnosing the type of the fault which possibly occurs by carrying out analog analysis on the similar examples, and providing a current fault processing method suggestion according to the fault processing method in the examples so as to provide a basis for teleoperation control.
The processing module 203 comprises: the remote control system comprises a mode switching unit, a remote control unit and a processing unit.
And the mode switching unit is used for switching the mode from a normal automatic operation mode to a remote operation safety processing mode. And the remote control unit is used for determining the fault type and proposing a safety processing mode suggestion based on the fault mode and the processing method knowledge base, and guiding teleoperation personnel to carry out remote safety processing. The processing unit is used for a teleoperation operator to operate the teleoperation controller to process safety problems, the teleoperation controller generates teleoperation signals, the teleoperation signals are transmitted to a PLC of the paint spraying equipment through conversion, the field equipment is controlled to execute teleoperation processing behaviors, and after the safety processing is finished, the teleoperation processing process of the safety problems is ended.
It is noted that, in this document, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Of course, the above description is not limited to the above examples, and technical features that are not described in this application may be implemented by or using the prior art, and are not described herein again; the above embodiments and drawings are only for illustrating the technical solutions of the present application and not for limiting the present application, and the present application is only described in detail with reference to the preferred embodiments instead, it should be understood by those skilled in the art that changes, modifications, additions or substitutions within the spirit and scope of the present application may be made by those skilled in the art without departing from the spirit of the present application, and the scope of the claims of the present application should also be covered.

Claims (10)

1.一种基于数字孪生的自动喷涂监测遥操作方法，其特征在于，所述方法包括：1. an automatic spraying monitoring teleoperation method based on digital twin, is characterized in that, described method comprises: 通过传感网络对自动喷漆现场进行实时监测并且建立现场实时增强模型；Real-time monitoring of the automatic painting site through the sensor network and establishment of a real-time enhanced model of the site; 如果监测到自动喷漆过程发生异常，则根据知识库对故障进行诊断，确定故障的类型和发生的原因，获得解决方法；If an abnormality is detected in the automatic painting process, the fault will be diagnosed according to the knowledge base, the type and cause of the fault will be determined, and the solution will be obtained; 获得解决方法后，从监测模式切换为故障遥操作模式，通过遥操作控制器对故障进行处理。After obtaining the solution, switch from the monitoring mode to the fault remote operation mode, and handle the fault through the remote operation controller. 2.根据权利要求1所述的方法，其特征在于，所述通过传感网络对自动喷漆现场进行实时监测并且建立现场实时增强模型包括：2. method according to claim 1, is characterized in that, described to carry out real-time monitoring to automatic painting scene by sensor network and establish scene real-time enhancement model comprises: 确定自动喷涂遥操作装配控制装置组成及布局；Determine the composition and layout of the automatic spraying remote operation assembly control device; 依次通过多源数据处理、多源信息配准和虚实信息融合实现自动喷涂现场增强现实建模的建立。Through multi-source data processing, multi-source information registration and virtual-real information fusion in turn, the establishment of augmented reality modeling of automatic spraying site is realized. 3.根据权利要求2所述的方法，其特征在于，所述依次通过多源数据处理、多源信息配准和虚实信息融合实现自动喷涂现场增强现实建模的建立，包括：3. method according to claim 2, is characterized in that, described successively realizes the establishment of automatic spraying on-site augmented reality modeling by multi-source data processing, multi-source information registration and virtual-real information fusion, comprising: 在车间特征数据模型和信息表达模板的支撑下，对现场采集的多源数据进行特征数据和关键信息的快速提取、识别和优选，进而实现异构信息的统一表达；Under the support of the workshop feature data model and information expression template, the multi-source data collected on the spot can be quickly extracted, identified and optimized for feature data and key information, thereby realizing the unified expression of heterogeneous information; 基于最小二乘规则和极大似然配准算法，对多源传感信息进行配准；Based on the least squares rule and the maximum likelihood registration algorithm, the multi-source sensor information is registered; 将现场实时信息和3D虚拟模型进行动态实时校准，并在模型上进行多源信息的交互，获得自动喷涂现场虚实融合增强模型。Dynamic real-time calibration is performed between the real-time information on site and the 3D virtual model, and multi-source information interaction is carried out on the model to obtain an enhanced model of virtual-real fusion of the automatic spraying site. 4.根据权利要求1所述的方法，其特征在于，将遥操作控制中用到的运行数据、故障数据、历史数据、工艺数据、资源数据以及安全处理数据等合理地存储在知识库中，所述知识库包括故障实例知识库和故障诊断规则知识库，所述故障实例知识库存储故障的故障现象、故障源和故障原因，所述故障诊断规则知识库存储对故障进行诊断、推理、判定、处理的规则。4. method according to claim 1 is characterized in that, operating data, fault data, historical data, process data, resource data and safe processing data etc. used in the remote operation control are reasonably stored in the knowledge base, The knowledge base includes a fault instance knowledge base and a fault diagnosis rule knowledge base, the fault instance knowledge base stores the fault phenomenon, fault source and fault cause of the fault, and the fault diagnosis rule knowledge base stores the fault diagnosis, reasoning and judgment. , processing rules. 5.根据权利要求4所述的方法，其特征在于，如果监测到自动喷漆过程发生异常，则根据知识库对故障进行诊断，确定故障的类型和发生的原因，获得解决方法，包括：5. method according to claim 4, is characterized in that, if monitoring automatic painting process abnormality, then according to knowledge base, the fault is diagnosed, the type of fault and the cause of occurrence are determined, and the solution method is obtained, comprising: 首先对故障现象进行加载，确定搜索策略，利用其对实例库进行搜索，获得相似故障实例集；First, load the fault phenomenon, determine the search strategy, and use it to search the instance library to obtain a set of similar fault instances; 通过实例匹配计算，获得满足相似度阈值的实例进行重用；Through instance matching calculation, obtain instances that meet the similarity threshold for reuse; 通过对相似实例进行类比分析，诊断出可能发生的故障类型，并根据实例中的故障处理方法提出当前故障处理方法建议，为遥操作控制提供基础。Through the analogy analysis of similar instances, the possible fault types are diagnosed, and the current fault handling method is proposed according to the fault handling method in the instance, which provides the basis for remote operation control. 6.根据权利要求5所述的方法，其特征在于，所述获得解决方法后，从监测模式切换为故障遥操作模式，通过遥操作控制器对故障进行处理，包括：6. The method according to claim 5, wherein after obtaining the solution method, switching from the monitoring mode to the fault remote operation mode, and processing the fault through the remote operation controller, comprising: 进行模式切换，从正常的自动化运行模式切换到遥操作安全处理模式；Perform mode switching from normal automatic operation mode to remote operation safe processing mode; 基于故障模式和处理方法知识库进行故障类型的确定和安全处理方式建议的提出，指导遥操作人员进行远程安全处理；Based on the knowledge base of fault modes and processing methods, determine fault types and propose safe processing methods, and guide remote operators to perform remote safe processing; 遥操作人员操纵遥操作控制器进行安全问题处理，遥操作控制器产生遥操作信号，通过转换发送至喷漆设备的PLC中，控制现场设备执行遥操作处理行为，安全处理完成后，结束安全问题遥操作处理进程。The remote operator manipulates the remote controller to deal with safety issues. The remote controller generates a teleoperation signal, which is sent to the PLC of the painting equipment through conversion to control the field equipment to perform teleoperation processing. After the safety processing is completed, the remote operation of the safety problem ends. Action handler. 7.一种基于数字孪生的自动喷涂监测遥操作系统，其特征在于，所述系统包括：7. A kind of automatic spraying monitoring teleoperation system based on digital twin, is characterized in that, described system comprises: 监测模块，用于通过传感网络对自动喷漆现场进行实时监测并且建立现场实时增强模型；The monitoring module is used to monitor the automatic painting site in real time through the sensor network and establish a real-time enhanced model of the site; 故障诊断判断模块，如果监测到自动喷漆过程发生异常，则根据知识库对故障进行诊断，确定故障的类型和发生的原因，获得解决方法；Fault diagnosis and judgment module, if an abnormality is detected in the automatic painting process, it will diagnose the fault according to the knowledge base, determine the type and cause of the fault, and obtain a solution; 处理模块，用于获得解决方法后，从监测模式切换为故障遥操作模式，通过遥操作控制器对故障进行处理。The processing module is used to switch from the monitoring mode to the fault remote operation mode after obtaining the solution, and process the fault through the remote operation controller. 8.根据权利要求7所述的系统，其特征在于，所述监测模块，包括：8. The system according to claim 7, wherein the monitoring module comprises: 确定单元，用于确定自动喷涂遥操作装配控制装置组成及布局；Determining unit, used to determine the composition and layout of the automatic spraying remote operation assembly control device; 模型建立单元，用于依次通过多源数据处理、多源信息配准和虚实信息融合实现自动喷涂现场增强现实建模的建立。The model establishment unit is used to realize the establishment of the augmented reality modeling of the automatic spraying scene through multi-source data processing, multi-source information registration and virtual-real information fusion in turn. 9.根据权利要求7所述的系统，其特征在于，所述故障诊断判断模块包括：9. The system according to claim 7, wherein the fault diagnosis and judgment module comprises: 第一获取单元，用于首先对故障现象进行加载，确定搜索策略，利用其对实例库进行搜索，获得相似故障实例集；The first acquisition unit is used to first load the fault phenomenon, determine the search strategy, and use it to search the instance library to obtain a set of similar fault instances; 第二获取单元，用于通过实例匹配计算，获得满足相似度阈值的实例进行重用；The second obtaining unit is used to obtain the instances that meet the similarity threshold for reuse through instance matching calculation; 诊断判断单元，用于通过对相似实例进行类比分析，诊断出可能发生的故障类型，并根据实例中的故障处理方法提出当前故障处理方法建议，为遥操作控制提供基础。The diagnosis and judgment unit is used for diagnosing possible fault types by analogy analysis of similar instances, and proposes the current fault handling method according to the fault handling method in the instance, so as to provide a basis for remote operation control. 10.根据权利要求9所述的系统，其特征在于，所述处理模块包括：10. The system of claim 9, wherein the processing module comprises: 模式切换单元，用于进行模式切换，从正常的自动化运行模式切换到遥操作安全处理模式；A mode switching unit, which is used for mode switching, from the normal automatic operation mode to the remote operation safety processing mode; 远程遥控单元，用于基于故障模式和处理方法知识库进行故障类型的确定和安全处理方式建议的提出，指导遥操作人员进行远程安全处理；The remote control unit is used to determine the fault type and propose the safe processing method based on the fault mode and processing method knowledge base, and guide the remote operator to carry out remote safe processing; 处理单元，用于遥操作人员操纵遥操作控制器进行安全问题处理，遥操作控制器产生遥操作信号，通过转换发送至喷漆设备的PLC中，控制现场设备执行遥操作处理行为，安全处理完成后，结束安全问题遥操作处理进程。The processing unit is used for the remote operator to manipulate the remote controller to handle safety issues. The remote controller generates a remote signal, which is sent to the PLC of the painting equipment through conversion to control the field device to perform the remote operation. After the safety processing is completed to end the security issue teleoperation processing process.
CN202011228212.7A 2020-11-06 2020-11-06 Automatic spraying monitoring teleoperation method and system based on digital twinning Active CN112547352B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011228212.7A CN112547352B (en) 2020-11-06 2020-11-06 Automatic spraying monitoring teleoperation method and system based on digital twinning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011228212.7A CN112547352B (en) 2020-11-06 2020-11-06 Automatic spraying monitoring teleoperation method and system based on digital twinning

Publications (2)

Publication Number Publication Date
CN112547352A true CN112547352A (en) 2021-03-26
CN112547352B CN112547352B (en) 2022-08-02

Family

ID=75041518

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011228212.7A Active CN112547352B (en) 2020-11-06 2020-11-06 Automatic spraying monitoring teleoperation method and system based on digital twinning

Country Status (1)

Country Link
CN (1) CN112547352B (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113657524A (en) * 2021-08-23 2021-11-16 北京奔驰汽车有限公司 Method and device for detecting quality of full process flow of coating workshop
CN113695109A (en) * 2021-08-06 2021-11-26 成都飞机工业（集团）有限责任公司 Program matching and iterative optimization method for automatic spraying process
CN113779769A (en) * 2021-08-18 2021-12-10 国网浙江省电力有限公司舟山供电公司 A cable test digital twin system and its working method
CN115202962A (en) * 2022-05-20 2022-10-18 江苏欧软信息科技有限公司 A method and system for rapid diagnosis of equipment faults based on an industrial Internet platform
CN116184938A (en) * 2021-11-26 2023-05-30 株式会社安川电机 Coating diagnosis system, coating diagnosis method
CN118818016A (en) * 2024-09-13 2024-10-22 潍柴动力股份有限公司 Engine paint quality detection method, device, storage medium and electronic equipment

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106003076A (en) * 2016-06-22 2016-10-12 潘小胜 Powder spraying robot based on stereoscopic vision
KR20170106536A (en) * 2016-03-10 2017-09-21 두림야스카와(주) Method for Managing Data of Painting Robot System
US20200050163A1 (en) * 2017-05-03 2020-02-13 Siemens Aktiengesellschaft Process image within controllers enabling visibility and accessibility of real world objects
CN110815224A (en) * 2019-11-14 2020-02-21 华南智能机器人创新研究院 A remote fault diagnosis and push method and device for a robot
CN111167668A (en) * 2020-03-06 2020-05-19 广东工业大学 Dispensing process optimization method for mobile phone manufacturing
CN111199573A (en) * 2019-12-30 2020-05-26 成都索贝数码科技股份有限公司 Virtual-real mutual reflection method, device, medium and equipment based on augmented reality
CN111596604A (en) * 2020-06-12 2020-08-28 中国科学院重庆绿色智能技术研究院 An intelligent fault diagnosis and self-healing control system and method for engineering equipment based on digital twin

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20170106536A (en) * 2016-03-10 2017-09-21 두림야스카와(주) Method for Managing Data of Painting Robot System
CN106003076A (en) * 2016-06-22 2016-10-12 潘小胜 Powder spraying robot based on stereoscopic vision
US20200050163A1 (en) * 2017-05-03 2020-02-13 Siemens Aktiengesellschaft Process image within controllers enabling visibility and accessibility of real world objects
CN110815224A (en) * 2019-11-14 2020-02-21 华南智能机器人创新研究院 A remote fault diagnosis and push method and device for a robot
CN111199573A (en) * 2019-12-30 2020-05-26 成都索贝数码科技股份有限公司 Virtual-real mutual reflection method, device, medium and equipment based on augmented reality
CN111167668A (en) * 2020-03-06 2020-05-19 广东工业大学 Dispensing process optimization method for mobile phone manufacturing
CN111596604A (en) * 2020-06-12 2020-08-28 中国科学院重庆绿色智能技术研究院 An intelligent fault diagnosis and self-healing control system and method for engineering equipment based on digital twin

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113695109A (en) * 2021-08-06 2021-11-26 成都飞机工业（集团）有限责任公司 Program matching and iterative optimization method for automatic spraying process
CN113779769A (en) * 2021-08-18 2021-12-10 国网浙江省电力有限公司舟山供电公司 A cable test digital twin system and its working method
CN113657524A (en) * 2021-08-23 2021-11-16 北京奔驰汽车有限公司 Method and device for detecting quality of full process flow of coating workshop
CN116184938A (en) * 2021-11-26 2023-05-30 株式会社安川电机 Coating diagnosis system, coating diagnosis method
CN115202962A (en) * 2022-05-20 2022-10-18 江苏欧软信息科技有限公司 A method and system for rapid diagnosis of equipment faults based on an industrial Internet platform
CN118818016A (en) * 2024-09-13 2024-10-22 潍柴动力股份有限公司 Engine paint quality detection method, device, storage medium and electronic equipment
CN118818016B (en) * 2024-09-13 2025-01-17 潍柴动力股份有限公司 Method and device for detecting paint spraying quality of engine, storage medium and electronic equipment

Also Published As

Publication number Publication date
CN112547352B (en) 2022-08-02

Similar Documents

Publication Publication Date Title
CN112547352A (en) Automatic spraying monitoring teleoperation method and system based on digital twinning
Liu et al. A method of NC machine tools intelligent monitoring system in smart factories
JP7012871B2 (en) Devices and methods for controlling the system
US7463945B2 (en) Electronic fingerprints for machine control and production machines
US20150026107A1 (en) System and apparatus that identifies, captures, classifies and deploys tribal knowledge unique to each operator in a semi-automated manufacturing set-up to execute automatic technical superintending operations to improve manufacturing system performance and the methods therefor
CN112418523B (en) After-sales equipment predictive maintenance cooperative system based on digital twinning
JP5077835B2 (en) Plant analysis system
Martinova et al. Creation of CNC system’s components for monitoring machine tool health
Shin et al. Smart manufacturing systems engineering for designing smart product-quality monitoring system in the Industry 4.0
CN118228122A (en) An intelligent industrial Internet of Things terminal detection system
Bertovic et al. NDE 4.0: new paradigm for the NDE inspection personnel
US20230297085A1 (en) Equipment effectiveness in manufacturing environment
CN111830871B (en) Automatic equipment abnormality diagnosis system
JP7181257B2 (en) Cause analysis system and method
CN115731073B (en) Scheduling operation abnormity monitoring method based on business scene analysis
Bertovic et al. NDE 4.0: new paradigm for the NDE inspection personnel
CN120429586A (en) Industrial equipment multi-mode fault prediction method and system based on deep learning
KR102306430B1 (en) Industrial Process Equipment System Management Method using Cloud-based Intergrated Context-Aware Processing Engine
CN119299484A (en) A safety and environmental inspection system
Nagy et al. Towards an ontology-based fault detection and diagnosis framework-a semantic approach
CN117193214A (en) Workshop collaborative intelligent manufacturing system based on Internet of things
JP6798968B2 (en) Noise cause estimation device
KR102570756B1 (en) Visualization apparatus for manufacturing execution, and method applied to the same
Bertovic et al. NDE 4.0: New Paradigm for the NDE Inspection Personnel
US20240289518A1 (en) Method for Recognizing Causes of Anomalies in a Physical Product

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
点击 这是indexloc提供的php浏览器服务，不要输入任何密码和下载