CN117235668A - CNN model fusion-based fault diagnosis method and system for heavy-duty gearbox - Google Patents
CNN model fusion-based fault diagnosis method and system for heavy-duty gearbox Download PDFInfo
- Publication number
- CN117235668A CN117235668A CN202311238628.0A CN202311238628A CN117235668A CN 117235668 A CN117235668 A CN 117235668A CN 202311238628 A CN202311238628 A CN 202311238628A CN 117235668 A CN117235668 A CN 117235668A
- Authority
- CN
- China
- Prior art keywords
- neural network
- convolutional neural
- heavy
- fusion
- network model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Testing Of Devices, Machine Parts, Or Other Structures Thereof (AREA)
Abstract
Description
技术领域Technical field
本发明涉及齿轮箱故障诊断技术,具体涉及一种基于CNN模型融合的重载齿轮箱故障诊断方法和系统。The invention relates to gearbox fault diagnosis technology, and specifically relates to a heavy-duty gearbox fault diagnosis method and system based on CNN model fusion.
背景技术Background technique
重载齿轮箱是现代装备制造业中传递运动和调配速度的重要功能部件,重载齿轮箱系统一般包括大型齿轮、轴承、轴和箱体四部分,与普通齿轮箱相同,重载齿轮箱的故障可以分为机械故障、电气故障、辅助系统故障。机械故障主要为齿轮故障、轴承故障、轴故障、箱体故障,电气故障和辅助系统故障主要有冷却故障、供油故障、传感器故障。在这三类故障中,电气故障和辅助系统故障虽然发生频率较高,但是产生的后果相对来说并不严重,处理也较为方便。机械故障次数少,但是由于重载齿轮箱结构复杂且尺寸巨大,因此机械故障处理困难、成本较高。据统计,73%的故障处理时间,都是在处理机械故障。Heavy-duty gearboxes are important functional components for transmitting motion and adjusting speed in modern equipment manufacturing. Heavy-duty gearbox systems generally include four parts: large gears, bearings, shafts and boxes. They are the same as ordinary gearboxes. Faults can be divided into mechanical faults, electrical faults, and auxiliary system faults. Mechanical failures are mainly gear failures, bearing failures, shaft failures, and box failures. Electrical failures and auxiliary system failures include cooling failures, oil supply failures, and sensor failures. Among these three types of faults, although electrical faults and auxiliary system faults occur more frequently, their consequences are relatively less serious and they are easier to deal with. The number of mechanical failures is low, but due to the complex structure and large size of the heavy-duty gearbox, mechanical failure treatment is difficult and costly. According to statistics, 73% of troubleshooting time is spent dealing with mechanical failures.
重载齿轮箱部件发生故障时会产生异常声,其声信号的幅值和频率成分都会随之发生相应的变化,而重载齿轮箱声信号里包含着大量的重载齿轮箱内部部件运行状态信息,因此利用重载齿轮箱产生的声信号能有效地反应出重载齿轮箱的运行状态。通过对重载齿轮箱声信号的分析,可以在不停机情况下对故障作出准确的判断,且声信号检测法具有不接触、诊断速度快、精度较高、故障部位判断较准确等优点,因此,声检测法目前在重载齿轮箱故障诊断中得到广泛应用。When a heavy-duty gearbox component fails, abnormal sound will be produced, and the amplitude and frequency components of the sound signal will change accordingly. The sound signal of the heavy-duty gearbox contains a large number of operating conditions of the internal components of the heavy-duty gearbox. information, so the acoustic signal generated by the heavy-duty gearbox can effectively reflect the operating status of the heavy-duty gearbox. By analyzing the acoustic signal of the heavy-loaded gearbox, the fault can be accurately judged without stopping the machine. The acoustic signal detection method has the advantages of non-contact, fast diagnosis speed, high accuracy, and more accurate fault location judgment. Therefore, , the acoustic detection method is currently widely used in fault diagnosis of heavy-duty gearboxes.
发明内容Contents of the invention
发明目的:本发明的目的在于提供一种基于CNN模型融合的重载齿轮箱故障诊断方法和系统,能够更精确地检测重载齿轮箱故障。Purpose of the invention: The purpose of the present invention is to provide a heavy-load gearbox fault diagnosis method and system based on CNN model fusion, which can detect heavy-load gearbox faults more accurately.
技术方案:本发明的一种基于CNN模型融合的重载齿轮箱故障诊断方法,包括以下步骤:Technical solution: A heavy-duty gearbox fault diagnosis method based on CNN model fusion of the present invention includes the following steps:
采集重载齿轮箱声信号,形成训练数据集;Collect the acoustic signals of the heavy-duty gearbox to form a training data set;
对采集到的声信号提取GFCC特征组成特征集,将特征集输入至预先构建的初始的一维卷积神经网络模型中进行训练,得到经过训练后一维卷积神经网络模型,并输出识别结果集;Extract GFCC features from the collected acoustic signals to form a feature set, input the feature set into the pre-built initial one-dimensional convolutional neural network model for training, obtain the trained one-dimensional convolutional neural network model, and output the recognition results set;
对采集到的声信号叠加声信号频谱得到声谱图,将声谱图输入至预先构建的初始的二维卷积神经网络模型中进行训练,得到训练后的二维卷积神经网络模型,并输出识别结果集;Add the acoustic signal spectrum to the collected acoustic signal to obtain a spectrogram, input the spectrogram into the pre-constructed initial two-dimensional convolutional neural network model for training, and obtain the trained two-dimensional convolutional neural network model, and Output the recognition result set;
将一维卷积神经网络模型和二维卷积神经网络模型的识别结果集输入融合模型中,进行融合训练,最终得到训练后的包括一维卷积神经网络模型、二维卷积神经网络模型、融合模型在内的融合卷积神经网络模型;Input the recognition result sets of the one-dimensional convolutional neural network model and the two-dimensional convolutional neural network model into the fusion model for fusion training, and finally obtain the trained one-dimensional convolutional neural network model and the two-dimensional convolutional neural network model. , fusion convolutional neural network model including fusion model;
获取故障声信号,对故障声信号进行对应特征提取和声谱图构建,将特征集和声谱图分别输入至经过训练的融合卷积神经网络模型中,得到故障识别结果,获得重载齿轮箱故障诊断信息。Obtain the fault sound signal, extract the corresponding features and construct the spectrogram of the fault sound signal, input the feature set and spectrogram respectively into the trained fusion convolutional neural network model, obtain the fault identification result, and obtain the heavy-duty gearbox Troubleshooting information.
进一步的,对采集到的声信号提取GFCC特征组成特征集,包括以下步骤:Further, extracting GFCC features from the collected acoustic signals to form a feature set includes the following steps:
对采集到的声信号进行分帧;Divide the collected acoustic signals into frames;
对分帧后的声信号进行加Hamming窗处理;Add Hamming window processing to the framed acoustic signal;
对加Hamming窗处理后的声信号进行离散傅里叶变换,得到能量谱;Perform discrete Fourier transform on the acoustic signal processed by the Hamming window to obtain the energy spectrum;
利用Gammatone滤波器组对能量谱进行滤波;Use Gammatone filter bank to filter the energy spectrum;
对滤波后的能量谱进行对数压缩,得到相应对数的能量信号;Perform logarithmic compression on the filtered energy spectrum to obtain the corresponding logarithmic energy signal;
对能量信号进行离散余弦变换;Perform discrete cosine transform on the energy signal;
引入信息熵来度量离散余弦变换后的能量信号的复杂度,设置阈值,得到GFCC特征参数,组成所需的特征集。Information entropy is introduced to measure the complexity of the energy signal after discrete cosine transform, and the threshold is set to obtain the GFCC characteristic parameters to form the required feature set.
进一步的,对采集到的声信号叠加声信号频谱得到声谱图,包括以下步骤:Further, superimposing the acoustic signal spectrum on the collected acoustic signal to obtain the spectrogram includes the following steps:
对采集到的声信号进行分帧;Divide the collected acoustic signals into frames;
对分帧后的声信号进行加Hamming窗处理;Add Hamming window processing to the framed acoustic signal;
对加Hamming窗处理后的声信号进行快速傅里叶变换,得到频谱。The fast Fourier transform is performed on the acoustic signal processed by the Hamming window to obtain the spectrum.
对快速傅里叶变换得到的频谱进行主成分分析降维;Perform principal component analysis on the spectrum obtained by fast Fourier transform to reduce dimensionality;
对降维后频谱进行叠加,得到声谱图。The spectrum after dimensionality reduction is superimposed to obtain the spectrogram.
进一步的,所述一维卷积神经网络模型包括一维特征参数输入层、两层卷积层、两层最大池化层、全连接层以及输出层。Further, the one-dimensional convolutional neural network model includes a one-dimensional feature parameter input layer, two convolutional layers, two maximum pooling layers, a fully connected layer and an output layer.
进一步的,所述卷积神经网络模型包括二维图像输入层、两层卷积层、两层最大池化层、全连接层以及输出层。Further, the convolutional neural network model includes a two-dimensional image input layer, two convolutional layers, two maximum pooling layers, a fully connected layer and an output layer.
进一步的,采用加权平均法对一维卷积神经网络模型和二维卷积神经网络模型的识别结果进行融合。Furthermore, the weighted average method is used to fuse the recognition results of the one-dimensional convolutional neural network model and the two-dimensional convolutional neural network model.
进一步的,对采集到的声信号进行分帧时,每帧涵盖的时间为2~3周期。Furthermore, when the collected acoustic signals are divided into frames, each frame covers 2 to 3 cycles.
基于相同的发明构思,本发明的一种基于CNN模型融合的重载齿轮箱声信号故障诊断系统,包括:Based on the same inventive concept, the invention's heavy-duty gearbox acoustic signal fault diagnosis system based on CNN model fusion includes:
信号获取模块,用于采集重载齿轮箱声信号,形成训练数据集;The signal acquisition module is used to collect the acoustic signals of the heavy-duty gearbox to form a training data set;
一维CNN模型构建模块,用于对采集到的声信号提取GFCC特征组成特征集,将特征集输入至预先构建的初始的一维卷积神经网络模型中进行训练,得到经过训练后一维卷积神经网络模型,并输出识别结果集;The one-dimensional CNN model building module is used to extract GFCC features from the collected acoustic signals to form a feature set. The feature set is input into the pre-built initial one-dimensional convolutional neural network model for training, and the trained one-dimensional volume is obtained. Accumulate the neural network model and output the recognition result set;
二维CNN模型构建模块,用于对采集到的声信号叠加声信号频谱得到声谱图,将声谱图输入至预先构建的初始的二维卷积神经网络模型中进行训练,得到训练后的二维卷积神经网络模型,并输出识别结果集;The two-dimensional CNN model building module is used to superimpose the acoustic signal spectrum on the collected acoustic signal to obtain a spectrogram. The spectrogram is input into the pre-built initial two-dimensional convolutional neural network model for training, and the trained result is obtained. Two-dimensional convolutional neural network model and output a recognition result set;
融合CNN模型构建模块,用于将一维卷积神经网络模型和二维卷积神经网络模型的识别结果集输入融合模型中,进行融合训练,最终得到训练后的包括一维卷积神经网络模型、二维卷积神经网络模型、融合模型在内的融合卷积神经网络模型;The fusion CNN model building module is used to input the recognition result sets of the one-dimensional convolutional neural network model and the two-dimensional convolutional neural network model into the fusion model, perform fusion training, and finally obtain the trained one-dimensional convolutional neural network model. , two-dimensional convolutional neural network model, fusion model including fusion convolutional neural network model;
故障识别模块,用于获取故障声信号,对故障声信号进行对应特征提取和声谱图构建,将特征集和声谱图分别输入至经过训练的融合卷积神经网络模型中,得到故障识别结果,获得重载齿轮箱故障诊断信息。The fault identification module is used to obtain the fault acoustic signal, extract the corresponding features and construct the spectrogram of the fault acoustic signal, and input the feature set and spectrogram into the trained fusion convolutional neural network model to obtain the fault identification result. , obtain heavy-duty gearbox fault diagnosis information.
基于相同的发明构思,本发明的一种基于CNN模型融合的重载齿轮箱故障诊断设备,包括处理器和存储器,所述存储器中存储有计算机指令,所述处理器用于执行所述存储器中存储的计算机指令,当所述计算机指令被处理器执行时该电子设备实现如上述基于CNN模型融合的重载齿轮箱故障诊断方法的步骤。Based on the same inventive concept, the present invention provides a heavy-duty gearbox fault diagnosis equipment based on CNN model fusion, including a processor and a memory. Computer instructions are stored in the memory, and the processor is used to execute the stored information in the memory. Computer instructions, when the computer instructions are executed by the processor, the electronic device implements the steps of the overloaded gearbox fault diagnosis method based on CNN model fusion.
基于相同的发明构思,本发明的一种计算机可读存储介质,其上存储有计算机程序,该程序被处理器执行时,实现如上述基于CNN模型融合的重载齿轮箱故障诊断方法的步骤。Based on the same inventive concept, the present invention provides a computer-readable storage medium on which a computer program is stored. When the program is executed by the processor, the steps of the overloaded gearbox fault diagnosis method based on CNN model fusion are implemented.
有益效果:与现有技术相比,本发明的显著技术效果为:Beneficial effects: Compared with the existing technology, the significant technical effects of the present invention are:
(1)重载齿轮箱尺寸巨大、结构复杂,因此声信号数据庞大、振幅与频域变化较快,而卷积神经网络具有较强的数据表示和分析能力,可挖掘数据局部特征,故采用卷积神经网络故障诊断技术,与其他故障诊断技术相比错误率更低、诊断时间更短、效率更高,省时省力。(1) The heavy-duty gearbox has a huge size and complex structure, so the acoustic signal data is huge and the amplitude and frequency domain change rapidly. However, the convolutional neural network has strong data representation and analysis capabilities and can mine local characteristics of the data, so it is used Compared with other fault diagnosis technologies, convolutional neural network fault diagnosis technology has lower error rate, shorter diagnosis time, higher efficiency, and saves time and effort.
(2)利用模型融合技术将基于能够完整描述声信号时域与频域联合分布特征的GFCC的一维CNN模型和基于能够表达声信号不同频段信号强度的声谱图的二维CNN模型进行融合加权,最终形成包括一维CNN模型、二维CNN模型、融合模型在内的融合CNN模型,充分发挥了不同模型的优势,有效改善了单个模型进行诊断时的特征丢失问题,提高了卷积神经网络模型的分类精度与诊断成功率。(2) Use model fusion technology to fuse the one-dimensional CNN model based on GFCC, which can completely describe the joint distribution characteristics of the acoustic signal in the time domain and frequency domain, and the two-dimensional CNN model based on the spectrogram, which can express the signal strength of the acoustic signal in different frequency bands. Weighted, the fusion CNN model including one-dimensional CNN model, two-dimensional CNN model and fusion model is finally formed, which fully utilizes the advantages of different models, effectively improves the problem of feature loss when a single model is used for diagnosis, and improves the convolutional neural network. Classification accuracy and diagnostic success rate of network models.
(3)在模型精度提高的情况下,该方法能够轻量化卷积神经网络模型,可以很好的应用于工业生产识别重载齿轮箱故障类别,解决了传统故障诊断方法识别错误率高的问题,在实际的工程中有良好的应用前景。(3) With the improvement of model accuracy, this method can lightweight the convolutional neural network model and can be well applied to industrial production to identify fault categories of heavy-duty gearboxes, solving the problem of high identification error rate of traditional fault diagnosis methods. , which has good application prospects in actual projects.
附图说明Description of drawings
图1是本发明实施例公开的一种基于CNN模型融合的重载齿轮箱故障诊断方法的流程示意图;Figure 1 is a schematic flow chart of a heavy-duty gearbox fault diagnosis method based on CNN model fusion disclosed in an embodiment of the present invention;
图2是本发明实施例公开的提取GFCC特征的流流程示意图;Figure 2 is a schematic flowchart of extracting GFCC features disclosed in the embodiment of the present invention;
图3是本发明实施例公开的构建声谱图的流程示意图;Figure 3 is a schematic flow chart of constructing a spectrogram disclosed in an embodiment of the present invention;
图4是本发明实施例公开的一维CNN与二维CNN模型融合的结构示意图;Figure 4 is a schematic structural diagram of the fusion of one-dimensional CNN and two-dimensional CNN models disclosed in the embodiment of the present invention;
图5是本发明实施例公开的一种基于CNN模型融合的重载齿轮箱故障诊断系统的结构示意图;Figure 5 is a schematic structural diagram of a heavy-duty gearbox fault diagnosis system based on CNN model fusion disclosed in an embodiment of the present invention;
图6是本发明实施例公开的一种基于CNN模型融合的重载齿轮箱故障诊断设备的结构示意图。Figure 6 is a schematic structural diagram of a heavy-duty gearbox fault diagnosis device based on CNN model fusion disclosed in an embodiment of the present invention.
具体实施方式Detailed ways
下面结合具体实施方式和说明书附图对本发明的技术方案进行详细介绍。The technical solution of the present invention will be introduced in detail below with reference to the specific embodiments and the accompanying drawings.
实施例1Example 1
如图1所示,本发明的一种基于CNN模型融合的重载齿轮箱故障诊断方法,可应用在重载齿轮箱故障诊断中,包括以下步骤:As shown in Figure 1, a heavy-duty gearbox fault diagnosis method based on CNN model fusion of the present invention can be applied in heavy-duty gearbox fault diagnosis, including the following steps:
S1、采集重载齿轮箱的声信号,形成训练数据集。S1. Collect the acoustic signals of the heavy-loaded gearbox to form a training data set.
在此步骤中,重载齿轮箱声信号可以在齿轮箱处于运行状态时由声压传感器采集。In this step, the heavy-duty gearbox acoustic signal can be collected by the sound pressure sensor while the gearbox is in operation.
本实施例中,采样传感器布置在距齿轮箱表面1m位置处,采集齿轮箱工作噪声;采样频率为3KHz,采样点数1000,采集的信号共四种:正常转动声音、齿轮故障声音、轴故障声音、轴承故障声音,将采集信号保存为.wav文件。In this embodiment, the sampling sensor is arranged 1m away from the surface of the gearbox to collect the operating noise of the gearbox; the sampling frequency is 3KHz, the number of sampling points is 1000, and there are four types of collected signals: normal rotation sound, gear failure sound, and shaft failure sound. , bearing failure sound, and save the collected signal as a .wav file.
本实施例中,在为检测重载齿轮箱故障而获取重载齿轮箱的声信号之前应提前训练重载齿轮箱卷积神经网络模型,由此,在检测重载齿轮箱故障时,可以直接调用预先建立的重载齿轮箱卷积神经网络模型。In this embodiment, the heavy-load gearbox convolutional neural network model should be trained in advance before acquiring the acoustic signal of the heavy-load gear box for detecting the fault of the heavy-load gear box. Therefore, when detecting the fault of the heavy-load gear box, the model can be directly Call the pre-built overloaded gearbox convolutional neural network model.
S2、对采集到的声信号提取GFCC Gammatone滤波器倒谱系数)特征,组成特征集,将特征集输入至预先构建的初始的一维卷积神经网络模型中进行训练,得到经过训练后一维卷积神经网络模型,并输出识别结果。S2. Extract the GFCC Gammatone filter cepstrum coefficient) features from the collected acoustic signals to form a feature set. The feature set is input into the pre-constructed initial one-dimensional convolutional neural network model for training, and the trained one-dimensional Convolutional neural network model and output recognition results.
在此步骤中,结合图2,图2示出了提取GFCC特征的流程图。在此示出的对声信号进行特征提取的步骤能够用于在图1中示出的实际检测重载齿轮箱故障过程中涉及的声信号的特征提取。In this step, combined with Figure 2, Figure 2 shows a flow chart for extracting GFCC features. The step of feature extraction of the acoustic signal shown here can be used for feature extraction of the acoustic signal involved in the actual process of detecting the fault of the heavy-duty gearbox shown in FIG. 1 .
如图2所示,本实施例中,对采集到的声信号提取GFCC特征,组成特征集,包括以下步骤:As shown in Figure 2, in this embodiment, extracting GFCC features from the collected acoustic signals to form a feature set includes the following steps:
S2.1、对采集到的声信号进行分帧,每帧涵盖的时间约为2~3周期;S2.1. Divide the collected acoustic signals into frames, and each frame covers about 2 to 3 cycles;
S2.2、对分帧后的声信号进行加Hamming窗处理,增加帧两端的连续性,减少频谱泄漏;S2.2. Add Hamming window processing to the framed acoustic signal to increase the continuity at both ends of the frame and reduce spectrum leakage;
S2.3、对加Hamming窗处理后的声信号进行离散傅里叶变换(DFT变换),得到能量谱;S2.3. Perform discrete Fourier transform (DFT transform) on the acoustic signal processed by the Hamming window to obtain the energy spectrum;
S2.4、利用Gammatone滤波器组对能量谱进行滤波;S2.4. Use Gammatone filter bank to filter the energy spectrum;
S2.5、对滤波后的能量谱进行对数压缩,得到相应对数的能量信号;S2.5. Perform logarithmic compression on the filtered energy spectrum to obtain the corresponding logarithmic energy signal;
S2.6、对能量信号进行离散余弦变换(DCT);S2.6. Perform discrete cosine transform (DCT) on the energy signal;
S2.7、引入信息熵来度量离散余弦变换后的能量信号的复杂度,设置阈值,得到GFCC特征参数,组成所需的特征集。S2.7. Introduce information entropy to measure the complexity of the energy signal after discrete cosine transform, set the threshold, obtain the GFCC characteristic parameters, and form the required feature set.
S3、对采集到的声信号叠加声信号频谱得到声谱图,将声谱图输入至预先构建的初始的二维卷积神经网络模型中进行训练,得到训练后的二维卷积神经网络模型,并输出识别结果集。S3. Add the acoustic signal spectrum to the collected acoustic signal to obtain a spectrogram, input the spectrogram into the pre-constructed initial two-dimensional convolutional neural network model for training, and obtain the trained two-dimensional convolutional neural network model. , and output the recognition result set.
在此步骤中,结合图3,图3示出了构建声谱图的流程图。在此示出的对声信号进行声谱图构建的步骤能够用于在图1中示出的实际检测重载齿轮箱故障过程中涉及的声信号的特征处理。In this step, combined with Figure 3, Figure 3 shows a flow chart for constructing a spectrogram. The step of spectrogram construction of the acoustic signal shown here can be used for the characteristic processing of the acoustic signal involved in the actual detection of a fault in a heavy-duty gearbox as shown in FIG. 1 .
如图3所示,本实施例中,对采集到的声信号叠加声信号频谱得到声谱图,包括以下步骤:As shown in Figure 3, in this embodiment, superimposing the acoustic signal spectrum on the collected acoustic signal to obtain a spectrogram includes the following steps:
S3.1、对采集到的声信号进行分帧,各帧重叠率设为50%;S3.1. Divide the collected acoustic signals into frames, and set the overlap rate of each frame to 50%;
S3.2、对分帧后的声信号进行加Hamming窗处理,增加帧两端的连续性,减少频谱泄漏;S3.2. Add Hamming window processing to the framed acoustic signal to increase the continuity at both ends of the frame and reduce spectrum leakage;
S3.3、对加Hamming窗处理后的声信号进行快速傅里叶变换(FFT变换),得到频谱;S3.3. Perform fast Fourier transform (FFT transform) on the acoustic signal processed by the Hamming window to obtain the spectrum;
S3.4、对快速傅里叶变换得到的频谱进行主成分分析(PCA)降维;S3.4. Perform dimensionality reduction using principal component analysis (PCA) on the spectrum obtained by fast Fourier transform;
S3.5、对降维后频谱进行叠加,得到声谱图。S3.5. Superpose the dimensionally reduced spectra to obtain a spectrogram.
S4、将一维卷积神经网络模型和二维卷积神经网络模型的识别结果集输入融合模型中,进行融合训练,最终得到训练后的包括一维卷积神经网络模型、二维卷积神经网络模型、融合模型在内的融合卷积神经网络模型。S4. Input the recognition result sets of the one-dimensional convolutional neural network model and the two-dimensional convolutional neural network model into the fusion model, perform fusion training, and finally obtain the trained model including the one-dimensional convolutional neural network model and the two-dimensional convolutional neural network model. Fusion convolutional neural network model including network model and fusion model.
在此步骤中,采用加权平均法对一维卷积神经网络模型和二维卷积神经网络模型的识别结果进行融合。In this step, the weighted average method is used to fuse the recognition results of the one-dimensional convolutional neural network model and the two-dimensional convolutional neural network model.
本方案中,训练模型算法为黑箱模型算法,即无法知晓其具体工作过程,只需输入足够多的数据,即可完成训练过程。In this solution, the training model algorithm is a black box model algorithm, that is, its specific working process cannot be known, and the training process can be completed by inputting enough data.
图4示出了重载齿轮箱故障诊断CNN融合模型的结构示意图。在本实施方式中,齿轮一维卷积神经网络模型包括一维特征参数输入层、两层卷积层、两层最大池化层、全连接层以及输出层。如图4所示,该一维模型的输入是大小为26x1维特征向量。第一个卷积层卷积核的数目为16,卷积核大小为8x1,步长为2,卷积后使用RELU激活函数引入非线性因素,卷积后得到的特征向量用0填充边缘部分;第一个卷积层后连接第一个池化层对卷积后的特征向量进行压缩,以简化网络计算复杂度,采用大小为2x1的卷积核进行最大池化,得到16个大小为8x1的特征向量;第二个卷积层卷积核的数目为32,卷积核大小为8x1,步长为2,激活函数是RELU,边缘部分用0填充;第二个池化层采用大小为2x1的卷积核进行最大池化,得到32个大小为2x1的特征向量;第二个池化层后是全连接层,用来连接所有的特征,将输出值送给softmax分类器,经全连接层后得到的64x1维特征向量;最后一个是输出层,大小为4x1。齿轮二维卷积神经网络模型包括二维图像输入层、两层卷积层、两层最大池化层、全连接层以及输出层。如图4所示,该二维模型的输入是大小为32像素x32像素x3通道(RGB)的声谱图。第一个卷积层卷积核的数目为32,卷积核大小为3x3,步长为2,卷积后使用RELU激活函数引入非线性因素,卷积后得到的特征向量用0填充边缘部分;第一个卷积层后连接第一个池化层对卷积后的特征图进行压缩,以简化网络计算复杂度,采用大小为2x2的卷积核进行最大池化,得到32个大小为16x16的特征图;第二个卷积层卷积核的数目为64,卷积核大小为3x3,步长为2,激活函数是;RELU,边缘部分用;0填充;第二个池化层采用大小为2x2的卷积核进行最大池化,得到64个大小为8x8的特征向量;第二个池化层后是全连接层,用来连接所有的特征,将输出值送给softmax分类器,经全连接层后得到的4096维特征向量;最后一个是输出层,大小为4x1。最后将输出结果集输入融合模型赋予权重,对两个CNN模型进行融合,输出融合结果。Figure 4 shows the structural diagram of the CNN fusion model for heavy-duty gearbox fault diagnosis. In this implementation, the gear one-dimensional convolutional neural network model includes a one-dimensional feature parameter input layer, two layers of convolutional layers, two layers of maximum pooling layers, a fully connected layer and an output layer. As shown in Figure 4, the input of this one-dimensional model is a feature vector of size 26x1. The number of convolution kernels in the first convolution layer is 16, the convolution kernel size is 8x1, and the step size is 2. After convolution, the RELU activation function is used to introduce nonlinear factors. The feature vector obtained after convolution fills the edge part with 0 ; After the first convolution layer, the first pooling layer is connected to compress the convolved feature vector to simplify the network calculation complexity. A convolution kernel of size 2x1 is used for maximum pooling, resulting in 16 pixels of size 8x1 feature vector; the number of convolution kernels in the second convolutional layer is 32, the convolution kernel size is 8x1, the stride is 2, the activation function is RELU, and the edge part is filled with 0; the second pooling layer uses size Maximum pooling is performed for the 2x1 convolution kernel to obtain 32 feature vectors of size 2x1; the second pooling layer is followed by a fully connected layer, which is used to connect all features and send the output value to the softmax classifier. The 64x1 dimensional feature vector obtained after the fully connected layer; the last one is the output layer, with a size of 4x1. The gear two-dimensional convolutional neural network model includes a two-dimensional image input layer, two convolutional layers, two maximum pooling layers, a fully connected layer and an output layer. As shown in Figure 4, the input to this 2D model is a spectrogram with a size of 32 pixels x 32 pixels x 3 channels (RGB). The number of convolution kernels in the first convolution layer is 32, the convolution kernel size is 3x3, and the step size is 2. After convolution, the RELU activation function is used to introduce nonlinear factors. The feature vector obtained after convolution fills the edge part with 0 ; After the first convolution layer, the first pooling layer is connected to compress the convolved feature map to simplify the network calculation complexity. A convolution kernel of size 2x2 is used for maximum pooling, and 32 pixels of size 2 are obtained. 16x16 feature map; the number of convolution kernels in the second convolution layer is 64, the convolution kernel size is 3x3, the stride is 2, the activation function is; RELU, the edge part is filled with; 0; the second pooling layer A convolution kernel of size 2x2 is used for maximum pooling, and 64 feature vectors of size 8x8 are obtained; the second pooling layer is followed by a fully connected layer, which is used to connect all features and send the output value to the softmax classifier. , the 4096-dimensional feature vector obtained after the fully connected layer; the last one is the output layer, with a size of 4x1. Finally, the output result set is input into the fusion model and weighted, the two CNN models are fused, and the fusion result is output.
S5、获取故障声信号,对故障声信号进行对应特征提取和声谱图构建,将特征集和声谱图分别输入至经过训练的融合卷积神经网络模型中,得到故障识别结果,获得重载齿轮箱故障诊断信息。S5. Obtain the fault acoustic signal, extract the corresponding features and construct the spectrogram of the fault acoustic signal, input the feature set and spectrogram respectively into the trained fusion convolutional neural network model, obtain the fault identification result, and obtain the overload Gearbox troubleshooting information.
当重载齿轮箱处于运行状态时,检测设备开始工作。首先,获取故障声信号,其次,对故障声信号进行对应特征提取和声谱图构建,将特征集和声谱图分别输入至训练好的融合卷积神经网络模型中,得到故障识别结果,获得重载齿轮箱故障诊断信息。借助根据本文的实施方式提供的用于检测重载齿轮箱故障的技术,能够借助卷积神经网络模型检测重载齿轮箱故障,精确度高。另外,在此提出的重载齿轮箱卷积神经网络模型具有客观性,不受主观因素影响,因此可靠性及稳定性较高。When the heavy-duty gearbox is in operation, the detection equipment starts to work. First, the fault sound signal is obtained. Secondly, the corresponding feature extraction and spectrogram construction of the fault sound signal are performed. The feature set and spectrogram are input into the trained fusion convolutional neural network model respectively to obtain the fault identification result. Obtain Heavy duty gearbox troubleshooting information. With the technology for detecting heavy-load gearbox faults provided according to embodiments of this article, heavy-load gearbox faults can be detected with high accuracy with the help of a convolutional neural network model. In addition, the heavy-duty gearbox convolutional neural network model proposed here is objective and not affected by subjective factors, so it has high reliability and stability.
由于重载齿轮箱声信号数据庞大,而卷积神经网络具有较强的数据表示和分析能力,故采用卷积神经网络故障诊断技术。在进行故障诊断时需要预先提取信号特征,GFCC(Gammatone滤波器倒谱系数)采用Gammatone滤波器模拟人耳耳蜗听觉模型,能较为完整的描述声信号时域与频域联合分布特征,且Gammatone滤波器的谱峰更加平坦,能有效改善信号分解的能量泄露问题,故提取GFCC特征进输入构建好的一维CNN模型中进行故障诊断。为提高识别准确率,构建基于能够表达声信号不同频段信号强度的声谱图的二维CNN模型,并将两种模型进行融合,输出融合识别结果。Since the acoustic signal data of the heavy-duty gearbox is huge, and the convolutional neural network has strong data representation and analysis capabilities, the convolutional neural network fault diagnosis technology is used. When performing fault diagnosis, signal features need to be extracted in advance. GFCC (Gammatone Filter Cepstral Coefficient) uses Gammatone filters to simulate the human cochlear hearing model, which can more completely describe the joint distribution characteristics of the acoustic signal in the time domain and frequency domain, and Gammatone filtering The spectrum peak of the detector is flatter, which can effectively improve the energy leakage problem of signal decomposition. Therefore, the GFCC features are extracted and input into the constructed one-dimensional CNN model for fault diagnosis. In order to improve the recognition accuracy, a two-dimensional CNN model based on the spectrogram that can express the signal intensity of different frequency bands of the acoustic signal is constructed, and the two models are fused to output the fusion recognition result.
在机械故障诊断的技术中,通过专业的传感器信号数据进行收集,利用深度学习算法建立重载齿轮箱故障卷积神经网络模型,相对于传统的人工探查,探伤设备探查,减少了故障诊断时间,人力及物力成本,在重载齿轮箱故障诊断的应用前景上具有巨大优势。In the mechanical fault diagnosis technology, professional sensor signal data is collected, and deep learning algorithms are used to establish a convolutional neural network model of heavy-duty gearbox faults. Compared with traditional manual detection and flaw detection equipment detection, fault diagnosis time is reduced. The cost of human and material resources has great advantages in the application prospects of fault diagnosis of heavy-duty gearboxes.
实施例2Example 2
如图5所示,本发明的一种基于CNN模型融合的重载齿轮箱故障诊断系统,可应用于重载齿轮箱故障诊断,具体包括:As shown in Figure 5, a heavy-duty gearbox fault diagnosis system based on CNN model fusion of the present invention can be applied to heavy-duty gearbox fault diagnosis, specifically including:
信号获取模块,用于采集重载齿轮箱声信号,形成训练数据集;The signal acquisition module is used to collect the acoustic signals of the heavy-duty gearbox to form a training data set;
一维CNN模型构建模块,用于对采集到的声信号提取GFCC特征组成特征集,将特征集输入至预先构建的初始的一维卷积神经网络模型中进行训练,得到经过训练后一维卷积神经网络模型,并输出识别结果集;The one-dimensional CNN model building module is used to extract GFCC features from the collected acoustic signals to form a feature set. The feature set is input into the pre-built initial one-dimensional convolutional neural network model for training, and the trained one-dimensional volume is obtained. Accumulate the neural network model and output the recognition result set;
二维CNN模型构建模块,用于对采集到的声信号叠加声信号频谱得到声谱图,将声谱图输入至预先构建的初始的二维卷积神经网络模型中进行训练,得到训练后的二维卷积神经网络模型,并输出识别结果集;The two-dimensional CNN model building module is used to superimpose the acoustic signal spectrum on the collected acoustic signal to obtain a spectrogram. The spectrogram is input into the pre-built initial two-dimensional convolutional neural network model for training, and the trained result is obtained. Two-dimensional convolutional neural network model and output a recognition result set;
融合CNN模型构建模块,用于将一维卷积神经网络模型和二维卷积神经网络模型的识别结果集输入融合模型中,进行融合训练,最终得到训练后的包括一维卷积神经网络模型、二维卷积神经网络模型、融合模型在内的融合卷积神经网络模型。The fusion CNN model building module is used to input the recognition result sets of the one-dimensional convolutional neural network model and the two-dimensional convolutional neural network model into the fusion model, perform fusion training, and finally obtain the trained one-dimensional convolutional neural network model. , two-dimensional convolutional neural network model, fusion model including fusion convolutional neural network model.
故障识别模块,用于获取故障声信号,对故障声信号进行对应特征提取和声谱图构建,将特征集和声谱图分别输入至经过训练的融合卷积神经网络模型中,得到故障识别结果,获得重载齿轮箱故障诊断信息。The fault identification module is used to obtain the fault acoustic signal, extract the corresponding features and construct the spectrogram of the fault acoustic signal, and input the feature set and spectrogram into the trained fusion convolutional neural network model to obtain the fault identification result. , obtain heavy-duty gearbox fault diagnosis information.
实施例3Example 3
如图6所示,本发明的一种基于CNN模型融合的重载齿轮箱故障诊断设备,可应用于重载齿轮箱故障诊断。该设备可以包括处理器和存储器,所述存储器中存储有计算机指令,所述处理器用于执行所述存储器中存储的计算机指令,当所述计算机指令被处理器执行时该电子设备实现如上述实施例所述方法的步骤,并能达到与上述方法一致的技术效果。As shown in Figure 6, a heavy-duty gearbox fault diagnosis equipment based on CNN model fusion of the present invention can be applied to heavy-duty gearbox fault diagnosis. The device may include a processor and a memory. Computer instructions are stored in the memory. The processor is configured to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the electronic device implements the above implementation. steps of the method described in the example, and can achieve the same technical effect as the above method.
存储器可以包括易失性存储器形式的计算机系统可读介质,例如随机存取存储器(RAM)和/或高速缓存存储器。设备可以进一步包括其它可移动/不可移动的、易失性/非易失性计算机系统存储介质。仅作为举例,存储器可以用于读写不可移动的、非易失性磁介质(通常称为“硬盘驱动器”)。具有一组(至少一个)程序模块的程序/实用工具,可以存储在例如存储器中,这样的程序模块包括但不限于操作系统、一个或者多个应用程序、其它程序模块以及程序数据,这些示例中的每一个或某种组合中可能包括网络环境的实现。程序模块通常执行本发明所描述的实施例中的功能和/或方法。Memory may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and/or cache memory. The device may further include other removable/non-removable, volatile/non-volatile computer system storage media. By way of example only, memory may be used to read and write to non-removable, non-volatile magnetic media (commonly referred to as "hard drives"). A program/utility having a set of (at least one) program modules, which may be stored, for example, in memory. Such program modules include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. In these examples Each of these, or some combination thereof, may include the implementation of a network environment. Program modules generally perform functions and/or methods in the described embodiments of the invention.
处理器通过运行存储在存储器中的程序,从而执行各种功能应用以及数据处理,例如实现本发明实施例一所提供的方法。The processor executes various functional applications and data processing by running programs stored in the memory, for example, implementing the method provided in Embodiment 1 of the present invention.
实施例4Example 4
本发明实施例4还提供了一种计算机可读存储介质,其上存储有计算机程序,该程序被处理器执行时,实现如上述实施例所述方法的步骤,并能达到与上述方法一致的技术效果。Embodiment 4 of the present invention also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the method described in the above embodiment are implemented, and the results consistent with the above method can be achieved. technical effects.
本发明实施例的计算机存储介质,可以采用一个或多个计算机可读的介质的任意组合。计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质。计算机可读存储介质例如可以是但不限于电、磁、光、电磁、红外线、或半导体的系统、装置或器件,或者任意以上的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括:具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本文件中,计算机可读存储介质可以是任何包含或存储程序的有形介质,该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。The computer storage medium in this embodiment of the present invention may be any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (non-exhaustive list) of computer readable storage media include: electrical connections having one or more conductors, portable computer disks, hard drives, random access memory (RAM), read only memory (ROM), Erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. As used herein, a computer-readable storage medium may be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device.
计算机可读的信号介质可以包括在基带中或者作为载波一部分传播的数据信号,其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式,包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质,该计算机可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device .
计算机可读介质上包含的程序代码可以用任何适当的介质传输,包括但不限于无线、电线、光缆、RF等等,或者上述的任意合适的组合。Program code embodied on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the foregoing.
可以以一种或多种程序设计语言或其组合来编写用于执行本发明操作的计算机程序代码,程序设计语言包括面向对象的程序设计语言,诸如Java、Smalltalk、C++,还包括常规的过程式程序设计语言,诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中,远程计算机可以通过任意种类的网络包括局域网(LAN)或广域网(WAN),连接到用户计算机,或者,可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。Computer program code for performing the operations of the present invention may be written in one or more programming languages, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages, or a combination thereof. A programming language, such as "C" or a similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In situations involving remote computers, the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer, such as through the Internet using an Internet service provider. ).
当然,本发明实施例所提供的一种包含计算机可执行指令的存储介质,其计算机可执行指令不限于如上的方法操作,还可以执行本发明任意实施例所提供的方法中的相关操作。Of course, embodiments of the present invention provide a storage medium containing computer-executable instructions, and the computer-executable instructions are not limited to the above method operations, and can also perform related operations in the method provided by any embodiment of the present invention.
以上所述的具体实施例,对本发明的目的,技术方案和有益效果进行了进一步详细说明,应理解的是,以上所述仅为本发明的具体实施例而已,并不用于限制本发明,凡在本发明的精神和原则之内,所做的任何修改,等同替换,改进等,均应包含在本发明的保护范围之内。The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Within the spirit and principles of the present invention, any modifications, equivalent substitutions, improvements, etc. shall be included in the protection scope of the present invention.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311238628.0A CN117235668A (en) | 2023-09-22 | 2023-09-22 | CNN model fusion-based fault diagnosis method and system for heavy-duty gearbox |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311238628.0A CN117235668A (en) | 2023-09-22 | 2023-09-22 | CNN model fusion-based fault diagnosis method and system for heavy-duty gearbox |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117235668A true CN117235668A (en) | 2023-12-15 |
Family
ID=89089195
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311238628.0A Pending CN117235668A (en) | 2023-09-22 | 2023-09-22 | CNN model fusion-based fault diagnosis method and system for heavy-duty gearbox |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117235668A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110047506A (en) * | 2019-04-19 | 2019-07-23 | 杭州电子科技大学 | A kind of crucial audio-frequency detection based on convolutional neural networks and Multiple Kernel Learning SVM |
| CN110059732A (en) * | 2019-04-01 | 2019-07-26 | 东南大学 | A kind of internal combustion engine Multiple faults diagnosis approach |
| WO2022082607A1 (en) * | 2020-10-22 | 2022-04-28 | Harman International Industries, Incorporated | Vocal track removal by convolutional neural network embedded voice finger printing on standard arm embedded platform |
| CN115132223A (en) * | 2022-06-29 | 2022-09-30 | 华中师范大学 | Audio data annotation precision enhancing method and system based on time frequency enhancement |
-
2023
- 2023-09-22 CN CN202311238628.0A patent/CN117235668A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110059732A (en) * | 2019-04-01 | 2019-07-26 | 东南大学 | A kind of internal combustion engine Multiple faults diagnosis approach |
| CN110047506A (en) * | 2019-04-19 | 2019-07-23 | 杭州电子科技大学 | A kind of crucial audio-frequency detection based on convolutional neural networks and Multiple Kernel Learning SVM |
| WO2022082607A1 (en) * | 2020-10-22 | 2022-04-28 | Harman International Industries, Incorporated | Vocal track removal by convolutional neural network embedded voice finger printing on standard arm embedded platform |
| CN115132223A (en) * | 2022-06-29 | 2022-09-30 | 华中师范大学 | Audio data annotation precision enhancing method and system based on time frequency enhancement |
Non-Patent Citations (1)
| Title |
|---|
| CHENG LIN ET AL.: "Application of acoustic digital twin model for fault monitoring of heavy duty gearbox", ICMIMT, 7 August 2023 (2023-08-07) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110534118B (en) | Transformer/reactor fault diagnosis method based on voiceprint recognition and neural network | |
| Chen et al. | Compound fault diagnosis for industrial robots based on dual-transformer networks | |
| WO2023185801A1 (en) | Vibration signal identification method and apparatus | |
| CN112329626A (en) | Modulation and deep learning fused equipment fault diagnosis method, system and medium | |
| Yin et al. | A fault diagnosis of nuclear power plant rotating machinery based on multi-sensor and deep residual neural network | |
| CN116933145A (en) | Fault determination methods for components in industrial equipment and related equipment | |
| CN116304832A (en) | Rolling bearing fault diagnosis method | |
| CN116399588A (en) | Rolling bearing fault diagnosis method based on WPD and AFRB-LWUNet under small sample | |
| CN118332488A (en) | Bearing diagnosis method and system based on variable operating condition multi-modal data fusion | |
| CN117725536A (en) | Cable joint diagnosis and abnormality early warning method and device based on multi-parameter fusion analysis | |
| CN117477764A (en) | Power system safety monitoring method and electronic equipment | |
| CN118521965A (en) | Anomaly detection method and system based on intelligent scene perception | |
| CN117739289B (en) | Leakage detection method and system based on sound-image fusion | |
| CN117235668A (en) | CNN model fusion-based fault diagnosis method and system for heavy-duty gearbox | |
| CN115389865A (en) | Power equipment fault diagnosis method and system based on cloud-edge-device collaboration architecture | |
| CN106355187A (en) | Application of visual information to electrical equipment monitoring | |
| CN119091909A (en) | A method and system for detecting abnormal sound based on neural network with multiple receptive fields | |
| CN117576574B (en) | A method, device, electronic equipment and medium for detecting ground object changes in electric power facilities | |
| CN118777865A (en) | A permanent magnet generator electrical fault diagnosis and analysis method and related device | |
| CN116610939B (en) | Asymmetric penalty sparse regularization pulse extraction method under Coiflet discrete wavelet | |
| CN118349827A (en) | A signal processing method based on deep neural network | |
| CN116913316A (en) | Power transformer typical fault voiceprint diagnosis method based on Mosaic data enhancement | |
| CN116296374A (en) | Method, system and device for diagnosing faults of gearbox | |
| Zhao et al. | Shuffle-fusion pyramid network for bearing fault diagnosis under noisy environments | |
| CN119988869A (en) | Bearing fault diagnosis method and system based on multi-source fusion deep adversarial network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |