KR20240141907A - Apparatus for determining resultant of auditory brainstem response based on artificial intelligence deep learning - Google Patents

Abstract

The present invention relates to a device and method for interpreting the results of a hearing test, and more particularly, to a device and method for interpreting the results of a hearing test for extracting a region of interest by interpreting a hearing test image of an auditory brainstem response test, and analyzing the section and time at which a specific signal of a waveform occurs by using artificial intelligence deep learning to interpret the presence or absence of hearing loss. The device for interpreting the results of a hearing test according to one embodiment of the present invention includes a data preprocessing unit for executing data preprocessing on a hearing test image of an auditory brainstem response test, and normalizing the value of each waveform of the data to a value between 0 and 1 by dividing the value of each waveform by the maximum value of each waveform, an AI module for extracting and analyzing the features of the waveform transmitted from the data preprocessing unit, and calculating a peak value and information on the time at which the peak occurs from the entire form of the waveform, and a hearing loss interpretation unit for interpreting the presence or absence of hearing loss from the peak value and information on the time at which the peak occurs of the waveform calculated by the AI module.

Description

{APPARATUS FOR DETERMINING RESULTANT OF AUDITORY BRAINSTEM RESPONSE BASED ON ARTIFICIAL INTELLIGENCE DEEP LEARNING}

The present invention relates to a hearing test result interpretation device, and more particularly, to a hearing test result interpretation device that interprets a hearing test image of an auditory brainstem response test to extract a region of interest, and analyzes the section and time at which a specific signal of a waveform occurs through artificial intelligence deep learning to interpret the presence or absence of hearing loss.

Hearing loss is when you can't hear sounds well. Hearing loss can be broadly divided into conductive hearing loss, sensorineural hearing loss, and mixed hearing loss depending on the abnormality, and is further subdivided based on the cause of the hearing loss.

Conductive hearing loss occurs when there is a problem with the outer or middle ear that conducts sound, sensorineural hearing loss occurs when there is a problem with the neural circuits after the inner ear, and mixed hearing loss occurs when there is a problem with the outer, middle, and inner ear.

Hearing loss interpretation is based on the results of pure tone audiometry (PTA) and auditory brainstem response (ABR) tests during the patient's hearing test.

Pure tone audiometry is one of the subjective evaluation methods that requires the testee's response. It is a test method that presents the testee with pure tones of each frequency band to calculate the quietest sound level that the individual can hear. Pure tone audiometry includes air conduction pure tone audiometry (ACT), in which the testee wears headphones and transmits pure tones to check the air conduction path, and bone conduction pure tone audiometry (BCT), in which the testee wears a bone conduction device and transmits sounds through the vibration of the skull to check the bone conduction path. The tester can differentiate conductive hearing loss, sensorineural hearing loss, and mixed hearing loss by interpreting the results of the hearing tests through air conduction pure tone audiometry (ACT) and bone conduction pure tone audiometry (BCT).

Auditory brainstem response (ABRS) testing is a test that records the electrical response from the auditory system to a sound stimulus using electrodes placed on the scalp. The ABRS is a response that occurs in the auditory nerve and brainstem within 10 msec after a sound stimulus, and can be measured regardless of anesthesia or sleep. Clicks and tone pips are used as stimulus sounds. Among these, clicks have no frequency characteristics, but they are widely used clinically because they have a short production time and duration and can simultaneously excite the auditory nerves. Tone pips are used to test frequency specificity, especially in cases of low-frequency hearing loss.

In order to be diagnosed with hearing impairment, you must take an auditory brainstem response test. The auditory brainstem response test is an objective test that determines the hearing threshold by checking the latency and amplitude of various waveforms during the process in which auditory signals are transmitted to the brainstem. The auditory brainstem response test is mainly conducted while lying down. In the case of children, it is difficult to take the test while they are awake, so the test is often conducted after they have taken sleeping pills. In the case of adults, it is also difficult to obtain accurate waveforms if they are tense while awake, so it is recommended to take the test at least during the meditation stage.

In relation to hearing loss tests, Japanese Patent Laid-Open No. 2001-231767 relates to a hearing test device having noise detection and judgment capabilities, and discloses a configuration that enables accurate detection of excessive noise and excessive non-physiological noise in the development of an auditory brain center response (ABR) judgment test that occurs in response to acoustic stimulation, and Korean Patent Laid-Open No. 2015-0131022 relates to a hearing test device, a hearing test method, and a method for writing words for a hearing test, and discloses a configuration that outputs a questionnaire sentence including at least one word that is likely to be misheard, and determines whether or not the respondent has hearing impairment based on the content of the response thereto, and Japanese Patent Registration No. 5221545 relates to an automatic hearing test method and an automatic objective hearing test device for an objective automatic hearing test, and discloses a method for automating a target hearing test based on the evaluation of an auditory evoked potential (AEP).

The above conventional technologies have room for improvement in data extraction and analysis for determining the presence or absence of hearing loss from auditory brainstem response (ABR) audiometric test images.

Japanese Publication Patent No. 2001-231767 Korean Patent Publication No. 2015-0131022 Japanese registered patent no. 5221545

The present invention has been created in consideration of the above circumstances, and the purpose of the present invention is to provide a hearing test result interpretation device that interprets a hearing test image of an auditory brainstem response test to extract a region of interest, and analyzes the section and time at which a specific signal of the waveform occurs through artificial intelligence deep learning to determine whether there is hearing loss.

A hearing test result interpretation device according to one embodiment of the present invention includes a data preprocessing unit that performs data preprocessing on a hearing test image of an auditory brainstem response test, normalizes the value of each waveform of the data to a value between 0 and 1 by dividing the value by the maximum value of each waveform, an AI module that extracts and analyzes features of the waveform transmitted from the data preprocessing unit, and calculates a peak value and information on the time at which the peak occurs from the entire shape of the waveform, and a hearing loss interpretation unit that determines whether there is hearing loss from the peak value of the waveform and information on the time at which the peak occurs calculated by the AI module.

The above data preprocessing unit extracts and analyzes peaks using the entire shape of the waveform, and through normalization (if the y value of the waveform has a category of (100, 200), all y values of the waveform are divided by the maximum value of 200 to have a category of (0.5, 1), and the values of each waveform are divided by the maximum value of each waveform to change them to values between 0 and 1.

When the above AI module receives the hearing test result data that has been preprocessed through the above data preprocessing unit, it extracts and analyzes the region of interest within the input data and outputs the V wave (peak 5) within the data waveform.

The machine learning technique used in the above AI module is deep learning, which is a multi-layered NN described in the artificial neural network (ANN), and uses a DNN (Deep Neural Network) in which learning is performed through a multi-layered neural network.

The above AI module

In order for each layer to perform the role of calculating the value input from the previous layer and sending it to the next layer, it is configured with multiple layers, and is configured to include the first to sixth layers, and the first layer includes the first input layer (Input1) and the second input layer (Input2), the second layer includes a Dense Layer and a Bidirectional RNN, the third layer includes a Reshape layer and a Time Distributed Dense Layer, the fourth layer includes a Concatenate Layer, the fifth layer includes a Dense Layer, and the sixth layer includes a softmax Dense Layer.

The device and method for interpreting hearing test results according to the present invention can increase the accuracy of interpreting the presence or absence of hearing loss by applying artificial intelligence deep learning technology to the hearing test image of the auditory brainstem response test to extract and analyze the region of interest.

FIG. 1 is a block diagram illustrating the configuration of a hearing test result reading device according to one embodiment of the present invention.
Figure 2 is a diagram showing the waveform of the hearing test results used as learning data.
Figure 3 is a diagram illustrating the neural network structure of the AI module of the present invention.
Figure 4 is a diagram illustrating the internal structure of a bidirectional RNN and a time distributed dense layer.

The terms or words used in this specification and claims are not to be construed as limited to their usual or dictionary meanings, and should be interpreted as meanings and concepts that conform to the technical idea of the present invention, based on the principle that the inventor can appropriately define the concept of the term in order to explain his or her own invention in the best manner.

Throughout the specification, when a part is said to "include" a certain component, this does not mean that other components are excluded, but rather that other components can be included, unless otherwise specifically stated. In addition, terms such as "part," "unit," "module," and "device" described in the specification mean a unit that processes at least one function or operation, and this can be implemented by a combination of hardware and/or software.

Throughout the specification, the term "and/or" should be understood to include all combinations possible from one or more of the associated items. For example, "the first item, the second item, and/or the third item" means not only the first, second, or third items, but also all combinations of items possible from two or more of the first, second, or third items.

Throughout the specification, identifiers (e.g., a, b, c, ...) for each step are used for convenience of explanation and do not limit the order of each step. Each step may occur in a different order than stated unless the context clearly indicates a specific order. That is, each step may occur in the same order as stated, may be performed substantially simultaneously, or may be performed in the opposite order.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram illustrating the configuration of a hearing test result reading device according to one embodiment of the present invention.

Referring to FIG. 1, the hearing test result reading device (100) of the present invention includes a data preprocessing unit (110), an AI module (120), and a hearing loss reading unit (130).

The data preprocessing unit (110) performs data preprocessing on the hearing test image of the auditory brainstem response test, normalizing the value of each waveform of the data to a value between 0 and 1 by dividing it by the maximum value of each waveform.

The AI module (120) extracts and analyzes the characteristics of the waveform transmitted from the data preprocessing unit (110), and calculates peak values and peak occurrence time information from the overall shape of the waveform. To this end, the AI module (120) may include a first layer (121), a second layer (122), a third layer (123), a fourth layer (124), a fifth layer (125), and a sixth layer (126), and the functions and roles of each layer will be described later.

The hearing loss interpretation unit (130) interprets whether there is hearing loss from the peak value of the waveform produced by the AI module (120) and the time information at which the peak occurs.

Figure 2 is a diagram showing the waveform of the hearing test results used as learning data.

Referring to Figure 2, there are a total of six waveforms, each of which is used as learning data, and each of the six waveforms represents the results of a hearing test for sound stimuli of different sizes (decibels).

First, when the hearing test result data that has been preprocessed through the data preprocessing unit (110) is input, the AI module (120) extracts and analyzes the region of interest within the data and outputs the V wave (peak 5) within the waveform of the data.

The waveform extracted in Fig. 2 is actual measurement data. The measurement data has different ranges of values depending on the sound size (decibels). If machine learning is executed in this state, the data range that the AI module (120) must calculate becomes very large, and the focus is on analyzing the actual peak value.

However, since the present invention extracts peaks using the entire shape of the waveform rather than the waveform value itself, the analysis is conducted with a focus on the entire shape of the waveform.

Therefore, through the preprocessing process (normalization), the value of each waveform is divided by the maximum value of each waveform and changed to a value between 0 and 1. For example, if the y value of the waveform has a category of (100, 200), all the y values of the waveform are divided by the maximum value of 200 to have a category of (0.5, 1).

As described above, the AI module (120) performs learning (DNN) on the data for which preprocessing has been performed. Hereinafter, an actual experimental example will be described for the convenience of understanding the present invention, but the spirit of the present invention is not limited thereto, and various modifications or alterations are possible.

The data extracted from the hearing test results is 2,098, of which 80%, or 1,684, are set as training data and 414 are set as test data to proceed with learning.

All machine learning uses learning data and test data to learn. Learning data is data that undergoes learning, and the AI module (120) learns the corresponding learning data using machine learning techniques.

Test data is data used to test an AI module (120) that has completed learning, and the performance of the actual AI module (120) is evaluated based on the test results of the test data.

The machine learning technique used in the AI module (120) of the present invention is deep learning, and among them, a DNN (Deep Neural Network) is used. DNN is a deep learning version of an artificial neural network (ANN). DNN is a NN described in ANN, that is, a neural network that is stacked in multiple layers, and learning is performed through a neural network with a multilayer structure.

Figure 3 is a diagram illustrating the neural network structure of the AI module of the present invention.

Referring to FIG. 3, the AI module (120) includes a first layer (121) to a sixth layer (126). The first layer (121) may include a first input layer (Input1) and a second input layer (Input2), the second layer (122) may include a dense layer and a bidirectional RNN, the third layer (123) may include a reshape layer and a time distributed dense layer, the fourth layer (124) may include a concatenate layer, the fifth layer (125) may include a dense layer, and the sixth layer may include a softmax dense layer.

Each layer calculates the values input from the previous layer and sends them to the next layer. In Figure 3, the arrows between each layer indicate the direction of data processing and movement.

The first layer (121) of the AI module (120) may be composed of two input layers. Each waveform of the auditory brainstem response test result is affected by the peak of the previously measured decibel waveform. In order to analyze the correlation between the peaks in each waveform, the peak value of the previously measured decibel waveform is input to the first input layer (Input1). The format of the input data is the same as the length of the total time (hereinafter T) of the waveform. Among them, the time t at which the peak is found is set to 1, and the time at which the peak is not found is set to 0.

If we express this as a formula, it is as follows.

[Mathematical Formula 1]

The second input layer (Input2) receives the preprocessed value of the current decibel waveform. The waveform input to the second input layer has a total time of T, similar to the waveform input to the first input layer.

The dense layer performs an overall analysis of the data. Typically, the previous layer of the dense layer is a layer that extracts data features. The dense layer, into which data features are input, analyzes or makes judgments based on the features.

The reshape layer is used to change the format of input data without any additional computational process. In the AI module (120) of the present invention, the reshape layer can be used to match data calculated and transmitted from the first input layer to the same format as data calculated and transmitted from the second input layer.

Bidirectional RNN is a type of RNN (Recurrent Neural Network). RNN is a neural network used to analyze data from the previous layer over time. The data input to the second input layer is a waveform in which the y value changes over time, which is the total time T, and RNN is used to analyze this waveform.

In the case of general RNN, it is used for analysis of time forward movement. The AI module (120) of the present invention can use a bidirectional RNN, and can precisely detect the peak of the auditory brainstem response test result by analyzing both the change in value according to time forward movement and the change in value according to time backward movement in the waveform.

Figure 4 is a diagram illustrating the internal structure of a bidirectional RNN and a time distributed dense layer.

Referring to Figure 4, the RNN is configured with a total number of LSTM (Long Short-Term Memory) cells T.

Each cell receives the y-value of the waveform transmitted from the second input layer. LSTM analyzes not only each y-value but also the results of other LSTMs within the same RNN.

The Time Distributed Dense Layer is similar to a general dense layer, but it differs from other dense layers in that it receives the output values from each LSTM cell at the rear of the RNN.

Unlike the general RNN, where only the result of the entire RNN is used as the input value of the dense layer, the time-distributed dense layer of the present invention has a dense layer positioned corresponding to each cell at the rear of all cells within the RNN.

The concatenate layer combines the result values transmitted from the reshaping layer and the time distribution dense layer of the third layer (123) into a single array.

The softmax dense layer uses the softmax activation function to calculate the result value of the dense layer. In addition, the softmax dense layer is the output layer of the AI module (120) of the present invention. The output value is a distribution of the probability that each time is a peak among the total time T.

The loss function calculates the difference between the output value calculated in the output layer and the actual peak value. The output value of the output layer is a probability distribution (1) with a total area of 1.

The actual peak value is a graph (2) in which the actual peak time t of the waveform used for learning is 1 and the rest are 0, and is the same as the mathematical expression 1 described above.

The loss function uses several calculation methods to calculate the values of (2) - (1).

In one embodiment of the present invention, it can be calculated using cosine similarity, and the output values of (2) - (1) can be passed back to each layer (121 to 126). This process is back propagation, and learning is performed by repeatedly calculating feedforward and back propagation in which data is sequentially passed from the first layer (121) to the sixth layer (126).

In the experimental example of the present invention, when each of the 1,684 learning data sets was subjected to forward and backward propagation calculations, it is called 1 epoch, and the AI module (120) performed learning for a total of 139 epochs. The performance of the AI module (120) was calculated using the MSE (Mean Squared Error), and as a result of calculating all test data, the average MSE was derived as 0.10515368729829788.

In other words, the value produced by the AI module appears to be the actual peak time t. The auditory brainstem response test determines whether there is hearing loss by determining the hearing threshold, and it is possible to determine whether there is hearing loss by calculating the peak time t value.

At the stage where the data waveform is input to the AI module, the y value of the waveform is normalized (0 to 1) through preprocessing, so the y value must be calculated again through the reverse process to determine the hearing threshold.

The embodiments described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and one or more software applications running on the OS. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone, but those skilled in the art will appreciate that the processing device may include multiple processing elements and/or multiple types of processing elements. For example, a processing device may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

The software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may independently or collectively command the processing device. The software and/or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal waves, for interpretation by the processing device or for providing instructions or data to the processing device. The software may also be distributed over network-connected computer systems, and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination. The program commands recorded on the medium may be those specially designed and configured for the embodiment, or may be those known to and usable by those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands, such as ROMs, RAMs, and flash memories. Examples of the program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiment, and vice versa.

Although the present invention has been described in detail with reference to preferred embodiments thereof, the present invention is not limited to the above-described embodiments, and it will be understood by those skilled in the art that various modifications or variations can be made therein without departing from the gist of the present invention as claimed in the following claims.

Although the present invention has been described in detail with respect to the described specific examples, it will be apparent to those skilled in the art that various modifications and variations are possible within the technical scope of the present invention, and it is natural that such modifications and variations fall within the scope of the appended patent claims.

100: Hearing test result interpretation device
110: Data preprocessing unit 120: AI module
121: 1st layer 122: 2nd layer
123: 3rd layer 124: 4th layer
125: 5th layer 126: 6th layer
130: Deaf Reading Department

Claims (12)

A data preprocessing unit that performs data preprocessing on the hearing test image of the auditory brainstem response test, normalizing the value of each waveform of the data to a value between 0 and 1 by dividing it by the maximum value of each waveform;
An AI module that extracts and analyzes the features of the waveform transmitted from the data preprocessing unit and calculates peak values and peak occurrence time information from the overall form of the waveform; and
A hearing test result interpretation device characterized by including a hearing loss interpretation unit that interprets whether there is hearing loss from the peak value of the waveform produced by the AI module and the time information at which the peak occurs.
In claim 1,
The above data preprocessing unit
A hearing test result interpretation device characterized by extracting and analyzing peaks using the entire shape of the waveform.
In claim 2,
The above data preprocessing unit
A hearing test result interpretation device characterized in that, if the y-value of a waveform has a category of (100, 200) through normalization, all y-values of the waveform are divided by the maximum value of 200 to have a category of (0.5, 1), and the values of each waveform are divided by the maximum value of each waveform to change them to values between 0 and 1.
In claim 1,
The above AI module
A hearing test result interpretation device characterized in that, when hearing test result data that has been preprocessed through the above data preprocessing unit is input, the device extracts and analyzes the region of interest within the input data and outputs the V wave (peak 5) within the waveform of the data.
In claim 1,
The machine learning technique used in the above AI module is
A hearing test result interpretation device characterized by using a DNN (Deep Neural Network) in which learning is executed through a multi-layered neural network, which is a NN described in an artificial neural network (ANN) with multiple layers of neural networks stacked together through deep learning. In claim 5,
The above AI module
A hearing test result reading device characterized by being configured with multiple layers so that each layer performs the role of calculating values input from the previous layer and sending them to the next layer.
In claim 6,
The above multiple layers
A hearing test result interpretation device characterized in that it is configured to include first to sixth layers, wherein the first layer includes a first input layer (Input1) and a second input layer (Input2), the second layer includes a dense layer and a bidirectional RNN, the third layer includes a reshape layer and a time distributed dense layer, the fourth layer includes a concatenate layer, the fifth layer includes a dense layer, and the sixth layer includes a softmax dense layer.
In claim 7,
The above first layer
A hearing test result interpretation device characterized in that each waveform of the auditory brainstem response test result is configured with two input layers based on the influence of the peak in the previously measured decibel waveform, and the peak value of the previously measured decibel waveform is input into the first input layer (Input1) in order to analyze the correlation between the peaks in each waveform, and the format of the input data is the same as the length of the total time (hereinafter T) of the waveform, and among them, the time t at which the peak is found is set to 1, the time at which the peak is not found is set to 0, and the preprocessed value of the current decibel waveform is input into the second input layer.
In claim 7,
The second layer above
A hearing test result interpretation device that uses the characteristics of data extracted from the previous layer to conduct analysis or judgment, and a bidirectional RNN is a type of RNN (Recurrent Neural Network) that is used to analyze data from the previous layer over time.
In claim 7,
The reshape layer of the third layer above is
A hearing test result reading device characterized by being used to change the format of input data without any additional computational process, the reformatting layer is used to match the data calculated and transmitted from the first input layer to the same format as the data calculated and transmitted from the second input layer, and the time distributed dense layer is similar to a general dense layer, but receives the values output from each LSTM cell at the rear of the RNN.
In claim 7,
The connection layer of the above fourth layer
A hearing test result reading device that performs the function of merging the result values transmitted from the reformation layer and the time distribution dense layer of the third layer into a single array.
In claim 7,
The software dense layer of the above 6th layer is
A hearing test result interpretation device characterized in that the output value of the dense layer is calculated using a softmax activation function, and the value output to the output layer of the AI module is a distribution regarding the probability that each time is a peak among the total time T.