CN118470440A - An early tumor recognition system based on deep learning and hyperspectral images - Google Patents

Abstract

The invention discloses a tumor early recognition system based on deep learning and hyperspectral images, and relates to the technical field of tumor early recognition. Comprising the following steps: the image dimension reduction module is used for acquiring a hyperspectral image and extracting spectral information in the hyperspectral image; the characteristic wave band selection module is used for carrying out differential analysis on the extracted spectrum information and determining a characteristic wave band according to an analysis result; the deep learning model module is used for carrying out preliminary tumor recognition on the spectrum information under the selected characteristic wave band by utilizing the tumor recognition model; and the integrated learning module is used for carrying out integrated learning on the preliminary tumor recognition result, optimizing a tumor recognition model according to the integrated learning result, and recognizing the hyperspectral image to be detected by adopting the optimized tumor recognition model to obtain a final tumor recognition result. The method combines the advantages of the spectrum information of the hyperspectral image and the integration of the prediction results of various deep learning models, and realizes the early and accurate identification of tumors.

Description

An early tumor recognition system based on deep learning and hyperspectral images

Technical Field

The present invention relates to the technical field of early tumor recognition, and in particular to an early tumor recognition system based on deep learning and hyperspectral images.

Background Art

The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

Cancer is one of the most common diseases in the world. In recent years, its morbidity and mortality rates have continued to rise, seriously endangering people's health. Cancer has become the "number one killer" of human health. At this stage, with the advancement of early cancer screening year by year, many problems have arisen in actual work. First, the early screening of tumors lacks comprehensiveness and standardization, that is, some screenings are only for specific types of cancer, and the screening cannot be comprehensive and thorough, which can easily miss the possibility of early detection of other cancers. In addition, accurate screening measures for common malignant tumors and a full-course health management system for early diagnosis and treatment have not yet been established. Secondly, the current means of early tumor identification mostly use tumor markers, which have the disadvantages of high cost and long waiting time, and this method requires labeling, which is easy to contaminate cells. Therefore, there is an urgent need for a comprehensive, rapid and label-free tumor early identification system to be applied in the field of early tumor identification.

Summary of the invention

In view of the shortcomings of the prior art, the purpose of the present invention is to provide an early tumor identification system based on deep learning and hyperspectral images, combining the advantages of the spectral information of hyperspectral images themselves and the integration of prediction results of multiple deep learning models to achieve accurate early tumor identification.

In order to achieve the above object, the present invention is implemented through the following technical solutions:

The first aspect of the present invention provides a system for early tumor recognition based on deep learning and hyperspectral images, comprising:

Image dimension reduction module, used to obtain hyperspectral images, extract spectral information in hyperspectral images, and convert three-dimensional hyperspectral images into one-dimensional data;

The characteristic band selection module is used to perform difference analysis on the extracted spectral information and determine the characteristic band according to the analysis results;

A deep learning model module is used to build a tumor recognition model and use the tumor recognition model to perform preliminary tumor recognition on the spectral information under the selected characteristic bands;

The integrated learning module is used to perform integrated learning on the preliminary tumor recognition results, optimize the tumor recognition model according to the integrated learning results, and use the optimized tumor recognition model to recognize the hyperspectral image to be tested to obtain the final tumor recognition result.

Furthermore, in the image dimension reduction module, the specific steps of extracting spectral information from the hyperspectral image are as follows:

Select some pixels for pixel sampling by adopting equal interval sampling method;

Extract the spectral information of the selected pixel.

Furthermore, the reduction factor is selected based on the proportion of the original image occupied by different tumor regions, thereby determining the sampling interval.

Furthermore, the length and width of the tumor region need to be considered in the process of shrinkage factor selection to determine the sampling intervals in the horizontal and vertical directions.

Furthermore, in the feature band selection module, the specific steps of performing difference analysis on the extracted spectral information are as follows:

The spectral information is extracted separately through a variety of feature selection methods;

The band with the largest spectral information difference in each feature selection method is selected as the candidate feature band;

The intersection of all selected candidate feature bands is selected as the appropriate feature band.

Furthermore, the multiple feature selection methods include two linear feature selection methods and two nonlinear feature selection methods.

Furthermore, in the deep learning model module, the tumor recognition model consists of two layers. The first layer model includes a one-dimensional convolutional neural network and a two-dimensional convolutional neural network, and the second layer model includes a self-attention residual network. The spectral information under the selected feature band passes through the one-dimensional convolutional neural network, the two-dimensional convolutional neural network and the self-attention residual network in sequence.

Furthermore, the specific steps of using the tumor recognition model to perform preliminary tumor recognition on the spectral information under the selected characteristic band are as follows:

A one-dimensional convolutional neural network is used to select a corresponding number of pixel point spectral information according to the characteristic band range to form a two-dimensional image;

Use a two-dimensional convolutional neural network to capture contextual information in a two-dimensional image and obtain a two-dimensional image with enhanced features;

The self-attention mechanism in the self-attention residual network is used to perform self-attention calculation on the two-dimensional image, and the preliminary tumor recognition result is obtained based on the calculation result.

Furthermore, the specific steps of the self-attention calculation are:

Map the two-dimensional image into three different spaces and perform self-attention calculations;

Assign different weights to different parts of the 2D image based on the attention score;

Different regions of the two-dimensional image are weighted according to the assigned weights.

Furthermore, in the integrated learning module, the specific steps of integrated learning of the preliminary tumor recognition results are as follows:

The preliminary tumor recognition results were used to train the one-dimensional convolutional neural network and the two-dimensional convolutional neural network respectively;

The two trained models are combined to form the first integrated model, and the prediction results of the two models are summarized as the first-level prediction results using the weighted voting method;

The self-attention residual network was trained using the same training set as the two-dimensional convolutional neural network. The prediction results of the trained self-attention residual network model were summarized using the weighted voting method. The two output results of the self-attention residual network model and the first integrated model were integrated to obtain the optimized tumor recognition model.

One or more of the above technical solutions have the following beneficial effects:

The present invention discloses an early tumor recognition system based on deep learning and hyperspectral images, which realizes accurate early tumor recognition by accurately extracting spectral information from hyperspectral images and constructing a double-layer deep learning network. The present invention further optimizes the tumor recognition model composed of the double-layer deep learning network by means of integrated learning, thereby improving the accuracy of tumor recognition and overcoming the problems of high cost, long time and easy contamination caused by tumor marking in the prior art.

Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

FIG1 is a framework diagram of the recognition process of the early tumor recognition system based on deep learning and hyperspectral images in the present invention.

DETAILED DESCRIPTION

It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

It should be noted that in the embodiments of the present invention, data related to hyperspectral images are involved. When the above embodiments of the present invention are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "include" and/or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and/or their combinations;

Embodiment 1:

Embodiment 1 of the present invention provides an early tumor recognition system based on deep learning and hyperspectral images, including an image dimension reduction module, a feature band selection module, a deep learning model module and an integrated learning module.

As shown in Figure 1, the image dimension reduction module is used to obtain hyperspectral images, extract spectral information in hyperspectral images, and convert three-dimensional hyperspectral images into one-dimensional data.

The characteristic band selection module is used to perform difference analysis on the extracted spectral information and determine the characteristic band according to the analysis results.

The deep learning model module is used to build a tumor recognition model and use the tumor recognition model to perform preliminary tumor recognition on the spectral information under the selected characteristic bands.

The integrated learning module is used to perform integrated learning on the preliminary tumor recognition results, optimize the tumor recognition model according to the integrated learning results, and use the optimized tumor recognition model to recognize the hyperspectral image to be tested to obtain the final tumor recognition result.

In a specific implementation, in the image dimension reduction module, the specific steps of extracting spectral information from the hyperspectral image are:

The pixel points are selected by equal-interval sampling, where the reduction factor is selected based on the proportion of the original image occupied by different tumor areas to determine the sampling interval. The length and width of the tumor area should be considered in the selection of the reduction factor to determine the sampling interval in the horizontal and vertical directions.

Specifically, let the size of the original image be W*H, and the number of pixels occupied by the tumor area be N. Select appropriate width and length reduction factors based on different tumors to ensure that the total number of pixels selected in the tumor area under the reduction factor is not less than 3N/4, set as k1 and k2 respectively, then the sampling interval is: W/k1, W/k2. That is, take a pixel every W/k1 in the horizontal direction of the original image and every W/k2 in the vertical direction to ensure that the taken pixel represents the characteristics of the original image. Extract the spectral information of the selected pixel and select the characteristic band.

In a specific implementation, in the characteristic band selection module, the specific steps of performing difference analysis on the extracted spectral information are as follows:

The features of the spectral information are extracted respectively by using a variety of feature selection methods, wherein the multiple feature selection methods include two linear feature selection methods and two nonlinear feature selection methods.

The band with the largest spectral information difference in each feature selection method is selected as the candidate feature band;

The intersection of all selected candidate feature bands is selected as the appropriate feature band.

This embodiment selects two traditional linear feature selection methods: principal component analysis method and t-test analysis method, and selects two new nonlinear feature selection methods: kernel principal component analysis method and genetic algorithm to select feature bands. In each selection method, the band with the largest difference in spectral information is selected as the feature band. For example, under the t-test, the band with a p value of <0.05 is selected as the most suitable feature band under this test method. The intersection of the four feature band results of the two linear methods and the two nonlinear methods is selected as the most suitable feature band, and the spectral information under the feature band is sent as input to the subsequent deep learning model to improve the accuracy of the deep learning model.

In a specific embodiment, in the deep learning model module, the tumor recognition model is composed of two layers, the first layer model includes a one-dimensional convolutional neural network and a two-dimensional convolutional neural network, the second layer model includes a self-attention residual network, and the one-dimensional spectral information under the selected feature band passes through the one-dimensional convolutional neural network; the one-dimensional spectral information is processed into a two-dimensional image and then sent to the two-dimensional convolutional neural network and the self-attention residual network, and then integrated learning is performed subsequently to improve robustness and reduce the occurrence of overfitting.

The specific steps of using the tumor recognition model to perform preliminary tumor recognition on the spectral information under the selected characteristic band are as follows:

A one-dimensional convolutional neural network is used to select the corresponding number of pixel spectral information according to the characteristic band range to arrange and form a two-dimensional image. Specifically, the spectral information of the pixel points extracted in the image dimension reduction module within the selected characteristic band range is expanded to the image grayscale range, that is, the maximum value in the spectral information is taken as I _max , then for any spectral value I , its expanded value P = I *255/ I _max , and the extracted pixel points are arranged to form a two-dimensional image, such as: if a total of 1024 pixels are extracted, and the selected characteristic band range is 400-700nm, then the size of the arranged two-dimensional image is 1024×300.

A two-dimensional convolutional neural network is used to capture the contextual information in a two-dimensional image and obtain a two-dimensional image with enhanced features. Specifically, the contextual information in a convolutional neural network can be understood as the feature information extracted after the convolution kernel slides on the image to perform a convolution operation. A normal convolution kernel is a connected n×n kernel, while a dilated convolution adds gaps (holes) between the elements of the convolution kernel. The specific number of gaps is controlled by the parameter expansion rate. The two-dimensional convolutional neural network introduces dilated convolution in the convolution layer of the network structure, so that the receptive field has a variable scale, and a larger receptive field is obtained without increasing the parameters and computational complexity of the neural network, thereby better capturing the contextual information in the spectral image.

The self-attention mechanism in the self-attention residual network is used to perform self-attention calculation on the two-dimensional image, and the preliminary tumor recognition result is obtained based on the calculation result.

Among them, the specific steps of self-attention calculation are:

The two-dimensional image is mapped to three different spaces and self-attention calculation is performed; the three spaces are Q, K, and V spaces when the self-attention mechanism is calculated.

Different weights are assigned to different parts of the 2D image according to the attention scores.

Different regions of the two-dimensional image are weighted according to the assigned weights to improve the accuracy of model classification.

Specifically, by applying the self-attention mechanism, the self-attention scores of different positions on the two-dimensional image are obtained, and the median of the scores obtained at all positions is selected as the threshold T. When the model is classified, the features in the area with a score less than T/2 are discarded, and the feature weights of the areas with scores between T/2 and 3T/2 are set to 1, and the feature weights of the areas with scores greater than 3T/2 are set to 2, so that the features used by the model are more representative.

In a specific embodiment, in the integrated learning module, a hierarchical integrated learning method strategy is adopted to hierarchically combine the prediction results of the basic model through a series of integrated modules. Each integrated module can be integrated according to different rules or weights, and the final prediction result is obtained by weighted average of multiple levels. Specifically, the specific steps of integrated learning of the preliminary tumor identification results are:

The preliminary tumor identification results were used to divide the training set and the validation set in a ratio of 7:3. The design level was 2 layers. The first layer was to train the one-dimensional convolutional neural network and the two-dimensional convolutional neural network respectively. The two trained models were combined to form the first integrated model. The weighted voting method was used to summarize the prediction results of the two models as the first-layer prediction results.

The specific steps of using the weighted voting method to summarize the prediction results of the two models as the first-layer prediction results are as follows: the first integrated model includes 6 one-dimensional convolutional neural networks and 4 two-dimensional convolutional neural networks as base learners, and the result of each base learner is used as one vote. The weight of the one-dimensional convolutional neural network is 1, and the weight of the two-dimensional convolutional neural network is set to 2; the classification votes must be multiplied by their own weights, and finally the weighted votes of each category are summed up, and the two results corresponding to the maximum vote value and the second largest vote value are output as the output A and output B of the first integrated model respectively. The initial weight of output A is set to 2, and the initial weight of output B is set to 1, and integrated learning is performed again with the self-attention residual network.

The second level is to train the self-attention residual network using the same training set as the two-dimensional convolutional neural network, and use the weighted voting method to summarize the prediction results of the trained self-attention residual network model. The two output results of the self-attention residual network model and the first integrated model are integrated to obtain the second integrated model as the optimized tumor recognition model.

The specific steps of summarizing the prediction results of the trained self-attention residual network model using the weighted voting method are as follows: the second integrated model includes the two output results A and B of the first integrated model, and three self-attention residual network models as base learners. The weight of the self-attention residual network is set to 1. If its output is the same as the output A and B of the first integrated model, the votes are attributed to A or B, and finally the result with the highest number of votes is selected as the final result of the overall model.

The validation set data was then used to evaluate the performance of the two-layer integrated model. After completing the training and evaluation of the model, the final tumor recognition model was applied to unknown data for prediction.

Although the specific implementation modes of the present invention are described above in conjunction with the accompanying drawings, this does not limit the protection scope of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the protection scope of the present invention.

Claims (10)

1. A tumor early recognition system based on deep learning and hyperspectral images, comprising:

the image dimension reduction module is used for acquiring a hyperspectral image, extracting spectral information in the hyperspectral image and converting the three-dimensional hyperspectral image into one-dimensional data;

The characteristic wave band selection module is used for carrying out differential analysis on the extracted spectrum information and determining a characteristic wave band according to an analysis result;

The deep learning model module is used for constructing a tumor recognition model and carrying out preliminary tumor recognition on spectrum information under the selected characteristic wave band by utilizing the tumor recognition model;

and the integrated learning module is used for carrying out integrated learning on the preliminary tumor recognition result, optimizing a tumor recognition model according to the integrated learning result, and recognizing the hyperspectral image to be detected by adopting the optimized tumor recognition model to obtain a final tumor recognition result.

2. The tumor early recognition system based on deep learning and hyperspectral image as claimed in claim 1, wherein the specific steps of extracting spectral information in the hyperspectral image in the image dimension reduction module are as follows:

selecting part of pixel points by adopting an equidistant sampling mode to sample the pixel points;

spectral information of the selected pixel point is extracted.

3. The deep learning and hyperspectral image based tumor early recognition system of claim 2 wherein the sampling interval is determined by selecting a reduction factor based on the proportion of the artwork occupied by the different tumor regions.

4. The early tumor recognition system based on deep learning and hyperspectral image as claimed in claim 3, wherein the reduction factor is selected by considering the length and width of the tumor region, so as to determine the sampling interval in the horizontal direction and the vertical direction.

5. The tumor early recognition system based on deep learning and hyperspectral image as claimed in claim 1, wherein the characteristic wave band selection module performs the specific steps of performing differential analysis on the extracted spectral information:

the spectrum information is subjected to characteristic extraction through a plurality of characteristic selection methods;

selecting a wave band with the largest spectrum information difference in each characteristic selection method as a candidate characteristic wave band;

and selecting the intersection of all the selected candidate characteristic wave bands as the proper characteristic wave band.

6. The deep learning and hyperspectral image based tumor early recognition system of claim 5, wherein the plurality of feature selection methods includes two linear feature selection methods and two nonlinear feature selection methods.

7. The tumor early recognition system based on deep learning and hyperspectral image as claimed in claim 1, wherein in the deep learning model module, the tumor recognition model is composed of two layers, the first layer model comprises a one-dimensional convolutional neural network and a two-dimensional convolutional neural network, the second layer model comprises a self-attention residual network, and the spectral information under the selected characteristic wave band sequentially passes through the one-dimensional convolutional neural network, the two-dimensional convolutional neural network and the self-attention residual network.

8. The tumor early recognition system based on deep learning and hyperspectral image as claimed in claim 7, wherein the specific steps of performing the preliminary recognition of the tumor on the spectral information under the selected characteristic wave band by using the tumor recognition model are as follows:

Selecting a corresponding number of pixel point spectrum information according to the characteristic wave band range by utilizing a one-dimensional convolutional neural network, and arranging to form a two-dimensional image;

capturing context information in the two-dimensional image by using a two-dimensional convolutional neural network to obtain a two-dimensional image with enhanced characteristics;

and performing self-attention calculation on the two-dimensional image by using a self-attention mechanism in the self-attention residual error network, and obtaining a tumor primary identification result according to a calculation result.

9. The tumor early recognition system based on deep learning and hyperspectral image as claimed in claim 8, wherein the specific steps of the self-attention calculation are:

Mapping the two-dimensional image to three different spaces and performing self-attention calculation;

assigning different weights to different portions of the two-dimensional image according to the attention score;

and weighting different areas of the two-dimensional image according to the assigned weights.

10. The tumor early recognition system based on deep learning and hyperspectral image as claimed in claim 9, wherein the integrated learning module performs the specific steps of integrated learning on the preliminary tumor recognition result:

training a one-dimensional convolutional neural network and a two-dimensional convolutional neural network respectively by using a tumor preliminary identification result;

combining the two trained models to form a first integrated model, and summarizing the prediction results of the two models by adopting a weighted voting method to serve as a first layer prediction result;

And training the self-attention residual error network by using the training set which is the same as the two-dimensional convolutional neural network, summarizing the predicted result of the trained self-attention residual error network model by adopting a weighted voting method, and integrating the two output results of the self-attention residual error network model and the first integrated model to obtain the optimized tumor recognition model.