CN107590804A - Screen Image Quality Evaluation Method Based on Channel Feature and Convolutional Neural Network - Google Patents

Abstract

The invention discloses a screen image quality evaluation method based on channel characteristics and a convolutional neural network, which extracts ten channel characteristic graphs of a distorted screen image to be evaluated and acquires respective normalized images; acquiring respective normalized images of ten channel characteristic diagrams of each distorted screen image in a training set in the same way, training respective subjective score values of all distorted screen images in the training set and the normalized images of the ten channel characteristic diagrams by using a convolutional neural network to obtain an optimal weight vector and an optimal bias term, further constructing to obtain a convolutional neural network regression training model, testing the normalized images corresponding to the distorted screen images to be evaluated according to the convolutional neural network regression training model, and obtaining objective quality evaluation predicted values of the distorted screen images to be evaluated; the method has the advantages that the influence of various characteristics of the screen image on the visual quality can be fully considered, so that the correlation between objective evaluation results and subjective perception can be improved.

Description

Screen Image Quality Evaluation Method Based on Channel Feature and Convolutional Neural Network

technical field

The invention relates to a no-reference image quality evaluation method, in particular to a screen image quality evaluation method based on channel features and a convolutional neural network.

Background technique

With the rapid development of the image processing industry, image quality evaluation has become an increasingly important component, and people's requirements for image quality are also increasing. Due to the process of image acquisition, storage, transmission and display, there are often different degrees of distortion, such as image blur, video terminal image distortion, image quality in the system is not up to standard, etc. Therefore, it is very important to establish an effective image quality evaluation mechanism For example, it can be used for performance comparison and parameter selection of various algorithms in the process of image denoising and image fusion; it can be used to guide the entire image transmission process and evaluate system performance in the field of image coding and communication; it can be used in the field of video surveillance. Detect the quality of video images, discover and adjust the quality of video surveillance in time.

Image quality evaluation methods can be roughly divided into two categories, namely subjective evaluation methods and objective evaluation methods. The subjective evaluation method is to directly judge the quality of the image through the human eye, which is controlled by subjective consciousness; the characteristics of the evaluation result are: the higher the comfort of the image, the higher the scoring result; it is a relatively reliable evaluation method, but time consuming. The objective evaluation method is the image quality index obtained by the machine according to a certain algorithm. It is mainly divided into three evaluation methods, namely, the full reference image quality evaluation method, the semi-reference image quality evaluation method and the no-reference image quality evaluation method. The current research is the most However, since the corresponding original images cannot be obtained in most applications, the research on no-reference image quality assessment methods is more practical.

No-reference image quality evaluation methods can be divided into specific distortion evaluation methods and general evaluation methods. Specific distortion evaluation methods can only evaluate images of a certain type of distortion, such as JPEG, JPEG2K and Gblur distortion, etc., and cannot evaluate other distortions. Types of images and images processed by multiple processing techniques for quality evaluation; the general evaluation method can simultaneously evaluate the quality of images with multiple types of distortion.

The existing general-purpose no-reference image quality evaluation methods are mainly aimed at general images, but there are relatively few studies on special images (such as screen images). Since screen images contain text, graphics, and images, etc., the general It is more challenging to perform quality assessment without reference image quality assessment methods.

Contents of the invention

The technical problem to be solved by the present invention is to provide a screen image quality evaluation method based on channel features and convolutional neural network, which can fully consider the impact of various characteristics of screen images on visual quality, thereby improving the objective evaluation results and subjective evaluation results. Correlation between perceptions.

The technical solution adopted by the present invention to solve the above-mentioned technical problems is: a method for evaluating the quality of screen images based on channel features and convolutional neural networks, characterized in that it comprises the following steps:

Step 1: Let {I _d (i, j)} represent the distorted screen image to be evaluated, where, 1≤i≤W, 1≤j≤H, W represents the width of {I _d (i,j)}, H Represents the height of {I _d (i, j)}, I _d (i, j) represents the pixel value of the pixel whose coordinate position is (i, j) in {I _d (i, j)};

Step 2: Use the lumped feature channel method to perform feature extraction on {I _d (i, j)}, and obtain ten channel feature maps of {I _d (i, j)}, which are L channel feature map and U channel feature Figure, V channel feature map, gradient amplitude channel feature map, first directional gradient histogram channel feature map, second directional gradient histogram channel feature map, third directional gradient histogram channel feature map, fourth The directional gradient histogram channel feature map, the 5th directional gradient histogram channel feature map, and the 6th directional gradient histogram channel feature map, correspondingly recorded as {L _d (m,n)}, {U _d (m,n) )}, {V _d (m,n)}, {G _d,0 (m,n)}, {G _d,1 (m,n)}, {G _d,2 (m,n)}, { G _d,3 (m,n)}, {G _d,4 (m,n)}, {G _d,5 (m,n)}, {G _d,6 (m,n)}, where 1 ≤m≤M, 1≤n≤N, symbol is the symbol of the rounding down operation, M represents the width of each channel feature map of {I _d (i,j)}, N represents the height of each channel feature map of {I _d (i,j)}, L _d (m,n) represents the pixel value of the pixel whose coordinate position is (m,n) in {L _d (m,n)}, and U _d (m,n) represents the coordinates in {U _d (m,n)} The pixel value of the pixel point whose position is (m, n), V _d (m, n) represents the pixel value of the pixel point whose coordinate position is (m, n) in {V _d (m, n)}, G _{d, 0} (m,n) represents the pixel value of the pixel point whose coordinate position is (m,n) in {G _d,0 (m,n)}, and G _d,1 (m,n) represents {G _d,1 ( The pixel value of the pixel whose coordinate position is (m,n) in m,n)}, G _d,2 (m,n) means that the coordinate position in {G _d,2 (m,n)} is (m,n) ), G _d,3 (m,n) represents the pixel value of the pixel whose coordinate position is (m,n) in {G _d,3 (m,n)}, G _d,4 ( m,n) represents the pixel value of the pixel point whose coordinate position is (m,n) in {G _d,4 (m,n)}, G _d,5 (m,n) represents {G _d,5 (m, The pixel value of the pixel whose coordinate position is (m,n) in n)}, G _d,6 (m,n) represents the pixel value of the coordinate position (m,n) in {G _d,6 (m,n)} The pixel value of the pixel point;

Step 3: Get {L _d (m,n)}, {U _d (m,n)}, {V _d (m,n)}, {G _d,0 (m,n)}, {G _{d, 1} (m,n)}, {G _d,2 (m,n)}, {G _d,3 (m,n)}, {G _d,4 (m,n)}, {G _d,5 ( The normalized images of m,n)} and {G _d,6 (m,n)} are correspondingly denoted as in, express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose middle coordinate position is (m,n);

Step 4: Using P original undistorted screen images, set up a set of distorted screen images under different distortion types and different degrees of distortion, and use the set of distorted screen images as a training set. The training set includes multiple distorted screen images; then use The subjective quality evaluation method evaluates the subjective score value of each piece of distorted screen image in the training set, and records the subjective score value of the jth piece of distorted screen image in the training set as DMOS _j ; Obtain the respective normalized images of the ten channel feature maps of each distorted screen image in the training set by means of , and record the respective normalized images of the ten channel feature maps of the jth distorted screen image in the training set as Among them, P>1, 1≤j≤K', K' indicates the total number of distorted screen images contained in the training set, 0≤DMOS _j ≤100, express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose middle coordinate position is (m,n);

Step 5: Use the convolutional neural network to train the respective subjective score values of all the distorted screen images in the training set and the normalized images of the respective ten channel feature maps, so that the relationship between the regression function value obtained after training and the subjective score value The error between is the smallest, and the optimal weight vector W ^opt and the optimal bias item b ^opt are obtained by fitting; then the convolutional neural network regression training model is obtained by using the W ^opt construction; and then according to the convolutional neural network regression training model, right Carry out the test and predict the objective quality evaluation prediction value of {I _d (i, j)}, denoted as Q, Q=f(x), Among them, Q is a function of x, f() is a function representation, x is an input variable, and x represents is the transpose vector of W ^opt , Convolution function for x.

The concrete process of described step three is:

Obtain the local average value image of {L _d (m,n)}, denoted as {μ ₁ (m,n)}; and obtain the local variance image of {L _d (m,n)}, denoted as {σ ₁ (m,n)}; then according to {L _d (m,n)}, {μ ₁ (m,n)}, {σ ₁ (m,n)}, get {L _d (m,n)} The normalized image of Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₁ (m,n) represents the pixel value of the pixel whose coordinate position is (m,n) in {μ ₁ (m,n)}, and σ ₁ (m,n) represents {σ ₁ (m,n) )} in the pixel value of the pixel whose coordinate position is (m, n);

Obtain the local average value image of {U _d (m,n)}, denoted as {μ ₂ (m,n)}; and obtain the local variance image of {U _d (m,n)}, denoted as {σ ₂ (m,n)}; then according to {U _d (m,n)}, {μ ₂ (m,n)}, {σ ₂ (m,n)}, get {U _d (m,n)} The normalized image of Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₂ (m,n) represents the pixel value of the pixel whose coordinate position is (m,n) in {μ ₂ (m,n)}, and σ ₂ (m,n) represents {σ ₂ (m,n) )} in the pixel value of the pixel whose coordinate position is (m, n);

Obtain the local average value image of {V _d (m,n)}, denoted as {μ ₃ (m,n)}; and obtain the local variance image of {V _d (m,n)}, denoted as {σ ₃ (m,n)}; then according to {V _d (m,n)}, {μ ₃ (m,n)}, {σ ₃ (m,n)}, get {V _d (m,n)} The normalized image of Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₃ (m,n) represents the pixel value of the pixel point whose coordinate position is (m,n) in {μ ₃ (m,n)}, and σ ₃ (m,n) represents {σ ₃ (m,n) )} in the pixel value of the pixel whose coordinate position is (m, n);

Obtain the local average value image of {G _d,0 (m,n)}, denoted as {μ ₄ (m,n)}; and obtain the local variance image of {G _d,0 (m,n)}, Denote it as {σ _{4 (} m,n) _{} ;} then get {G _{d ,0} (m,n)} normalized image Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₄ (m,n) represents the pixel value of the pixel point whose coordinate position is (m,n) in {μ ₄ (m,n)}, and σ ₄ (m,n) represents {σ ₄ (m,n) )} in the pixel value of the pixel whose coordinate position is (m, n);

Obtain the local average image of {G _d,1 (m,n)}, denoted as {μ ₅ (m,n)}; and obtain the local variance image of {G _d,1 (m,n)}, Recorded as {σ ₅ (m,n)}; then according to {G _d,1 (m,n)}, {μ ₅ (m,n)}, {σ ₅ (m,n)}, get {G _{d ,1} (m,n)} normalized image Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₅ (m,n) represents the pixel value of the pixel point whose coordinate position is (m,n) in {μ ₅ (m,n)}, and σ ₅ (m,n) represents {σ ₅ (m,n) )} in the pixel value of the pixel whose coordinate position is (m, n);

Obtain the local average value image of {G _d,2 (m,n)}, denoted as {μ ₆ (m,n)}; and obtain the local variance image of {G _d,2 (m,n)}, Denote it as {σ ₆ (m,n)}; then according to {G _d,2 (m,n)}, {μ ₆ (m,n)}, {σ ₆ (m,n) _} , get _,2 (m,n)} normalized image Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₆ (m,n) represents the pixel value of the pixel point whose coordinate position is (m,n) in {μ ₆ (m,n)}, and σ ₆ (m,n) represents {σ ₆ (m,n) )} in the pixel value of the pixel whose coordinate position is (m, n);

Obtain the local average image of {G _d,3 (m,n)}, denoted as {μ ₇ (m,n)}; and obtain the local variance image of {G _d,3 (m,n)}, _Denote it as {σ ₇ (m,n) _{} ;} then get {G _{d ,3} (m,n)} normalized image Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₇ (m,n) represents the pixel value of the pixel point whose coordinate position is (m,n) in {μ ₇ (m,n)}, and σ ₇ (m,n) represents {σ ₇ (m,n) )} in the pixel value of the pixel whose coordinate position is (m, n);

Obtain the local average image of {G _d,4 (m,n)}, denoted as {μ ₈ (m,n)}; and obtain the local variance image of {G _d,4 (m,n)}, _Denote it as { _{σ 8} (m,n)} _; then get {G _{d ,4} (m,n)} normalized image Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₈ (m,n) represents the pixel value of the pixel whose coordinate position is (m,n) in {μ ₈ (m,n)}, and σ ₈ (m,n) represents {σ ₈ (m,n) )} in the pixel value of the pixel whose coordinate position is (m, n);

Obtain the local average value image of {G _d,5 (m,n)}, denoted as {μ ₉ (m,n)}; and obtain the local variance image of {G _d,5 (m,n)}, _Denote it as { _{σ 9} (m,n)} _; then get {G _{d ,5} (m,n)} normalized image Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₉ (m,n) represents the pixel value of the pixel whose coordinate position is (m,n) in {μ ₉ (m,n)}, and σ ₉ (m,n) represents {σ ₉ (m,n) )} in the pixel value of the pixel whose coordinate position is (m, n);

Obtain the local average value image of {G _d,6 (m,n)}, denoted as {μ ₁₀ (m,n)}; and obtain the local variance image of {G _d,6 (m,n)}, _Denote it as {σ ₁₀ (m,n) _{} ;} then get {G _{d ,6} (m,n)} normalized image Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₁₀ (m,n) represents the pixel value of the pixel point whose coordinate position is (m,n) in {μ ₁₀ (m,n)}, and σ ₁₀ (m,n) represents {σ ₁₀ (m,n) )} The pixel value of the pixel whose coordinate position is (m,n).

Compared with the prior art, the present invention has the advantages of:

1) The inventive method carries out feature extraction to the distorted screen image, obtains ten channel feature maps of the distorted screen image, and then obtains the normalized image of the ten channel feature maps of the distorted screen image, and separates all distorted screen images in the training set Input the normalized images of the ten channel feature maps of the convolutional neural network into the convolutional neural network for training, and obtain the convolutional neural network regression training model; then input the normalized images of the ten channel feature maps of the distorted screen image to be evaluated into the In the convolutional neural network regression training model, the objective quality evaluation prediction value of the distorted screen image to be evaluated is predicted, and since multiple features of the distorted screen image are combined, these features can describe the distorted screen image more accurately, that is, the present invention The method takes full account of the influence of various characteristics of the screen image on the visual quality, so that the correlation between the objective evaluation results and the subjective perception can be effectively improved.

2) The method of the present invention adopts a supervised learning method based on a convolutional neural network, that is, a supervised feature learning method that simulates the processing of human visual characteristics for training and testing, so that the method of the present invention can fully take into account the sensitivity of visual perception characteristics to image distortion .

3) The method of the present invention adopts the convolutional neural network conforming to the mechanism characteristics of the human brain to predict the objective quality evaluation prediction value of the distorted screen image to be evaluated, which can effectively improve the correlation between the objective evaluation result and the subjective perception.

Description of drawings

Fig. 1 is the overall realization block diagram of the inventive method;

Figure 2 is a distorted screen image;

Fig. 3 is the L channel feature map of the distorted screen image shown in Fig. 2;

Fig. 4 is the U channel feature map of the distorted screen image shown in Fig. 2;

Fig. 5 is the V channel feature map of the distorted screen image shown in Fig. 2;

Fig. 6 is the gradient amplitude channel feature map of the distorted screen image shown in Fig. 2;

Fig. 7 is the first directional gradient histogram channel feature map of the distorted screen image shown in Fig. 2;

Fig. 8 is the second directional gradient histogram channel feature map of the distorted screen image shown in Fig. 2;

Fig. 9 is the 3rd directional gradient histogram channel feature map of the distorted screen image shown in Fig. 2;

Fig. 10 is the 4th directional gradient histogram channel feature map of the distorted screen image shown in Fig. 2;

Fig. 11 is the 5th directional gradient histogram channel feature map of the distorted screen image shown in Fig. 2;

FIG. 12 is a feature map of the sixth directional gradient histogram channel of the distorted screen image shown in FIG. 2 .

detailed description

The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments.

A method for evaluating screen image quality based on channel features and convolutional neural networks proposed by the present invention, its overall implementation block diagram is shown in Figure 1, and it includes the following steps:

Step 1: Let {I _d (i, j)} represent the distorted screen image to be evaluated, where, 1≤i≤W, 1≤j≤H, W represents the width of {I _d (i,j)}, H Indicates the height of {I _d (i, j)}, and I _d (i, j) indicates the pixel value of the pixel at the coordinate position (i, j) in {I _d (i, j)}.

Step 2: Use the existing aggregated feature channel method (Aggregate Channel Features, ACF) to perform feature extraction on {I _d (i, j)}, and obtain ten channel feature maps of {I _d (i, j)}, They are L channel feature map, U channel feature map, V channel feature map, gradient magnitude channel feature map, gradient histogram channel feature map in the first direction, gradient histogram channel feature map in the second direction, and third direction The gradient histogram channel feature map, the fourth directional gradient histogram channel feature map, the fifth directional gradient histogram channel feature map, and the sixth directional gradient histogram channel feature map, correspondingly denoted as {L _d (m,n )}, {U _d (m,n)}, {V _d (m,n)}, {G _d,0 (m,n)}, {G _d,1 (m,n)}, {G _{d ,2} (m,n)}, {G _d,3 (m,n)}, {G _d,4 (m,n)}, {G _d,5 (m,n)}, {G _d,6 (m,n)}, where, 1≤m≤M, 1≤n≤N, symbol is the symbol of the rounding down operation, M represents the width of each channel feature map of {I _d (i,j)}, N represents the height of each channel feature map of {I _d (i,j)}, L _d (m,n) represents the pixel value of the pixel whose coordinate position is (m,n) in {L _d (m,n)}, and U _d (m,n) represents the coordinates in {U _d (m,n)} The pixel value of the pixel point whose position is (m, n), V _d (m, n) represents the pixel value of the pixel point whose coordinate position is (m, n) in {V _d (m, n)}, G _{d, 0} (m,n) represents the pixel value of the pixel point whose coordinate position is (m,n) in {G _d,0 (m,n)}, and G _d,1 (m,n) represents {G _d,1 ( The pixel value of the pixel whose coordinate position is (m,n) in m,n)}, G _d,2 (m,n) means that the coordinate position in {G _d,2 (m,n)} is (m,n) ), G _d,3 (m,n) represents the pixel value of the pixel whose coordinate position is (m,n) in {G _d,3 (m,n)}, G _d,4 ( m,n) represents the pixel value of the pixel point whose coordinate position is (m,n) in {G _d,4 (m,n)}, G _d,5 (m,n) represents {G _d,5 (m, The pixel value of the pixel whose coordinate position is (m,n) in n)}, G _d,6 (m,n) represents the pixel value of the coordinate position (m,n) in {G _d,6 (m,n)} The pixel value of the pixel point.

Figure 2 shows a distorted screen image, and Figures 3 to 12 correspond to the L channel feature map, U channel feature map, V channel feature map, and gradient magnitude channel feature map of the distorted screen image shown in Figure 2 , the first directional gradient histogram channel feature map, the second directional gradient histogram channel feature map, the third directional gradient histogram channel feature map, the fourth directional gradient histogram channel feature map, and the fifth directional gradient Histogram channel feature map, the sixth directional gradient histogram channel feature map.

Step 3: Use the existing average calculation method to obtain the local average value image of {L _d (m,n)}, denoted as {μ ₁ (m,n)}; and use the existing variance to obtain method to obtain the local variance image of {L _d (m,n)}, denoted as {σ ₁ (m,n)}; then according to {L _d (m,n)}, {μ ₁ (m,n) }, {σ ₁ (m,n)}, get the normalized image of {L _d (m,n)}, denoted as Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₁ (m,n) represents the pixel value of the pixel whose coordinate position is (m,n) in {μ ₁ (m,n)}, and σ ₁ (m,n) represents {σ ₁ (m,n) )} The pixel value of the pixel whose coordinate position is (m,n).

Using the existing mean value calculation method, obtain the local mean value image of {U _d (m,n)}, denoted as {μ ₂ (m,n)}; and use the existing variance calculation method, find Get the local variance image of {U _d (m,n)}, denoted as {σ ₂ (m,n)}; then according to {U _d (m,n)}, {μ ₂ (m,n)}, { σ ₂ (m,n)}, get the normalized image of {U _d (m,n)}, denoted as Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₂ (m,n) represents the pixel value of the pixel whose coordinate position is (m,n) in {μ ₂ (m,n)}, and σ ₂ (m,n) represents {σ ₂ (m,n) )} The pixel value of the pixel whose coordinate position is (m,n).

Using the existing average calculation method, obtain the local average value image of {V _d (m,n)}, denoted as {μ ₃ (m,n)}; and use the existing variance calculation method, to obtain Get the local variance image of {V _d (m,n)}, denoted as {σ ₃ (m,n)}; then according to {V _d (m,n)}, {μ ₃ (m,n)}, { σ ₃ (m,n)}, get the normalized image of {V _d (m,n)}, denoted as Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₃ (m,n) represents the pixel value of the pixel point whose coordinate position is (m,n) in {μ ₃ (m,n)}, and σ ₃ (m,n) represents {σ ₃ (m,n) )} The pixel value of the pixel whose coordinate position is (m,n).

Using the existing average calculation method, obtain the local average value image of {G _d,0 (m,n)}, denoted as {μ ₄ (m,n)}; and use the existing variance calculation method , get the local variance image of {G _d,0 (m,n)}, denoted as {σ ₄ (m,n)}; then according to {G _d,0 (m,n)}, {μ ₄ (m ,n)}, {σ ₄ (m,n)}, get the normalized image of {G _d,0 (m,n)}, denoted as Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₄ (m,n) represents the pixel value of the pixel point whose coordinate position is (m,n) in {μ ₄ (m,n)}, and σ ₄ (m,n) represents {σ ₄ (m,n) )} The pixel value of the pixel whose coordinate position is (m,n).

Using the existing average value calculation method, obtain the local average value image of {G _d,1 (m,n)}, denoted as {μ ₅ (m,n)}; and use the existing variance calculation method , get the local variance image of {G _d,1 (m,n)}, denoted as {σ ₅ (m,n)}; then according to {G _d,1 (m,n)}, {μ ₅ (m ,n)}, {σ ₅ (m,n)}, get the normalized image of {G _d,1 (m,n)}, denoted as Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₅ (m,n) represents the pixel value of the pixel point whose coordinate position is (m,n) in {μ ₅ (m,n)}, and σ ₅ (m,n) represents {σ ₅ (m,n) )} The pixel value of the pixel whose coordinate position is (m,n).

Using the existing average calculation method, obtain the local average value image of {G _d,2 (m,n)}, denoted as {μ ₆ (m,n)}; and use the existing variance calculation method , get the local variance image of {G _d,2 (m,n)}, denoted as {σ ₆ (m,n)}; then according to {G _d,2 (m,n)}, {μ ₆ (m ,n)}, {σ ₆ (m,n)}, get the normalized image of {G _d,2 (m,n)}, denoted as Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₆ (m,n) represents the pixel value of the pixel point whose coordinate position is (m,n) in {μ ₆ (m,n)}, and σ ₆ (m,n) represents {σ ₆ (m,n) )} The pixel value of the pixel whose coordinate position is (m,n).

Using the existing average calculation method, obtain the local average value image of {G _d,3 (m,n)}, denoted as {μ ₇ (m,n)}; and use the existing variance calculation method , get the local variance image of {G _d,3 (m,n)}, denoted as {σ ₇ (m,n)}; then according to {G _d,3 (m,n)}, {μ ₇ (m ,n)}, {σ ₇ (m,n)}, get the normalized image of {G _d,3 (m,n)}, denoted as Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₇ (m,n) represents the pixel value of the pixel point whose coordinate position is (m,n) in {μ ₇ (m,n)}, and σ ₇ (m,n) represents {σ ₇ (m,n) )} The pixel value of the pixel whose coordinate position is (m,n).

Using the existing average calculation method, obtain the local average value image of {G _d,4 (m,n)}, denoted as {μ ₈ (m,n)}; and use the existing variance calculation method , get the local variance image of {G _d,4 (m,n)}, denoted as {σ ₈ (m,n)}; then according to {G _d,4 (m,n)}, {μ ₈ (m ,n)}, {σ ₈ (m,n)}, get the normalized image of {G _d,4 (m,n)}, denoted as Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₈ (m,n) represents the pixel value of the pixel whose coordinate position is (m,n) in {μ ₈ (m,n)}, and σ ₈ (m,n) represents {σ ₈ (m,n) )} The pixel value of the pixel whose coordinate position is (m,n).

Using the existing average calculation method, obtain the local average value image of {G _d,5 (m,n)}, denoted as {μ ₉ (m,n)}; and use the existing variance calculation method , get the local variance image of {G _d,5 (m,n)}, denoted as {σ ₉ (m,n)}; then according to {G _d,5 (m,n)}, {μ ₉ (m ,n)}, {σ ₉ (m,n)}, get the normalized image of {G _d,5 (m,n)}, denoted as Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₉ (m,n) represents the pixel value of the pixel whose coordinate position is (m,n) in {μ ₉ (m,n)}, and σ ₉ (m,n) represents {σ ₉ (m,n) )} The pixel value of the pixel whose coordinate position is (m,n).

Using the existing average calculation method, obtain the local average value image of {G _d,6 (m,n)}, denoted as {μ ₁₀ (m,n)}; and use the existing variance calculation method , get the local variance image of {G _d,6 (m,n)}, denoted as {σ ₁₀ (m,n)}; then according to {G _d,6 (m,n)}, {μ ₁₀ (m ,n)}, {σ ₁₀ (m,n)}, get the normalized image of {G _d,6 (m,n)}, denoted as Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₁₀ (m,n) represents the pixel value of the pixel point whose coordinate position is (m,n) in {μ ₁₀ (m,n)}, and σ ₁₀ (m,n) represents {σ ₁₀ (m,n) )} The pixel value of the pixel whose coordinate position is (m,n).

Step 4: Using P original undistorted screen images, set up a set of distorted screen images under different distortion types and different degrees of distortion, and use the set of distorted screen images as a training set. The training set includes multiple distorted screen images; then use Existing subjective quality evaluation method evaluates the subjective scoring value of each distorted screen image in the training set, and the subjective scoring value of the jth distorted screen image in the training set is recorded as DMOS _j ; and according to the operation of step 1 to step 3, Obtain the normalized images of the ten channel feature maps of each distorted screen image in the training set in the same way, and record the normalized images of the ten channel feature maps of the jth distorted screen image in the training set correspondingly for Among them, P>1, such as taking P=100, 1≤j≤K', K' indicates the total number of distorted screen images contained in the training set, 0≤DMOS _j ≤100, express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel whose middle coordinate position is (m,n).

Step 5: Convolution Neural Network (CNN) is a machine learning method based on brain neural network design, which can effectively suppress overfitting, so the method of the present invention utilizes Convolution Neural Network for all distorted screens in the training set The respective subjective score values of the images and the normalized images of the respective ten channel feature maps are used for training, so that the error between the regression function value obtained after training and the subjective score value is the smallest, and the optimal weight vector W is obtained by fitting. ^opt and the optimal bias item b ^opt t; then use W ^opt to construct the convolutional neural network regression training model; then according to the convolutional neural network regression training model, for Carry out the test and predict the objective quality evaluation prediction value of {I _d (i, j)}, denoted as Q, Q=f(x), Among them, Q is a function of x, f() is a function representation, x is an input variable, and x represents (W ^opt ) ^T is the transpose vector of W ^opt , Convolution function for x.

The convolutional neural network includes two convolutional layers, a fully connected layer, and an output layer. The number of filter kernels used in the convolutional layer is 64, and the size is 3×3. The first convolutional layer is followed by A max pooling layer, a second convolutional layer followed by a max pooling layer and an average pooling layer, both max pooling layers and an average pooling layer are of size 7×7 with a stride of 2 . The activation functions used are Rectified Linear Unit (ReLU). ReLU can be represented by f(y)=max(0,y), y is the input variable, and max() is the maximum value function. The reason for using ReLU as the activation function is: ReLU can be implemented by simply thresholding the zero activation matrix; ReLU will not saturate, and using ReLU as the activation function can greatly speed up the convergence of stochastic gradient descent.

In order to further verify the feasibility and effectiveness of the method of the present invention, experiments were carried out.

For this reason, the distorted screen image database is used to analyze the correlation between the objective quality evaluation prediction value and the subjective rating value of the distorted screen image obtained by the method of the present invention. Here, three commonly used objective parameters for evaluating image quality evaluation methods are used as evaluation indicators, namely Pearson correlation coefficient (Pearson linear correlation coefficient, PLCC) under nonlinear regression conditions, Spearman correlation coefficient (Spearman rank order correlation coefficient, SROCC), mean square Error (root mean squared error, RMSE), PLCC and RMSE reflect the accuracy of the objective quality evaluation prediction value of the distorted screen image, and SROCC reflects its monotonicity.

Using the existing subjective quality evaluation method to obtain the subjective scoring value of each distorted screen image in the distorted screen image database, and then using the method of the present invention to calculate the objective quality evaluation prediction value of each distorted screen image in the distorted screen image database. The objective quality evaluation prediction value of the distorted screen image calculated by the method of the present invention is used as a five-parameter Logistic function nonlinear fitting, the higher the PLCC and SROCC values, the lower the RMSE value, indicating the objective evaluation results and subjective scoring values of the objective evaluation method The better the correlation between. The PLCC, SROCC and RMSE correlation coefficients reflecting the quality evaluation performance of the method of the present invention are listed in Table 1. As can be seen from the data listed in Table 1, the correlation between the objective quality evaluation prediction value and the subjective scoring value of the distorted screen image obtained by the method of the present invention is very good, showing that the objective evaluation result and the result of subjective perception of human eyes are relatively good. Consistent enough to illustrate the feasibility and effectiveness of the method of the present invention. Table 1 Utilizes the difference between the objective quality evaluation prediction value and the subjective rating value of the distorted screen image obtained by the method of the present invention

Correlation

Claims (2)

1. a screen image quality evaluation method based on channel feature and convolutional neural network, is characterized in that comprising the following steps: Step 1: Let {I _d (i, j)} represent the distorted screen image to be evaluated, where, 1≤i≤W, 1≤j≤H, W represents the width of {I _d (i,j)}, H Represents the height of {I _d (i, j)}, I _d (i, j) represents the pixel value of the pixel whose coordinate position is (i, j) in {I _d (i, j)}; Step 2: Use the lumped feature channel method to perform feature extraction on {I _d (i, j)}, and obtain ten channel feature maps of {I _d (i, j)}, which are L channel feature map and U channel feature Figure, V channel feature map, gradient amplitude channel feature map, first directional gradient histogram channel feature map, second directional gradient histogram channel feature map, third directional gradient histogram channel feature map, fourth The directional gradient histogram channel feature map, the 5th directional gradient histogram channel feature map, and the 6th directional gradient histogram channel feature map, correspondingly recorded as {L _d (m,n)}, {U _d (m,n) )}, {V _d (m,n)}, {G _d,0 (m,n)}, {G _d,1 (m,n)}, {G _d,2 (m,n)}, { G _d,3 (m,n)}, {G _d,4 (m,n)}, {G _d,5 (m,n)}, {G _d,6 (m,n)}, where 1 ≤m≤M, 1≤n≤N, symbol is the symbol of the rounding down operation, M represents the width of each channel feature map of {I _d (i,j)}, N represents the height of each channel feature map of {I _d (i,j)}, L _d (m,n) represents the pixel value of the pixel whose coordinate position is (m,n) in {L _d (m,n)}, and U _d (m,n) represents the coordinates in {U _d (m,n)} The pixel value of the pixel point whose position is (m, n), V _d (m, n) represents the pixel value of the pixel point whose coordinate position is (m, n) in {V _d (m, n)}, G _{d, 0} (m,n) represents the pixel value of the pixel point whose coordinate position is (m,n) in {G _d,0 (m,n)}, and G _d,1 (m,n) represents {G _d,1 ( The pixel value of the pixel whose coordinate position is (m,n) in m,n)}, G _d,2 (m,n) means that the coordinate position in {G _d,2 (m,n)} is (m,n) ), G _d,3 (m,n) represents the pixel value of the pixel whose coordinate position is (m,n) in {G _d,3 (m,n)}, G _d,4 ( m,n) represents the pixel value of the pixel point whose coordinate position is (m,n) in {G _d,4 (m,n)}, G _d,5 (m,n) represents {G _d,5 (m, The pixel value of the pixel whose coordinate position is (m,n) in n)}, G _d,6 (m,n) represents the pixel value of the coordinate position (m,n) in {G _d,6 (m,n)} The pixel value of the pixel point; Step 3: Get {L _d (m,n)}, {U _d (m,n)}, {V _d (m,n)}, {G _d,0 (m,n)}, {G _{d, 1} (m,n)}, {G _d,2 (m,n)}, {G _d,3 (m,n)}, {G _d,4 (m,n)}, {G _d,5 ( The normalized images of m,n)} and {G _d,6 (m,n)} are correspondingly denoted as in, express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose middle coordinate position is (m,n); Step 4: Using P original undistorted screen images, set up a set of distorted screen images under different distortion types and different degrees of distortion, and use the set of distorted screen images as a training set. The training set includes multiple distorted screen images; then use The subjective quality evaluation method evaluates the subjective scoring value of each piece of distorted screen image in the training set, and records the subjective scoring value of the jth piece of distorted screen image in the training set as DMOS _j ; method to obtain the normalized images of the ten channel feature maps of each distorted screen image in the training set, and record the normalized images of the ten channel feature maps of the jth distorted screen image in the training set as Among them, P>1, 1≤j≤K', K' indicates the total number of distorted screen images contained in the training set, 0≤DMOS _j ≤100, express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose coordinate position is (m,n), express The pixel value of the pixel point whose middle coordinate position is (m,n); Step 5: Use the convolutional neural network to train the respective subjective score values of all the distorted screen images in the training set and the normalized images of the respective ten channel feature maps, so that the relationship between the regression function value obtained after training and the subjective score value The error between is the smallest, and the optimal weight vector W ^opt and the optimal bias item b ^opt are obtained by fitting; then the convolutional neural network regression training model is obtained by using the W ^opt construction; and then according to the convolutional neural network regression training model, right Carry out the test and predict the objective quality evaluation prediction value of {I _d (i, j)}, denoted as Q, Q=f(x), Among them, Q is a function of x, f() is a function representation, x is an input variable, and x represents is the transpose vector of W ^opt , Convolution function for x. 2. the screen image quality evaluation method based on channel feature and convolutional neural network according to claim 1, is characterized in that the concrete process of described step 3 is: Obtain the local average value image of {L _d (m,n)}, denoted as {μ ₁ (m,n)}; and obtain the local variance image of {L _d (m,n)}, denoted as {σ ₁ (m,n)}; then according to {L _d (m,n)}, {μ ₁ (m,n)}, {σ ₁ (m,n)}, get {L _d (m,n)} The normalized image of Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₁ (m,n) represents the pixel value of the pixel whose coordinate position is (m,n) in {μ ₁ (m,n)}, and σ ₁ (m,n) represents {σ ₁ (m,n) )} in the pixel value of the pixel whose coordinate position is (m, n); Obtain the local average value image of {U _d (m,n)}, denoted as {μ ₂ (m,n)}; and obtain the local variance image of {U _d (m,n)}, denoted as {σ ₂ (m,n)}; then according to {U _d (m,n)}, {μ ₂ (m,n)}, {σ ₂ (m,n)}, get {U _d (m,n)} The normalized image of Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₂ (m,n) represents the pixel value of the pixel whose coordinate position is (m,n) in {μ ₂ (m,n)}, and σ ₂ (m,n) represents {σ ₂ (m,n) )} in the pixel value of the pixel whose coordinate position is (m, n); Obtain the local average value image of {V _d (m,n)}, denoted as {μ ₃ (m,n)}; and obtain the local variance image of {V _d (m,n)}, denoted as {σ ₃ (m,n)}; then according to {V _d (m,n)}, {μ ₃ (m,n)}, {σ ₃ (m,n)}, get {V _d (m,n)} The normalized image of Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₃ (m,n) represents the pixel value of the pixel point whose coordinate position is (m,n) in {μ ₃ (m,n)}, and σ ₃ (m,n) represents {σ ₃ (m,n) )} in the pixel value of the pixel whose coordinate position is (m, n); Obtain the local average value image of {G _d,0 (m,n)}, denoted as {μ ₄ (m,n)}; and obtain the local variance image of {G _d,0 (m,n)}, Denote it as {σ _{4 (} m,n) _{} ;} then get {G _{d ,0} (m,n)} normalized image Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₄ (m,n) represents the pixel value of the pixel point whose coordinate position is (m,n) in {μ ₄ (m,n)}, and σ ₄ (m,n) represents {σ ₄ (m,n) )} in the pixel value of the pixel whose coordinate position is (m, n); Obtain the local average image of {G _d,1 (m,n)}, denoted as {μ ₅ (m,n)}; and obtain the local variance image of {G _d,1 (m,n)}, Recorded as {σ ₅ (m,n)}; then according to {G _d,1 (m,n)}, {μ ₅ (m,n)}, {σ ₅ (m,n)}, get {G _{d ,1} (m,n)} normalized image Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₅ (m,n) represents the pixel value of the pixel point whose coordinate position is (m,n) in {μ ₅ (m,n)}, and σ ₅ (m,n) represents {σ ₅ (m,n) )} in the pixel value of the pixel whose coordinate position is (m, n); Obtain the local average value image of {G _d,2 (m,n)}, denoted as {μ ₆ (m,n)}; and obtain the local variance image of {G _d,2 (m,n)}, Denote it as {σ ₆ (m,n)}; then according to {G _d,2 (m,n)}, {μ ₆ (m,n)}, {σ ₆ (m,n) _} , get _,2 (m,n)} normalized image Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₆ (m,n) represents the pixel value of the pixel point whose coordinate position is (m,n) in {μ ₆ (m,n)}, and σ ₆ (m,n) represents {σ ₆ (m,n) )} in the pixel value of the pixel whose coordinate position is (m, n); Obtain the local average image of {G _d,3 (m,n)}, denoted as {μ ₇ (m,n)}; and obtain the local variance image of {G _d,3 (m,n)}, _Denote it as {σ ₇ (m,n) _{} ;} then get {G _{d ,3} (m,n)} normalized image Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₇ (m,n) represents the pixel value of the pixel point whose coordinate position is (m,n) in {μ ₇ (m,n)}, and σ ₇ (m,n) represents {σ ₇ (m,n) )} in the pixel value of the pixel whose coordinate position is (m, n); Obtain the local average image of {G _d,4 (m,n)}, denoted as {μ ₈ (m,n)}; and obtain the local variance image of {G _d,4 (m,n)}, _Denote it as { _{σ 8} (m,n)} _; then get {G _{d ,4} (m,n)} normalized image Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₈ (m,n) represents the pixel value of the pixel whose coordinate position is (m,n) in {μ ₈ (m,n)}, and σ ₈ (m,n) represents {σ ₈ (m,n) )} in the pixel value of the pixel whose coordinate position is (m, n); Obtain the local average value image of {G _d,5 (m,n)}, denoted as {μ ₉ (m,n)}; and obtain the local variance image of {G _d,5 (m,n)}, _Denote it as { _{σ 9} (m,n)} _; then get {G _{d ,5} (m,n)} normalized image Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₉ (m,n) represents the pixel value of the pixel whose coordinate position is (m,n) in {μ ₉ (m,n)}, and σ ₉ (m,n) represents {σ ₉ (m,n) )} in the pixel value of the pixel whose coordinate position is (m, n); Obtain the local average value image of {G _d,6 (m,n)}, denoted as {μ ₁₀ (m,n)}; and obtain the local variance image of {G _d,6 (m,n)}, _Denote it as {σ ₁₀ (m,n) _{} ;} then get {G _{d ,6} (m,n)} normalized image Will The pixel value of the pixel point whose coordinate position is (m, n) is recorded as Among them, μ ₁₀ (m,n) represents the pixel value of the pixel point whose coordinate position is (m,n) in {μ ₁₀ (m,n)}, and σ ₁₀ (m,n) represents {σ ₁₀ (m,n) )} The pixel value of the pixel whose coordinate position is (m,n).