CN117390517A - Method for establishing prejudgment model based on classification model to knowledge degree of information - Google Patents

Abstract

The invention discloses a method for establishing a pre-judging model based on the knowledge degree of a classification model on information, which comprises the steps of presenting a to-be-read language material on a display; the tested eye movement data tested in the process of first reading the material of the language is collected; after the tested reading carefully is determined, the read material of the language is presented again on the display for re-reading, and the eyeball movement data of the tested person in the process of re-reading the material of the language is collected; extracting key characteristic indexes in the tested eyeball movement data in the process of reading the material twice; taking key characteristic indexes in eyeball motion data as input, constructing an eye motion prediction model based on a neural network, and performing classification judgment on whether to reread or not; judging whether the tested material is reread on the new language according to the trained eye movement prediction model; according to the invention, through establishing the reading text, 20 key features related to eye movement information are extracted to establish the prediction model, and whether the reading is first or second reading can be predicted with the accuracy of 78%.

Description

A method for establishing a predictive model based on the classification model’s knowledge of information

Technical field

The present invention relates to the field of artificial intelligence technology, and specifically relates to a method for establishing a predictive model based on the degree of knowledge of information by a classification model.

Background technique

The rereading strategy is recognized as the most widely used reading comprehension strategy. It can effectively help readers read quickly, improve their familiarity with information content, remember details better, deepen their understanding of text content, and improve reading efficiency.

Chinese patent document CN 112545451 A discloses a reading eye movement recording method and device, including: S1 registering the subject's age and education level, measuring the subject's vision level, and selecting appropriate style, font, and font size from the character library reading materials; the S2 control device sends the reading materials to the display through the connecting device and displays them at a set time; when the materials are displayed on the S3 display, the subjects are required to read according to their own reading habits, and at the end of reading and then close your eyes; the S4 eye tracker will record eye movement data in real time during the reading process, and transmit the eye movement data back to the control device through the connecting device for storage. The eye movement data includes gaze, gaze, saccades, and return glances. , reading, smooth visual tracking, linear eye movement behavior and rapid eye movement behavior data; the S5 control device conducts eye movement data analysis to obtain the subject's reading habits and reading strategies. The above-mentioned patent document uses a non-invasive method to record eye movement coordinate points in the reading process in real time, analyzes eye movement data such as saccades and fixations, and then quantitatively evaluates reading ability indicators. It is an evaluation of the subject's reading ability and is used for dynamic observation. The impact of diseases on the eyes, however, the eye movement indicators used in the above studies are few, general and not precise, and no overall reading level assessment model has been constructed. In addition, there is no method in the existing technology to judge the degree of information awareness through text reading combined with eye movements. Most of them fuse visual image features and text features. For example, Chinese patent document CN 116522212 discloses a lie based on image and text fusion. The detection method uses a classification model to output lie detection results by fusing visual image features and text features.

Contents of the invention

In order to solve the above-mentioned technical problems, the present invention can obtain the subject's awareness of the text information by reading the text, making the detection of the awareness of the information simpler and more convenient. To this end, the present invention provides a method based on A method for establishing a prediction model for the classification model’s knowledge of information.

The present invention adopts the following technical solutions:

On the one hand, the present invention provides a method for establishing a predictive model based on the classification model's knowledge of information, and presents the text material to be read on the display; the subject completes the first reading of the text material and collects the first reading text material. The subject's eye movement data during the process; after confirming that the subject has read carefully, the text material that the subject has read is presented on the monitor again for re-reading, and the subject's eye movement data during the process of re-reading the text material is collected; two The key feature indicators in the subject's eye movement data during the second reading of the text material; using the key feature indicators in the eye movement data as input, an eye movement prediction model based on the neural network is constructed to classify and judge whether to reread; combined with the training The eye movement prediction model can determine whether the subject stresses the new discourse material.

Further, after the subject completes the first reading, reading questions are presented on the monitor for the subject to answer. When the subject reaches the set accuracy rate, it is confirmed that the subject has read the text material carefully; after the subject completes the re-reading of the text, Change the order of presentation of the first reading questions on the monitor for subjects to answer again. The reading questions here are preferably single-choice questions.

Furthermore, the key feature indicators in the extracted reading eye movement data are eye movement indicators that are significantly different between first reading and rereading.

Preferably, the extracted key feature indicators include overall indicators and local interest area indicators;

The overall indicators include: total reading time, fixation time ratio, total number of fixations, average fixation time, number of forward saccades, average pupil size, average lookback distance, total number of lookbacks, average lookback time within a line, average within a line Looking back distance, number of looking back within a line, number of looking back across lines, average looking back time across lines, and average looking back distance across lines;

The local area of interest indicators include: IA_total fixation time, IA_total number of fixations, IA_average number of look-backs, IA_selective look-back path reading time, IA_average first fixation time, IA_average first fixation time Reading time, IA_Average number of first fixations, IA_Average second fixation time, IA_Average second reading time, and IA_Average number of second fixations.

Further, the method for judging intra-line glances, cross-line glances and forward saccades is:

Step 1. Determine the line spacing A_pix of the discourse material, the top Y-axis coordinate B_pix of the first line, the starting point coordinate current_sac_start_Y and the end point coordinate current_sac_end_Y of the saccade event;

Step 2. Calculate the number of rows where the saccade event starts and round it up:

start_row＝(“current_sac_start_Y”-B)/A

Step 3. Calculate the number of rows where the end point of the saccade event is and round it up:

end_row＝(“current_sac_end_Y”-B)/A

Step 4. Calculate the row skip value row_skip="end_row"-"start_row";

Step 5. Calculate x_skip within the same saccade,

x_skip="current_sac_end_X"-"current_sac_start_X"

If row_skip<0, it is a cross-row glance; if row_skip=0 and x_skip<0, it is an intra-row glance; if row_skip>0, it is a forward saccade.

Furthermore, the calculation method of the intra-line look-back distance and the cross-line look-back distance is:

In-line lookback distance (number): characters=x_skip/C;

Looking back distance across rows (characters): characters=(D*row skip-x_skip)/C;

Among them, C is the known pixel size of each letter, and D is the length dimension of the rectangular discourse material, in pix.

The technical solution of the present invention has the following advantages:

A. The method for judging the degree of information awareness based on the neural network classification model provided by the present invention, by establishing a reading text, monitoring the subject's eye movement information when reading the text, extracting 20 key features involving eye movement information, and establishing The eye movement prediction model based on the neural network can accurately predict whether the reader's awareness of the text material is reading it for the first time or re-reading it; this invention can detect whether an individual is familiar with the text material with an accuracy of 77%, which is helpful for evaluation in the teaching field. Students' knowledge mastery can help detect whether they are lying in the field of polygraph detection.

B. Currently, the saccade series indicators derived by eye movement data analysis software on the market basically only include the overall saccade distance, saccade time, etc. In order to meet the needs of eye movement characteristics of discourse materials, the present invention further divides saccades into The distance and time of forward saccades, intra-line glances, and cross-line glances significantly improve the detection accuracy of rereading.

Description of the drawings

In order to explain the specific embodiments of the present invention more clearly, the drawings needed to be used in the specific implementations will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present invention. For ordinary people in the art, For technical personnel, other drawings can also be obtained based on these drawings without exerting creative work.

Figure 1 is a block diagram of a method for establishing a predictive model provided by the present invention;

Figure 2 is a test flow chart provided by the present invention;

Figure 3 is a flow chart for building a text stress eye movement prediction model based on neural networks provided by the present invention.

Detailed ways

The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

As shown in Figures 1 and 2, the present invention provides a method for establishing a predictive model based on the classification model's awareness of information, which includes a computer host, a monitor, an eye tracker, etc., and the computer host is configured with a discourse Reread software system. First, a nine-point calibration is performed on the subject's reader's glasses. The number "3" is displayed in the upper left corner of the screen. Guide the subject's gaze to the upper left corner of the screen, and the number begins to count down and changes to "3-2-1".

The specific methods for establishing the prediction model are as follows:

[S01] Present the discourse material to be read on the display. The discourse materials are text materials retrieved from the computer host. Corresponding discourse materials can be retrieved based on the actual situation of different subjects. The retrieved discourse materials can be texts that the subjects are interested in. Of course, Can be uninteresting text.

[S02] The subject completed the first reading of the text material, and collected the subject’s eye movement data during the first reading of the text material.

The subjects read carefully from the beginning according to their daily reading speed and habits. There is no time limit for reading, and it is subject to understanding the content of the article. Please pay attention to keeping your eyes on the screen during the process. After reading is completed, you can enter the question-answering stage by operating the monitor, for example, pressing the space button to enter the next four reading comprehension questions. These are four-choice questions, and the subjects can answer according to the text materials they read.

[S03] After confirming that the subject has read carefully, the text material that the subject has read is presented on the monitor again for re-reading, and the eye movement data of the subject during the process of re-reading the text material is collected. When the subject reaches the set correct rate of answering questions, it is considered that the subject has carefully read the text material; then the text material read for the first time is presented on the display again for the subject to reread, and the subject's reading data is collected simultaneously. Eye movement data. Before re-reading, you can ask the subject to rest for a few minutes (for example, 3 minutes), ask the subject if he or she has rested well, and read the subject a second time after receiving an affirmative answer. Before re-reading begins, the nine-point eye movement calibration needs to be performed again, and a reminder is given, asking the subject: "The content of the second article is the same as the first article, but the question settings are different, so please treat it as a new one." Treat it" to prevent subjects from not reading carefully when re-reading. Of course, after re-reading the text material, you can also present multiple-choice reading questions on the monitor. You can use the same questions as those presented after the first reading or change the order of presentation of the reading questions. Of course, you can also re- Set reading questions, and determine whether the subjects are serious when rereading based on the correct rate of their answers. If both readings are determined to be serious, it is determined that the eye movement data can be adopted and used for subsequent prediction model building.

[S04] Extract key feature indicators from the subjects’ eye movement data during the two readings of discourse materials.

The present invention preferably uses an eye tracker to collect the subject's eye movement data during two readings of text materials, and extracts several key characteristic indicators in the eye movement data. The key characteristic indicators of the reading eye movement data extracted by the present invention are the eye movement indicators with significant differences between first reading and rereading.

The present invention creatively introduces parameters of forward saccade, intra-line glancing and cross-line glancing, and further sorts out the subject's eye movement characteristics through the following calculation method, so that the present invention has a higher degree of discrimination on whether the subject is rereading, Becomes the key to establishing a predictive rereading model. The specific calculation and judgment methods for intra-line gazing, cross-line gazing and forward saccades are as follows:

Step 1. Determine the line spacing A_pix of the discourse material, the top Y-axis coordinate B_pix of the first line, the starting point coordinate current_sac_start_Y and the end coordinate current_sac_end_Y of the saccade event; for example, the line spacing A_pix is 62pix, and the top Y-axis coordinate B_pix of the first line is 177pix.

Step 2. Calculate the number of rows where the saccade event starts and round it up:

start_row＝(“current_sac_start_Y”-177)/62

Step 3. Calculate the number of rows where the end point of the saccade event is and round it up:

end_row＝(“current_sac_end_Y”-177)/62

Step 4. Calculate the row skip value row_skip="end_row"-"start_row";

Step 5. Calculate x_skip within the same saccade,

x_skip="current_sac_end_X"-"current_sac_start_X"

If row_skip<0, it is a cross-row glance; if row_skip=0 and x_skip<0, it is an intra-row glance; if row_skip>0, it is a forward saccade.

Step 6. Calculate the intra-line looking back distance and the cross-line looking back distance. The unit is the number of letters. The specific method is:

In-line lookback distance (number): characters=x_skip/C;

Looking back distance across rows (characters): characters=(D*row_skip-x_skip)/C;

Among them, C is the known pixel size of each letter, and D is the length size of the rectangular discourse material, in pix. For example, the pixel size of each letter can be set through the computer, for example, set to C=21pix, and each letter in the entire rectangular text The length of the line can also be set through the computer, for example, set to D=1574pix, which will not be described here.

Since the current eye tracker cannot directly obtain the key characteristic indicators preferred by the present invention, after deriving the saccade information in the saccade indicators through the eye movement data analysis software, and then combined with the above calculation and judgment method provided by the present invention, it is concluded whether It is the key characteristic index of forward saccades, intra-line retrogazes, cross-line retrogazes and retrogaze distances.

Next, by testing the eye movement data obtained by each subject reading text materials, the overall indicators and local interest area indicators are analyzed respectively, and eye movement indicators with significant differences are extracted.

(1) Overall indicator analysis

Table 1 Description of overall eye movement indicators during rereading by readers with different reading levels (n ₁ = 63, n ₂ = 40)

Table 2 Repeated measurement results of overall eye movement indicators when readers with different reading levels reread (N=103)

It can be concluded from Table 1 and Table 2 that, in addition to the proportion of fixation time and average pupil size, the total reading time (F _(1,101) =107.25, p<0.001) and the total number of fixations (F _(1,101) =103.83, p< 0.001), average fixation time (F _(1,101) = 17.22, p < 0.001), and number of forward saccades ( F _(1,101) = 476.69, p < 0.001). The main effect of rereading was significant. The total reading time after rereading, The average fixation time is significantly shorter than that of the first reading, and the total number of fixations and the number of forward saccades after rereading are significantly shorter than that of the first reading; the main effect of reading level and the interaction effect between the number of readings and reading level are not significant (p>0.05 ).

Table 3 Description of retrospection indicators when readers with different reading levels reread (n ₁ = 63, n ₂ = 40)

Table 4 Repeated measurement results of retrospection indicators during rereading by readers with different reading levels (N=103)

As can be seen from Table 3 and Table 4, in addition to the average look-back time within a row and the average look-back time across rows, the average look-back distance (F _(1,101) = 7.77, p<0.01), the total number of looks back (F _{(1,101) )} ＝82.74, p<0.001), average intra-row gaze distance (F _(1,101) ＝9.27, p<0.01), intra-line gaze times (F _(1,101) ＝90.97, p<0.001), cross-line gaze times (F _(1,101) ＝703.79, p<0.001) and the average gaze distance across lines (F _(1,101) ＝11.54, p<0.001) were significant. The average gaze distance after rereading and the average gaze distance within a line were significantly longer than those for the first time. Reading, the distance of cross-line glances after rereading is significantly shorter than that of first reading, and the number of total glances, the number of within-line glances, and the number of cross-line glances are significantly less than those of first reading; except for the number of cross-line glances (F _(1,101) = 7.58, p< 0.01), the main effect of reading level is not significant in other overall looking back indicators (p>0.05), readers with low reading level look back across lines significantly more times than readers with high reading level; except for the number of looking back within a line (F _{( 1,101)} =4.13, p<0.05). On other overall retrospection indicators, the interaction effect between reading times and reading level is not significant (p>0.05).

(2)Local analysis

Table 5 Description of local eye movement indicators during rereading by readers with different reading levels (n ₁ = 63, n ₂ = 40)

Table 6 Repeated measures variance analysis of local eye movement indicators when readers with different reading levels reread (N=103)

It can be seen from Table 5 and Table 6 that the main effect of reading times is significant on all local eye movement indicators. The IA_total fixation time and IA_selective retrospection path reading time after rereading are significantly shorter than those for the first reading. IA_total fixation times and IA_average look-out times were significantly less than the first reading; except for IA_average look-out times (F _(1,101) = 4.34, p<0.05) and IA_selective look-back path reading time ( F _(1,101) =6.41, p<0.05), the main effect of reading level on other local indicators is not significant (p>0.05), readers with low reading level IA_ average number of look backs are significantly more than those with high reading level For readers, the reading time of IA_selective retrospection path is significantly shorter than that of readers with high reading level; except for IA_average number of retrospections (F _(1,101) = 4.60, p<0.05), IA_selective retrospection path reading time (F _(1,101) =4.38, p<0.05). On other local eye movement indicators, the interaction effect between reading times and reading level was not significant (p>0.05).

Table 7 Description of local eye movement indicators for the first and second times when readers with different reading levels reread (n ₁ = 63, n ₂ = 40)

Table 8 Repeated measures variance analysis of local eye movement indicators for the first and second times when readers with different reading levels reread (N=103)

It can be seen from Table 7 and Table 8 that the main effect of the number of readings is significant on all local eye movement indicators of the first and second passes. IA_average first fixation time after rereading, IA_average first reading Time, IA_average second fixation time, IA_average second reading time are significantly shorter than first reading time, IA_average first fixation number, IA_average second fixation number after rereading are significantly shorter than first reading time ;Except for IA_average first-pass reading time (F _(1,101) =5.11, p<0.05) and IA_average first-pass fixation times (F _(1,101) =4.90, p<0.05), in other first-pass and The main effect of reading level on the local indicators of the second pass is not significant (p>0.05). The average first-pass reading time of readers with low reading level IA_ is significantly longer than that of readers with high reading level. The average number of IA_ first-pass fixations is significant. More than readers with high reading level; except IA_average first fixation times (F _(1,101) =4.44, p<0.05), IA_average second fixation time (F _(1,101) =6.99, p<0.05) , on other local indicators of the first and second passes, the interaction effect between the number of readings and the reading level is not significant (p>0.05).

Through the above experimental analysis, a total of 20 key feature indicators were selected for the eye movement indicators that have significant differences between first reading and rereading in the above results, as shown in the table below.

[S05] Taking the key feature indicators in reading eye movement data as input, construct an eye movement prediction model based on neural networks to classify whether to reread or not. As shown in Figure 3, the input data is the extracted 20 key feature indicators. The present invention initially selects the feedforward neural network model (FNN), using a two-layer network, with 10 intermediate neurons preferably, and the output variable is whether it is the first time. There are 2 neurons for reading and rereading; 5-fold cross-validation is used, 1 set is used as a test set, and the remaining 4 sets are used as training sets for model training. Repeat the second step 4 times, and mark the first reading and rereading for each sample. Tag of. Optimize and adjust parameters of the model by adjusting network structure, activation function, learning rate, regularization and other parameters to improve the performance and generalization ability of the model; use the data of the test set to evaluate and test the model, and calculate the accuracy of the model Rate, precision, recall, F1 value, loss and other indicators to evaluate the performance and prediction ability of the model.

The following table is the obtained model prediction results:

The 5-fold cross-validation results (acc) are as follows:

1 2 3 4 5 MEAN 0.714 0.732 0.756 0.707 0.806 0.74

As can be seen from the table above, the final average accuracy can reach 74%.

The full set results are shown in the table below:

LOSS ACC PRECISION RECALL F1 0.548 0.769 0.774 0.788 0.781

[S06] Combined with the trained eye movement prediction model, determine whether the subject stresses the new discourse material.

Based on 20 eye movement characteristic indicators when subjects read text, this invention can detect whether an individual is familiar with text materials with an accuracy of 77%. It is helpful in the teaching field to evaluate students' knowledge mastery and is useful in the field of lie detection. It helps to assist in detecting lies and has important application value in the field of lie detection.

Obviously, the above-mentioned embodiments are only examples for clear explanation and are not intended to limit the implementation. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. An exhaustive list of all implementations is neither necessary nor possible. The obvious changes or modifications derived therefrom are still within the protection scope of the present invention.

Claims (6)

1. A method for establishing a prediction model based on the classification model's awareness of information, which is characterized by presenting the text material to be read on the display; the subject completes the first reading of the text material and collects the process of reading the text material for the first time After confirming that the subject has read carefully, the text material that the subject has read is presented on the monitor again for re-reading, and the eye movement data of the subject during the process of re-reading the text material is collected; extracted twice Key feature indicators in the subject's eye movement data during reading text materials; using the key feature indicators in the eye movement data as input, a neural network-based eye movement prediction model is constructed to classify and judge whether to reread; combined with the trained The eye movement prediction model determines whether the subject stresses the new discourse material. 2. The method for establishing a predictive model based on the classification model's awareness of information according to claim 1, characterized in that after the subject completes the first reading, reading questions are presented on the display for the subject to answer. When the subject reaches After setting the correct rate of answering questions, it is confirmed that the subject has carefully read the text material; after the subject completes the re-reading of the text, the subject will be asked to answer again by changing the order of presentation of the first reading questions on the monitor. 3. The method for establishing a prediction model based on the classification model's awareness of information according to claim 1, characterized in that the key feature indicators in the extracted reading eye movement data are eye movements with significant differences between first reading and rereading. index. 4. The method for establishing a predictive model based on the classification model's knowledge of information according to claim 3, characterized in that the extracted key feature indicators include overall indicators and local interest area indicators; The overall indicators include: total reading time, fixation time ratio, total number of fixations, average fixation time, number of forward saccades, average pupil size, average lookback distance, total number of lookbacks, average lookback time within a line, average within a line Looking back distance, number of looking back within a line, number of looking back across lines, average looking back time across lines, and average looking back distance across lines; The local area of interest indicators include: IA_total fixation time, IA_total number of fixations, IA_average number of look-backs, IA_selective look-back path reading time, IA_average first fixation time, IA_average first fixation time Reading time, IA_Average number of first fixations, IA_Average second fixation time, IA_Average second reading time, and IA_Average number of second fixations. 5. The method for establishing a predictive model based on the classification model's knowledge of information according to claim 4, characterized in that the method for judging within-line glances, cross-line glances and forward saccades is: Step 1. Determine the line spacing A pix of the discourse material, the top Y-axis coordinate B pix of the first line, the starting point coordinate current_sac_start_Y and the end point coordinate current_sac_end_Y of the saccade event; Step 2. Calculate the number of rows where the saccade event starts and round it up: start_row＝(“current_sac_start_Y”-B)/A Step 3. Calculate the number of rows where the end point of the saccade event is and round it up: end_row＝(“current_sac_end_Y”-B)/A Step 4. Calculate the row skip value row_skip="end_row"-"start_row"; Step 5. Calculate x_skip within the same saccade, x_skip="current_sac_end_X"-"current_sac_start_X" If row_skip<0, it is a cross-row saccade; if row_skip=0 and x_skip<0, it is an intra-row saccade; if row_skip>0, it is a forward saccade. 6. The prediction model establishment method based on the classification model's awareness of information according to claim 5, characterized in that the calculation method of the intra-line look-back distance and the cross-line look-back distance is: In-line lookback distance (number): characters=x_skip/C; Looking back distance across rows (characters): characters=(D*row skip-x_skip)/C; Among them, C is the known pixel size of each letter, and D is the length dimension of the rectangular discourse material, in pix.