WO2025050579A1 - Molecular structure recognition method and apparatus, electronic device and storage medium - Google Patents

Abstract

The present disclosure provides a molecular structure recognition method and apparatus, an electronic device and a storage medium. The method comprises: acquiring a molecule image; initializing an empty angle set and, when a molecular structure is decoded on the basis of image features of the molecule image and a branch angle is obtained for the first time, storing the branch angle into the angle set; extracting one branch angle from the angle set, using the branch angle as a guide to decode the molecular structure at the branch angle on the basis of the image features of the molecular image, and updating the angle set on the basis of a new decoded branch angle for decoding the molecular structure at the next branch angle until the angle set is empty; and, on the basis of decoding results at each branch angle, determining the molecular structure corresponding to the molecule image.

Description

Molecular structure recognition method, device, electronic device and storage medium

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to Chinese patent application No. 2023111369517, filed on September 4, 2023, entitled “Molecular structure identification method, device, electronic device and storage medium”, which is incorporated herein by reference in its entirety.

Technical Field

The present disclosure relates to the field of image recognition technology, and in particular to a molecular structure recognition method, device, electronic device and storage medium.

Background Art

Chemical molecular structure recognition can be widely used in fields including pharmaceutical research and development, human-computer interaction, biochemistry, education, organic synthesis, etc.

Currently, research on handwritten chemical molecular structure recognition relies too much on rule-based post-processing methods, which simplifies the complexity of molecular structure recognition itself, making it difficult to parse and recognize complex handwritten molecular layouts under existing solutions.

Summary of the invention

The present disclosure provides a molecular structure recognition method, device, electronic device and storage medium to solve the defect of the prior art that it is difficult to recognize complex handwritten molecular structures.

The present disclosure provides a molecular structure recognition method, comprising:

Acquire molecular images;

Initialize an empty angle set, and when decoding the molecular structure based on the image features of the molecular image and decoding the branch angle for the first time, store the branch angle in the angle set; take a branch angle from the angle set, and use the branch angle For guidance, the molecular structure at the branch angle is decoded based on the image features of the molecular image, and the angle set is updated based on the new branch angle obtained by decoding for decoding of the molecular structure at the next branch angle, until the angle set is empty;

Based on the decoding results at each branch angle, the molecular structure corresponding to the molecular image is determined.

According to a molecular structure recognition method provided by the present disclosure, the updating of the angle set based on the new branch angle obtained by decoding includes:

Perform chemical bond detection on the new branch angle and each existing branch angle in the angle set;

Based on the detection result of the chemical bond detection, the angle set is updated.

According to a molecular structure recognition method provided by the present disclosure, the updating of the angle set based on the detection result of the chemical bond detection includes:

When the detection result indicates that there is no chemical bond, storing the new branch angle into the angle set;

In the case where the detection result indicates the presence of a chemical bond, the branch angle that forms a chemical bond with the new branch angle is deleted from the angle set.

According to a molecular structure recognition method provided by the present disclosure, the molecular structure at the branching angle is decoded based on the image features of the molecular image with the branching angle as a guide, including:

Determining visual context features at a current decoding moment at the branch angle based on the decoding features of the branch angle and a decoding state at a previous decoding moment at the branch angle;

Based on the visual context feature, the molecular structure of the image feature at the current decoding moment is decoded to obtain the decoding state at the current decoding moment, and the current decoding moment is returned as the previous decoding moment for decoding until the decoding at the branch angle is completed.

According to a molecular structure recognition method provided by the present disclosure, the method of taking A branch angle is obtained, and the molecular structure under the branch angle is decoded based on the image features of the molecular image with the branch angle as a guide, and the angle set is updated based on the new branch angle obtained by decoding for decoding the molecular structure under the next branch angle, until the angle set is empty, including:

Based on the recognition model, a branch angle is taken out from the angle set, and the molecular structure under the branch angle is decoded based on the image features of the molecular image with the branch angle as a guide, and the angle set is updated based on the new branch angle obtained by decoding for decoding of the molecular structure under the next branch angle, until the angle set is empty;

The recognition model is trained based on the sample image and the molecular structure label corresponding to the sample image;

The molecular structure label is obtained by connecting the atomic groups and chemical bonds in the molecular formula corresponding to the sample image into a molecular structure graph and then performing graph traversal.

According to a molecular structure recognition method provided by the present disclosure, the step of determining the molecular structure tag includes:

Connecting the atomic groups and chemical bonds in the molecular formula corresponding to the sample image into a molecular structure diagram;

The molecular structure graph is traversed, and the molecular structure label is generated based on the atomic groups, chemical bonds, angles, nesting symbols and reconnection marks obtained through the traversal.

According to a molecular structure recognition method provided by the present disclosure, the training step of the recognition model includes:

Based on the initial model, molecular structure recognition is performed on the sample image to obtain a chemical bond detection result between the sample branch angle decoded in the molecular structure recognition process and the existing branch angles in the sample angle set, as well as a structural recognition result of the sample image;

Based on the structure recognition result and the molecular structure label, as well as the chemical bond detection and the reconnection mark in the molecular structure label, the initial model is iterated with parameters to obtain the recognition model.

The present disclosure also provides a molecular structure recognition device, comprising:

An acquisition unit, used for acquiring a molecular image;

The identification unit is used to initialize an empty angle set, and when decoding the molecular structure based on the image features of the molecular image and decoding to a branch angle for the first time, store the branch angle in the angle set; take out a branch angle from the angle set, and decode the molecular structure under the branch angle based on the image features of the molecular image with the branch angle as a guide, and update the angle set based on the new branch angle obtained by decoding for decoding of the molecular structure under the next branch angle, until the angle set is empty;

The output unit is used to determine the molecular structure corresponding to the molecular image based on the decoding results at each branch angle.

The present disclosure also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the molecular structure recognition method described above is implemented.

The present disclosure also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the molecular structure recognition methods described above.

The present disclosure also provides a computer program product, including a computer program, wherein when the computer program is executed by a processor, the molecular structure recognition method described above is implemented.

The molecular structure recognition method, device, electronic device and storage medium provided by the present invention add a decoding mechanism for each branch angle in the chemical molecular structure and a maintenance mechanism for the branch angle to be explored, and use the branch angle in the chemical molecular structure as a guiding condition during decoding to enrich the information of molecular structure decoding, improve the reliability of molecular structure decoding, and improve the decoding accuracy for complex chemical molecular structures.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the present disclosure or the prior art, A brief introduction is given to the drawings required for use in the examples or descriptions of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

FIG1 is a schematic diagram of a molecular structure recognition method provided by the present disclosure;

FIG2 is a schematic diagram of molecular structure decoding provided by the present disclosure;

FIG3 is a handwritten molecular image provided by the present disclosure;

FIG4 is a molecular structure diagram provided by the present disclosure;

FIG5 is a molecular structure tag provided by the present disclosure;

FIG6 is a schematic diagram of the structure of a molecular structure recognition device provided by the present disclosure;

FIG. 7 is a schematic diagram of the structure of an electronic device provided by the present disclosure.

DETAILED DESCRIPTION

In order to make the purpose, technical solutions and advantages of the present disclosure clearer, the technical solutions in the present disclosure will be clearly and completely described below in conjunction with the drawings in the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

In recent years, due to the progress of computer vision technology in object detection and semantic segmentation, satisfactory recognition performance has been achieved for simple and controllable printed molecular images. At the same time, research on the recognition of handwritten chemical molecular structures has made significant progress. However, these recognition methods for handwritten chemical molecular structures may ignore the importance of recognizing handwritten molecular images in human-computer interaction, or oversimplify the complexity of molecular structure recognition and over-rely on rule-based post-processing methods, resulting in the recognition of large and complex molecular structures still being a major challenge.

FIG. 1 is a schematic diagram of a molecular structure recognition method provided by the present disclosure. 1, the method comprises:

Step 110, acquiring a molecular image.

The molecular image here refers to an image with a chemical molecular structure drawn on it. Furthermore, the chemical molecular structure in the molecular image may be printed or handwritten, and this is not specifically limited in the embodiments of the present disclosure.

Step 120, initialize an empty angle set, and when decoding the molecular structure based on the image features of the molecular image and decoding to a branch angle for the first time, store the branch angle in the angle set; take out a branch angle from the angle set, and decode the molecular structure at the branch angle based on the image features of the molecular image with the branch angle as a guide, and update the angle set based on the new branch angle obtained by decoding for decoding of the molecular structure at the next branch angle, until the angle set is empty.

Taking into account the complexity of the chemical molecular structure itself, when performing molecular structure recognition on molecular images, we can add an angle decoding and maintenance mechanism for each branch in the chemical molecular structure under the conventional encoder + decoder architecture, and use the branch angle in the chemical molecular structure as a guiding condition for decoding, so as to improve the decoding reliability and accuracy for complex chemical molecular structures.

In a specific implementation, branch-by-branch decoding may be performed based on the image features of the molecular image. It is understandable that the branch-by-branch decoding here is, that is, a graph traversal of the molecular image.

Figure 2 is a schematic diagram of molecular structure decoding provided by the present invention. As shown in Figure 2, in (a) of Figure 2, at the beginning of the graph traversal, an empty angle set Memory can be initialized. During the traversal, a branch point is encountered. At this branch point, two branch angles can be decoded, namely branch angle ① and branch angle ②, wherein branch angle ① points upward and branch angle ② points to the lower right. Branch angle ① and branch angle ② are the branch angles decoded for the first time and are both stored in the angle set Memory.

Then, in (b) of Figure 2, the branch angle is taken from the angle set Memory ①, traverse upward along branch angle ① until a new branch point is encountered. In Figure 2 (c), two branch angles are obtained through decoding, namely branch angle ③ and branch angle ④, where branch angle ③ points to the right and branch angle ④ points downward. Branch angle ③ and branch angle ④ are both stored in the angle set Memory. Then, in Figure 2 (d), branch angles ①, ③, ⑤, and ⑦ are taken out from the angle set Memory in turn to traverse the graph until a branch traversal is completed.

In Figure 2 (e), after completing the traversal of a branch, the branch angle ⑥ is taken out from the angle set Memory, and the traversal continues along the branch angle ⑥. At this time, it is found that the traversed atom is connected to the previously visited atom, that is, the atoms marked as two solid circles in Figure 2 (e) are connected. For this situation, in Figure 2 (f), it can be predicted whether the branch angle ⑧ stored in the angle set Memory is a loop key. If so, it can be considered that the branch angle ⑧ has completed the traversal and is taken out and deleted from the angle set Memory. The loop key this time is understood as whether the two branch angles form a reconnection relationship (reconnection), that is, a cycle of branch angles is formed.

Based on the above traversal method, after completing the branch decoding traversal under all branch angles in the angle set Memory, it can be considered that the graph traversal for the molecular image is completed.

It can be understood that the angle set is a set for storing branch angles to be traversed. At the beginning of the graph traversal, an angle set can be initialized first. At this time, the angle set is empty, that is, at the beginning of the graph traversal, there are no branch angles to be traversed in the angle set. In the process of graph traversal, when the branch point is traversed for the first time, that is, when the branch angle is decoded for the first time, the branch angle decoded for the first time can be stored in the angle set. At this time, the angle set is no longer empty, and the branch angles to be traversed are stored in the angle set, and in subsequent traversals, the branch angles can be taken out from the angle set for exploration. Therefore, the angle set is used to store and maintain the branch angles to be traversed, and the angle set can be updated and adjusted accordingly based on the decoded new branch angles during the decoding process of the branch angles to facilitate the decoding of the next branch angle.

Furthermore, in the process of decoding any branch angle, it is necessary to first decode the angle The branch angle is taken out from the degree set, that is, the branch angle is confirmed to be the branch angle being explored or has been explored, so that the angle set can always maintain the branch angle that has not been explored.

After taking out the branch angle, the molecular structure under the branch angle can be decoded. Here, when decoding, the molecular structure can be decoded along the branch angle, that is, the branch angle is used as conditional information to guide the decoding of the molecular structure, thereby enriching the information of the molecular structure decoding and improving the reliability of the molecular structure decoding.

In the process of decoding the molecular structure under the branch angle, if a new branch angle is decoded, the angle set can be updated based on the new branch angle. Specifically, the new branch angle can be added to the angle set as a branch angle that has not been explored for subsequent decoding. In addition, before adding the new branch angle as a branch angle that has not been explored to the angle set, the relationship between the branch angle and the existing branch angles in the angle set can also be judged, that is, whether the new branch angle and the existing branch angle constitute a reconnection relationship in the molecular structure. If a reconnection relationship is formed, it can be considered that the new branch angle and the existing branch angles that constitute a reconnection relationship with it have been explored, and the new branch angle does not need to be stored in the angle set, and the existing branch angles that constitute a reconnection relationship with it can also be deleted from the angle set, so that the angle set can always maintain the branch angles that have not been explored.

Step 130: Determine the molecular structure corresponding to the molecular image based on the decoding results at each branch angle.

Specifically, after decoding the molecular structure at each branch angle in the molecular image, the decoding results at each branch angle can be obtained. The decoding results here may include chemical bonds, angles, atomic groups, etc. at the branch angle. On this basis, combined with the decoding results at each branch angle, the molecular structure corresponding to the molecular image can be obtained. The molecular structure here can be a string recorded in a markup language based on an organic chemical structural formula, or a representation diagram of the molecular structure, or a vector representation of the molecular structure in a specific coding language. The embodiments of the present disclosure do not specifically limit this.

The method provided in the embodiment of the present disclosure adds the ability to identify the branches in the chemical molecular structure. The decoding of angles and the maintenance mechanism of branch angles to be explored, and the branch angles in the chemical molecular structure are used as guiding conditions for decoding to enrich the information of molecular structure decoding, improve the reliability of molecular structure decoding, and improve the decoding accuracy for complex chemical molecular structures.

Based on the above embodiment, in step 120, updating the angle set based on the new branch angle obtained by decoding includes:

Perform chemical bond detection on the new branch angle and each existing branch angle in the angle set;

Based on the detection result of the chemical bond detection, the angle set is updated.

Specifically, considering that there may be reconnection relationships in the molecular structure, for example, the single bonds of 6 in the benzene ring are connected end to end, if we do not consider whether the new branch angle obtained by decoding constitutes a chemical bond with the branch angle that has not been explored in the angle set, it is very likely to lead to repeated traversal, that is, decoding the molecular structures under the two branch angles separately, so that the molecular structure that should have been under one branch is incorrectly decoded into two molecular structures, affecting the accuracy of molecular structure recognition.

Therefore, after decoding to obtain a new branch angle, the new branch angle can be respectively compared with each existing branch angle in the angle set for chemical bond detection. Here, for the new branch angle and any branch angle in the angle set, it is possible to detect whether a chemical bond is formed between the two branch angles, and the detection result obtained thereby can include whether a chemical bond exists, and can further include the type of the formed chemical bond, such as a single bond, a double bond, etc., when it is determined that a chemical bond is included.

After obtaining the test results of the chemical bond detection between the new branch angle and each existing branch angle in the angle set, the angle set can be updated based on this. It can be understood that if there is no chemical bond between the new branch angle and each existing branch angle, it means that the reconnection relationship has not been decoded yet, and the new branch angle can also be placed in the angle set as a branch angle that has not been explored; if there is a chemical bond between the new branch angle and one of the existing branch angles, it means that the reconnection relationship has been decoded, and the chemical bond between the new branch angle and the new branch angle will be placed in the angle set. Existing branch angles are deleted from the angle collection, and new branch angles are not stored in the angle collection.

Based on any of the above embodiments, in step 120, updating the angle set based on the detection result of the chemical bond detection includes:

When the detection result indicates that there is no chemical bond, storing the new branch angle into the angle set;

In the case where the detection result indicates the presence of a chemical bond, the branch angle that forms a chemical bond with the new branch angle is deleted from the angle set.

Specifically, for a new branch angle, in the detection results of the chemical bond detection between the branch angle and each existing branch angle, if all the detection results indicate that there is no chemical bond, that is, there is no chemical bond between the new branch angle and each existing branch angle, it means that the reconnection relationship has not been decoded yet, and the new branch angle also needs to be explored. At this time, the new branch angle can be stored in the angle set, so that the angle set can continue to maintain all branch angles that have not been explored;

If there is a detection result indicating the existence of a chemical bond, that is, there is a chemical bond between the new branch angle and an existing branch angle in the angle set, at this time there is a reconnection relationship between the new branch angle and the existing branch angle, it can be considered that the new branch angle and the existing branch angle that forms a reconnection relationship with it have been explored, and the new branch angle does not need to be stored in the angle set, and the existing branch angle that forms a reconnection relationship with it can also be deleted from the angle set, so that the angle set can always maintain the branch angles that have not been explored.

Based on any of the above embodiments, in step 120, the decoding of the molecular structure at the branching angle based on the image features of the molecular image and guided by the branching angle includes:

Determining visual context features at a current decoding moment at the branch angle based on the decoding features of the branch angle and a decoding state at a previous decoding moment at the branch angle;

Based on the visual context feature, the molecular structure of the image feature at the current decoding moment is decoded to obtain the decoding state at the current decoding moment, and the current decoding moment is returned as the previous decoding moment for decoding until the decoding at the branch angle is completed.

Specifically, in the process of decoding a molecular structure at a branch angle, the branch angle can be used as a guiding condition for decoding the molecular structure. When a branch is encountered, decoding can continue along the direction of the branch angle taken from the angle set, and the branch angle applied at this time guides the generation of visual context features when decoding the molecular structure at the branch angle.

That is, when generating a visual context feature at a decoding moment, compared with the prior art that only applies the decoding state of the previous decoding moment to determine the visual context feature at the current decoding moment, the decoding feature of the branch angle is also considered. It can be understood that here, the decoding feature of the branch angle is stored when the branch angle is decoded during the decoding process of the molecular structure, and the decoding feature of the branch angle can specifically include the attention weight information and state feature of the branch angle.

For example, the calculation of the visual context features at the current decoding moment can be expressed as the following formula:

Where, e _t,i is the energy of the i-th position in the feature map at the t-th decoding moment, w, W ^x , W ^y , W ^S , ^Wa , W ^sp , and W ^ap are all projection parameters of the attention module. _xi is the image feature of the molecular image at the i-th position, is the decoding at the t-1th decoding time As a result, s _t-1 is the decoding state at the t-1th decoding time, a _t-1 and a _t are the attention weights at the t-1th decoding time and the tth decoding time, respectively, and a _b and s _b are the attention weight information and state features during branch angle decoding, respectively. It should be noted that when there is no branch angle for decoding, a _b and s _b are both zero vectors, and the calculation formula of e _t,i is consistent with the calculation formula of e _t,i in the visual context feature acquisition method in the conventional string decoder.

a _t,i is the attention weight at the i-th position at the t-th decoding moment, h and w are the height and width of the molecule image respectively; c _t is the visual context feature at the t-th decoding moment.

After obtaining the visual context features at the current decoding moment, the molecular structure decoding at the current decoding moment can be performed based on the features, thereby obtaining the decoding state at the current decoding moment, which can be specifically expressed as the following formula:

p _t = softmax(W ^c s _t );

Where s _t is the decoding state at the current decoding moment, GRU represents the computational unit of the recursive neural network, _{p t} represents the probability distribution of the decoding result output at the current decoding moment, and W ^c is the parameter of the classification layer.

It is understandable that after the molecular structure decoding at the current decoding moment is completed, the decoding state at the current decoding moment can be used as the decoding state at the previous decoding moment, and the molecular structure decoding at the next decoding moment can be performed. In addition, in this process, at different moments of decoding the molecular structure at a branch angle, the decoding features of the same branch angle are applied.

In the disclosed embodiment, the neural network structure for implementing molecular structure decoding is recorded as a random conditional guided decoder. The random conditional guided decoder is significantly better than the traditional string decoder in generalizing complex structures without adding significant additional parameters or computational burden. The use of conditional guidance mechanism and path selection can effectively improve the accuracy of multi-path decoding.

Based on any of the above embodiments, step 120 includes:

Based on the recognition model, a branch angle is taken from the angle set, and the branch angle The molecular structure under the branch angle is decoded based on the image features of the molecular image, and the angle set is updated based on the new branch angle obtained by decoding for decoding the molecular structure under the next branch angle, until the angle set is empty;

The recognition model is trained based on the sample image and the molecular structure label corresponding to the sample image;

The molecular structure label is obtained by connecting the atomic groups and chemical bonds in the molecular formula corresponding to the sample image into a molecular structure graph and then performing graph traversal.

Specifically, step 120 can be implemented by a pre-trained recognition model. Here, the recognition model can be in the form of an encoder-decoder. The training of the recognition model can be implemented by supervised learning. It can be understood that the sample of supervised learning is a sample image with a chemical molecular structure drawn, and the label of supervised learning is the molecular structure label corresponding to the sample image, where the molecular structure label reflects the molecular structure in the form of language.

Conventional markup languages for organic chemical structures can be Chemfig, SMILES, etc. Compared with the most commonly used structural markup language SMILES, Chemfig requires almost no abstract chemical knowledge. It is designed to use the chemical structure graphics themselves to improve relevance and accuracy. The syntax of Chemfig also provides a description of "angle", which can be used to specify the direction of the bonds in the molecule. In the handwritten molecular image shown in Figure 3, the atom "N" is connected to two double bonds, one pointing to the upper right and the other pointing to the lower right. The angle codes in the Chemfig syntax, such as "[1]" and "[-1]", can be used to indicate the approximate orientation of the two double bonds, thereby making the visual presentation of the molecule more accurate and comprehensive.

However, if the original Chemfig markup language is directly used as the label for recognition model training, the following problems still exist:

First, Chemfig has grammatical ambiguity: due to different starting points and traversal orders, there are multiple correct Chemfig sequences that can represent the same molecular image. This ambiguity is inevitable and increases the difficulty of model learning.

Second, Chemfig syntax is complex: it is necessary to incorporate prior rules and domain knowledge. The grammar is built into the Chemfig framework to keep it simple. For example, rules such as using brackets to group atoms, and the need to remove redundant chemical symbols and valence dashes to reduce grammatical complexity. In addition, domain knowledge such as IUPAC (International Union of Pure and Applied Chemistry) naming rules are necessary to ensure that compound structures are correctly labeled. The use of these rules and domain knowledge increases the complexity of Chemfig's grammar. For example, a benzene ring is simply represented as "*6(––)", and the directions of the six single bonds need to be automatically inferred based on the rules for benzene rings. These a priori rules are complex and increase the difficulty of learning the model.

In order to overcome the above problems, the disclosed embodiment proposes parsing the atomic groups and chemical bonds in the molecular formula corresponding to the sample image and connecting them into a molecular structure graph, and then performing graph traversal to obtain a molecular structure label.

Here, considering that the molecular formula corresponding to the sample image can be represented by different character strings, in order to resolve this ambiguity at the grammatical level, in the embodiment of the present disclosure, the atomic groups and chemical bonds are manually parsed and restored from the molecular formula character string, and then all the obtained atomic groups and chemical bonds are connected to generate a graph, which is recorded here as a molecular structure graph.

For example, "HO-**6(---(-COOH)---)", "COOH-[4]**6(---(-OH)---)" and "**6([:30]-(-OH)---(-COOH)--)" are different string representations of the same molecular formula. By parsing and restoring the atomic groups and chemical bonds, a molecular structure diagram as shown in Figure 4 can be constructed, thereby achieving unified representation in the form of a structure diagram.

In Figure 4, character strings are represented as atom groups, such as "HO", "COOH". The ring-shaped benzene molecular structure is regarded as a special atom group. The lines between atom groups represent chemical bonds, which can be single bonds "-", double bonds "=" or triple bonds. An atom group can be connected to another atom group through a single or multiple chemical bonds. By using atom groups as vertices and chemical bonds as edges, a molecular structure diagram can be obtained. Therefore, even if different Chemfig label strings are used, the graphical representation of the same molecule is the same, thus eliminating possible ambiguity in the labels.

The molecular structure diagram obtained is a complex data structure and needs to be Traverse and convert the molecular structure graph into a label format suitable for model training. Here, the molecular structure graph needs to be converted into an equivalent one-dimensional text representation through graph traversal. The conversion rule here can be a fixed rule, which can be to traverse the molecular structure graph from a specific point and append the traversed nodes (atomic groups) and edges (chemical bonds) to the end of the string, thereby obtaining a concise molecular structure label without grammatical ambiguity.

The method provided by the embodiment of the present disclosure connects the atomic groups and chemical bonds in the molecular formula into a molecular structure graph and then performs a graph traversal to obtain a concise molecular structure label without grammatical ambiguity, which can reduce the difficulty of supervised learning of the recognition model and improve the reliability of molecular structure recognition based on the recognition model.

Based on any of the above embodiments, the step of determining the molecular structure tag includes:

Connecting the atomic groups and chemical bonds in the molecular formula corresponding to the sample image into a molecular structure diagram;

The molecular structure graph is traversed, and the molecular structure label is generated based on the atomic groups, chemical bonds, angles, nesting symbols and reconnection marks obtained through the traversal.

Specifically, the molecular structure graph obtained by structural analysis and connection of molecular formulas is a complex data structure that must be converted into a form suitable for model training. For the recognition model of the encoder-decoder structure, the form suitable for training is usually a one-dimensional string or a standardized label. Standardized labels involve converting the molecular structure graph into an equivalent one-dimensional text representation.

In order to ensure that the rules for converting molecular structure labels are consistent and unambiguous, the specific rules can be pre-set fixed rules. For the traversal of the molecular structure graph, the molecular structure graph can be traversed from a specific point, and the nodes (atomic groups) and edges (chemical bonds) are appended to the end of the string. After the traversal is completed, a standardized label can be obtained. For example, the molecular structure graph shown in Figure 4 is traversed to obtain the following standardized labels:

H O-[:0]? [a](-[:60]-[:0]-[:300](-[:0]C O O H)-[:240]-[:180]?[a,{-}](--[:60]\circle))

Among them, each unit is separated by a space. Graph traversal can use Depth-First Search (DFS). The above standardized labels can contain four types of elements, namely "atom group", "chemical bond", "angle", "nested symbol" and "reconnection mark". Specifically in the above standardized labels, "H O" represents the atomic group, "?[a]", "?[a,{-}]" are reconnection marks, "-" represents the chemical bond, "[:0]" represents the angle, and "(" represents the nested symbol.

On this basis, the standardized labels can be integrated to obtain structure-specific molecular structure labels. Among them, if the atom group does not contain any characters, it can be omitted. The entity characters can be represented using LaTeX notation. Since "angle" and "chemical bond" are closely related in visual information, the corresponding angle can be appended to each chemical bond as "[:]" in the standardized label to represent the "bond angle" modeling unit. The angle value can be calculated according to the format during the graphic construction and output process. In addition, the nested symbol "()" can be used to represent nested branches. In addition, the original Chemfig syntax can be retained for reconnection labeling, represented by "?[a]", which represents the unit surrounded by the cyclic unit represented by "?[a,-]". When reconnection occurs, the reconstructed angle value will not be represented. Finally, "○" can be used to represent a specific atom in benzene, and a virtual bond angle unit "–[:]" can be constructed to represent the connection between "○" and any atom on the ring.

For example, the standardized labels shown above can be integrated to form the molecular structure label shown in Figure 5, in which the nested relationship and the reconnection relationship at different angles are marked. Among them, "\eob" indicates that the decoding is completed at a branch angle, the line with an arrow between "\angle[:60]" and "-[:60]" indicates the nested relationship, and the double-arrow line between "\angle[:300]" and "\angle[:120]" and "reconnection" indicate the reconnection mark.

The method provided by the embodiment of the present disclosure connects the atomic groups and chemical bonds in the molecular formula into a molecular structure graph, and forms a molecular structure label by traversing the molecular structure graph. The molecular structure label thus formed has the advantage of being more consistent with the image than the mainstream language SMILES, and is more conducive to model learning. In addition, the syntax of the molecular structure label is not affected by chemical knowledge. This enables it to represent erroneous or non-existent molecular structures and makes it more versatile. The above method provides an innovative approach for the characterization of molecular structures and has potential applications in various fields of chemical structure identification and analysis.

Based on any of the above embodiments, the training step of the recognition model includes:

Based on the initial model, molecular structure recognition is performed on the sample image to obtain a chemical bond detection result between the sample branch angle decoded in the molecular structure recognition process and the existing branch angles in the sample angle set, as well as a structural recognition result of the sample image;

Based on the structure recognition result and the molecular structure label, as well as the chemical bond detection and the reconnection mark in the molecular structure label, the initial model is iterated with parameters to obtain the recognition model.

Specifically, for the recognition model, an initial model of the encoder + decoder structure can be obtained, where the model parameters of the initial model can be initialized. During the training process, the molecular structure of the sample image can be recognized based on the initial model. Specifically, the image feature of the sample image can be extracted based on the initial model, and then a sample branch angle is taken out from the sample angle set. The sample branch angle is used as a guide, and the molecular structure under the sample branch angle is decoded based on the image features of the sample image, and the sample angle set is updated based on the new sample branch angle obtained by decoding for the molecular structure decoding under the next sample branch angle, until the sample angle set is empty.

It can be understood that when the initial model is decoded to obtain a new sample branch angle, the sample angle set needs to be updated based on the new sample branch angle obtained by decoding. When the sample angle set is updated, the new sample branch angle needs to be chemically bonded with each existing branch angle in the sample angle set respectively, thereby obtaining the chemical bond detection result between the sample branch angle and the existing branch angles in the sample angle set.

Here, the presence or absence of chemical bonds in the chemical bond detection results is used to reflect whether there is a reconnection relationship between the branch angles of the two samples. The chemical bond detection results can be compared with the reconnection mark in the molecular structure label to measure the initial model from the perspective of reconnection relationship detection. Training loss.

In addition, the structure recognition results and molecular structure labels can be compared to measure the training loss of the initial model from the perspective of molecular structure recognition.

Furthermore, the loss L _ce of molecular structure recognition can be expressed based on the following formula:

Where _Lce is the cross entropy loss, V represents the number of unit characters of the model, _yti represents whether the molecular structure label is the i-th character at the t-th decoding time, and _pti represents the probability that the structure recognition result is the i-th character at the t-th decoding time.

The detection loss L _bc of the reconnection relationship, ie, the loss caused by the cycle, can be expressed based on the following formula:

q _tb =softmax(W ^m s _b + W ^o s _t );

Where s _t is the decoding state of the branch angle obtained by decoding at the current decoding time t. ^{W m} and W ^o are parameters for chemical bond detection. The chemical bond detection probability distribution between the state feature s _b of the existing branch angle b stored in the angle set and the branch angle t is expressed as Where N is the total number of chemical bond types, and N+1 includes N types of chemical bonds and the case where there are no chemical bonds.

B is the total number of branch angles in the angle set, z _tbi is whether the chemical bond type between the molecular structure label and the branch angle b is i at the tth decoding time, and q _tbi represents the probability that the chemical bond type i between the branch angle b is detected at the tth decoding time in the chemical bond detection result.

By combining the above two losses, the overall loss for the recognition model training can be obtained. The overall loss L here can be expressed as L _ce +L _bc , or as the result of the weighted sum of L _ce and L _bc , which is not specifically limited in the embodiments of the present disclosure. After obtaining the overall loss, the parameters of the initial model can be iterated based on the overall loss, thereby obtaining a fully trained recognition model.

The method provided in the embodiment of the present disclosure realizes molecular structure recognition for molecular images through end-to-end modeling and training.

Based on any of the above embodiments, the initial model is an encoder + decoder structure. Among them, the encoder is DenseNet, which includes three dense blocks for converting the input RGB three-channel image into high-dimensional features. The growth rate and depth of each dense block are set to 24 and 32, respectively. In addition, both the encoder and the decoder use a GRU with a hidden state dimension of 256 as the recurrent unit of the RNN, and the dimension of the attention projection parameter is set to 128. In addition, the dimension of the embedding parameter is set to 256, and a drop-out rate of 15% is applied. For the decoder, the projection parameter dimension of the chemical bond detection classification is set to 256.

The optimizer used for parameter iteration of the initial model can be Adam, the initial learning rate is 2e-4, the learning rate decay strategy is multi-step decay, Pytorch's MultiStepLR is used to adjust the learning rate, and the decay factor gamma is set to 0.5.

Based on any of the above embodiments, the disclosed embodiments provide a molecular structure recognition method, which is implemented based on a recognition model. The recognition model here includes a random conditional decoder different from a string decoder, and the random conditional decoder combines three mechanisms to solve the problems existing in the prior art: conditional guided attention, loop and path selection.

Among them, the conditionally guided attention mechanism can take advantage of the natural graph structure of molecular structures and regard its recognition process as a graph traversal problem. When the recognition model traverses the graph, it encounters multiple branch angles, and the order of these angles in the proposed modeling unit follows a fixed counterclockwise direction. However, if decoding is performed only according to a fixed angle order, the model may forget which angle units have not yet been decoded due to the extension of the later decoding time. To solve this problem, under the conditionally guided attention mechanism, the branch angle can be used as conditional information to guide the decoding process. When the recognition model encounters a branch, it continues decoding along the specified branch angle direction. The updated decoder no longer uses "(" and ")" to indicate the start and end of the branch. Instead, the branch angle of each branch is first predicted, that is, "\angle[:<angle value>]", and the decoded features of each branch angle obtained by decoding are stored separately into the angle set. Memory, that is, the context and attention weight information of each branch angle.

Subsequently, in the decoding process of the molecular structure at a branch angle, when "\eob" is decoded, it indicates that there are no additional branch angles to be predicted, and a decoding feature of a branch angle can be selected from the angle set Memory as a condition for continuing the decoding process.

In addition, the main difference between the graph structure and the tree structure is that the graph structure has a cyclic characteristic. In the molecular structure recognition method provided by the present disclosure embodiment, the branch angle corresponding to the loop has been stored in the angle set Memory and has not been decoded. Therefore, a simple multi-label classification module can be constructed for chemical bond detection to determine the direction corresponding to the loop angle and classify the type of loop bond (such as single bond, double bond, etc.). The result obtained by chemical bond detection may be empty or the type of bond, and empty means that there is no loop with a corresponding branch angle that has not been decoded. If there is a cyclic chemical bond in the test result, the corresponding branch angle is deleted from the angle set Memory, and the branch angle obtained by decoding at the current moment is not stored. On the contrary, if there is no loop, that is, the test result is empty, the branch angle at the current moment is saved in the angle set Memory for future decoding selection.

Finally, the recognition process of the random conditional decoder can be viewed as a graph traversal problem, where different traversals of the graph can produce multiple target sequences. Therefore, a single graph can have multiple training labels. Although the conditionally guided attention mechanism still decodes according to a fixed counterclockwise order, it may cause overfitting and recognition errors in complex or uncommon structures. Therefore, the disclosed embodiment can be combined with a path selection mechanism for decoding, which randomly selects different paths during the training process to improve the alignment between visual information and decoded characters using a conditionally guided attention mechanism. During the reasoning process, the recognition model can attempt to decode all branch angles stored in the angle set and participate in the calculation of the beam search path score PK, thereby selecting the path with the highest score to continue decoding.

Through the above method, the recognition model can have better versatility and recognition efficiency.

Furthermore, for the training of the recognition model, the disclosed embodiment uses the publicly accessible CASIA-CSDB dataset as a benchmark to establish a handwriting dataset for use as sample images. Real-world handwritten molecular structures are available from educational settings. The dataset includes many instances with writing errors and non-existent structure data.

Furthermore, the handwritten dataset consists of 52,987 images of handwritten molecular structures collected in educational scenarios. These images were obtained using various devices such as cameras, scanners, and screens, and were labeled as native Chemfig strings. In addition, there is a mixture of molecular structures and regular molecular formulas in this dataset. In addition to isolated isolated molecular structures, many instances in the dataset contain a combination of formulas and molecular structures. In addition, different structural writing styles are integrated into the dataset, such as the use/non-use of abbreviations, the type of Keckler ring representation of benzene, and the inclusion/exclusion of hydrogen atoms. The dataset also contains a large number of examples of human writing errors that violate chemical principles or even non-existent structures. The recognition of such structures by the model trained based on this can potentially be applied to correct and modify handwritten answers. In addition, in order to test the generalization performance of the model, the molecular structures in the dataset need to maintain a certain complexity. Specifically, the sum of the number of atoms and bonds can be used to evaluate the complexity level of the molecular structure. The complexity level of about 10% of the dataset exceeds the complexity level of the most complex sample in the regular training set.

Based on any of the above embodiments, after the recognition model training is completed, an experimental test may be performed on the recognition model.

Specifically, considering that the molecular structure label itself is concise enough, we can directly compare whether the string of the predicted recognition result is the same as the string of the label. We can convert both the string of the label and the string of the recognition result into molecular structure diagrams, and then compare whether the molecular structure diagrams of the two match. Assuming T is the number of samples and R is the number of recognition results that match the label, the Exact Match (EM) score can be calculated using the following formula:

In addition, considering that there is a mixture of formulas and molecular structures in the handwritten dataset, a single image may contain multiple molecular structures. Therefore, the present embodiment defines two auxiliary indicators: as follows:

Structure EM: For samples containing mixed molecular structures and rule formulas, when all molecular structure recognition results match the labeled graph, it is determined that the structure in the sample is correctly recognized. Let T be the number of samples, R _struct represents the number of correctly recognized structure samples, and the structure EM score is as follows:

The structural EM score EM _struct and the single EM score EM can measure the recognition performance of the recognition model for the molecular structures in the handwritten dataset under mixed mode.

From the experimental results, the molecular structure recognition method provided by the disclosed embodiment significantly improves the recognition performance compared with the SMILES-based recognition method. This is mainly due to the reduced ambiguity of the molecular structure label and the stronger consistency between the image and the label. In addition, the random conditional decoder proposed in the disclosed embodiment has significant advantages over the string decoder. It should be emphasized that the computational and parameter costs of the random conditional decoder are almost the same as those of the string decoder.

Based on any of the above embodiments, FIG6 is a schematic diagram of the structure of a molecular structure recognition device provided by the present disclosure. As shown in FIG6 , the device includes:

An acquisition unit 610, configured to acquire a molecular image;

The identification unit 620 is used to initialize an empty angle set, and when the molecular structure is decoded based on the image features of the molecular image and a branch angle is decoded for the first time, the branch angle is stored in the angle set; a branch angle is taken out from the angle set, and the molecular structure under the branch angle is decoded based on the image features of the molecular image with the branch angle as a guide, and the angle set is updated based on the new branch angle obtained by decoding for the molecular structure decoding under the next branch angle, until the angle set is empty;

The output unit 630 is used to determine the molecular structure corresponding to the molecular image based on the decoding results at each branch angle.

The device provided by the embodiment of the present disclosure adds a decoding mechanism for each branch angle in the chemical molecular structure and a maintenance mechanism for the branch angle to be explored, and uses the branch angle in the chemical molecular structure as a guiding condition during decoding to enrich the information of molecular structure decoding, improve the reliability of molecular structure decoding, and improve the decoding accuracy for complex chemical molecular structures.

Based on any of the above embodiments, the identification unit 620 includes a set updating unit, which is used to:

Perform chemical bond detection on the new branch angle and each existing branch angle in the angle set;

Based on the detection result of the chemical bond detection, the angle set is updated.

Based on any of the above embodiments, the set updating unit is specifically used for:

When the detection result indicates that there is no chemical bond, storing the new branch angle into the angle set;

In the case where the detection result indicates the presence of a chemical bond, the branch angle that forms a chemical bond with the new branch angle is deleted from the angle set.

Based on any of the above embodiments, the identification unit 620 includes a branch decoding unit, which is used to:

Determining visual context features at a current decoding moment at the branch angle based on the decoding features of the branch angle and a decoding state at a previous decoding moment at the branch angle;

Based on the visual context feature, the molecular structure of the image feature at the current decoding moment is decoded to obtain the decoding state at the current decoding moment, and the current decoding moment is returned as the previous decoding moment for decoding until the decoding at the branch angle is completed.

Based on any of the above embodiments, the identification unit 620 is specifically used for:

Based on the recognition model, a branch angle is taken from the angle set, and the molecular structure under the branch angle is decoded based on the image features of the molecular image with the branch angle as a guide, and the angle set is updated based on the new branch angle obtained by decoding for the next step. Decoding the molecular structure at a branch angle until the angle set is empty;

The recognition model is trained based on the sample image and the molecular structure label corresponding to the sample image;

The molecular structure label is obtained by connecting the atomic groups and chemical bonds in the molecular formula corresponding to the sample image into a molecular structure graph and then performing graph traversal.

Based on any of the above embodiments, the device further includes a label conversion unit, which is used to:

Connecting the atomic groups and chemical bonds in the molecular formula corresponding to the sample image into a molecular structure diagram;

The molecular structure graph is traversed, and the molecular structure label is generated based on the atomic groups, chemical bonds, angles, nesting symbols and reconnection marks obtained through the traversal.

Based on any of the above embodiments, the device further includes a model training unit, which is used to:

Based on the initial model, molecular structure recognition is performed on the sample image to obtain a chemical bond detection result between the sample branch angle decoded in the molecular structure recognition process and the existing branch angles in the sample angle set, as well as a structural recognition result of the sample image;

Based on the structure recognition result and the molecular structure label, as well as the chemical bond detection and the reconnection mark in the molecular structure label, the initial model is iterated with parameters to obtain the recognition model.

FIG7 illustrates a schematic diagram of the physical structure of an electronic device. As shown in FIG7 , the electronic device may include: a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communications interface 720, and the memory 730 communicate with each other via the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the molecular structure recognition method, which includes:

Acquire molecular images;

Initialize an empty angle set, and when the molecular structure is decoded based on the image features of the molecular image and the branch angle is decoded for the first time, store the branch angle in the The angle set; taking a branch angle from the angle set, using the branch angle as a guide, decoding the molecular structure under the branch angle based on the image features of the molecular image, and updating the angle set based on the new branch angle obtained by decoding for decoding the molecular structure under the next branch angle, until the angle set is empty;

Based on the decoding results at each branch angle, the molecular structure corresponding to the molecular image is determined.

In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

On the other hand, the present disclosure further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the molecular structure recognition method provided by the above methods, the method comprising:

Acquire molecular images;

Initialize an empty angle set, and when decoding the molecular structure based on the image features of the molecular image and decoding to a branch angle for the first time, store the branch angle in the angle set; take a branch angle from the angle set, and decode the molecular structure under the branch angle based on the image features of the molecular image with the branch angle as a guide, and update the angle set based on the new branch angle obtained by decoding for the next step. Decoding the molecular structure under the branch angle until the angle set is empty;

Based on the decoding results at each branch angle, the molecular structure corresponding to the molecular image is determined.

In another aspect, the present disclosure further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the molecular structure recognition method provided by the above methods, the method comprising:

Acquire molecular images;

Initialize an empty angle set, and when decoding the molecular structure based on the image features of the molecular image and decoding to a branch angle for the first time, store the branch angle in the angle set; take a branch angle from the angle set, and decode the molecular structure under the branch angle based on the image features of the molecular image with the branch angle as a guide, and update the angle set based on the new branch angle obtained by decoding for decoding the molecular structure under the next branch angle, until the angle set is empty;

Based on the decoding results at each branch angle, the molecular structure corresponding to the molecular image is determined.

The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, or of course by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM/RAM, a disk, an optical disk, or a computer-readable medium. The disk, etc., includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present disclosure.

Claims (16)

A molecular structure recognition method, comprising: Acquire molecular images; Initialize an empty angle set, and when decoding the molecular structure based on the image features of the molecular image and decoding to a branch angle for the first time, store the branch angle in the angle set; take a branch angle from the angle set, and decode the molecular structure under the branch angle based on the image features of the molecular image with the branch angle as a guide, and update the angle set based on the new branch angle obtained by decoding for decoding the molecular structure under the next branch angle, until the angle set is empty; Based on the decoding results at each branch angle, the molecular structure corresponding to the molecular image is determined. The molecular structure recognition method according to claim 1, wherein the updating of the angle set based on the new branch angle obtained by decoding comprises: Perform chemical bond detection on the new branch angle and each existing branch angle in the angle set; Based on the detection result of the chemical bond detection, the angle set is updated. The molecular structure recognition method according to claim 2, wherein the updating of the angle set based on the detection result of the chemical bond detection comprises: When the detection result indicates that there is no chemical bond, storing the new branch angle into the angle set; In the case where the detection result indicates the presence of a chemical bond, the branch angle that forms a chemical bond with the new branch angle is deleted from the angle set. The molecular structure recognition method according to claim 1, wherein the decoding of the molecular structure at the branching angle based on the image features of the molecular image guided by the branching angle comprises: Based on the decoding features of the branch angle and the previous solution under the branch angle The decoding state at the decoding moment determines the visual context features at the current decoding moment under the branch angle; Based on the visual context feature, the molecular structure of the image feature at the current decoding moment is decoded to obtain the decoding state at the current decoding moment, and the current decoding moment is returned as the previous decoding moment for decoding until the decoding at the branch angle is completed. The molecular structure recognition method according to any one of claims 1 to 4, wherein taking out a branch angle from the angle set, decoding the molecular structure at the branch angle based on the image features of the molecular image with the branch angle as a guide, and updating the angle set based on the new branch angle obtained by decoding for decoding the molecular structure at the next branch angle until the angle set is empty, comprises: Based on the recognition model, a branch angle is taken out from the angle set, and the molecular structure under the branch angle is decoded based on the image features of the molecular image with the branch angle as a guide, and the angle set is updated based on the new branch angle obtained by decoding for decoding of the molecular structure under the next branch angle, until the angle set is empty; The recognition model is trained based on the sample image and the molecular structure label corresponding to the sample image; The molecular structure label is obtained by connecting the atomic groups and chemical bonds in the molecular formula corresponding to the sample image into a molecular structure graph and then performing graph traversal. The molecular structure recognition method according to claim 5, wherein the step of determining the molecular structure tag comprises: Connecting the atomic groups and chemical bonds in the molecular formula corresponding to the sample image into a molecular structure diagram; The molecular structure graph is traversed, and the molecular structure label is generated based on the atomic groups, chemical bonds, angles, nesting symbols and reconnection marks obtained through the traversal. The molecular structure recognition method according to claim 5, wherein the step of training the recognition model comprises: Based on the initial model, molecular structure recognition is performed on the sample image to obtain a chemical bond detection result between the sample branch angle decoded in the molecular structure recognition process and the existing branch angles in the sample angle set, as well as a structural recognition result of the sample image; Based on the structure recognition result and the molecular structure label, as well as the chemical bond detection and the reconnection mark in the molecular structure label, the initial model is iterated with parameters to obtain the recognition model. A molecular structure recognition device, comprising: An acquisition unit, used for acquiring a molecular image; The identification unit is used to initialize an empty angle set, and when decoding the molecular structure based on the image features of the molecular image and decoding to a branch angle for the first time, store the branch angle in the angle set; take out a branch angle from the angle set, and decode the molecular structure under the branch angle based on the image features of the molecular image with the branch angle as a guide, and update the angle set based on the new branch angle obtained by decoding for decoding of the molecular structure under the next branch angle, until the angle set is empty; The output unit is used to determine the molecular structure corresponding to the molecular image based on the decoding results at each branch angle. The molecular structure recognition device according to claim 8, wherein the updating of the angle set based on the new branch angle obtained by decoding comprises: Perform chemical bond detection on the new branch angle and each existing branch angle in the angle set; Based on the detection result of the chemical bond detection, the angle set is updated. The molecular structure recognition device according to claim 9, wherein the updating of the angle set based on the detection result of the chemical bond detection comprises: When the detection result indicates that there is no chemical bond, storing the new branch angle into the angle set; In the case where the detection result indicates the presence of a chemical bond, the new branch angle Branch angles that form chemical bonds are deleted from the angle set. The molecular structure recognition device according to claim 8, wherein the decoding of the molecular structure at the branching angle based on the image features of the molecular image guided by the branching angle comprises: Determining visual context features at a current decoding moment at the branch angle based on the decoding features of the branch angle and a decoding state at a previous decoding moment at the branch angle; Based on the visual context feature, the molecular structure of the image feature at the current decoding moment is decoded to obtain the decoding state at the current decoding moment, and the current decoding moment is returned as the previous decoding moment for decoding until the decoding at the branch angle is completed. The molecular structure recognition device according to any one of claims 8 to 11, wherein taking out a branch angle from the angle set, decoding the molecular structure at the branch angle based on the image features of the molecular image with the branch angle as a guide, and updating the angle set based on the new branch angle obtained by decoding for decoding the molecular structure at the next branch angle until the angle set is empty, comprises: Based on the recognition model, a branch angle is taken out from the angle set, and the molecular structure under the branch angle is decoded based on the image features of the molecular image with the branch angle as a guide, and the angle set is updated based on the new branch angle obtained by decoding for decoding of the molecular structure under the next branch angle, until the angle set is empty; The recognition model is trained based on the sample image and the molecular structure label corresponding to the sample image; The molecular structure label is obtained by connecting the atomic groups and chemical bonds in the molecular formula corresponding to the sample image into a molecular structure graph and then performing graph traversal. The molecular structure recognition device according to claim 12, wherein the step of determining the molecular structure tag comprises: Connect the atomic groups and chemical bonds in the molecular formula corresponding to the sample image into molecules Structural diagram; The molecular structure graph is traversed, and the molecular structure label is generated based on the atomic groups, chemical bonds, angles, nesting symbols and reconnection marks obtained through the traversal. The molecular structure recognition device according to claim 12, wherein the step of training the recognition model comprises: Based on the initial model, molecular structure recognition is performed on the sample image to obtain a chemical bond detection result between the sample branch angle decoded in the molecular structure recognition process and the existing branch angles in the sample angle set, as well as a structural recognition result of the sample image; Based on the structure recognition result and the molecular structure label, as well as the chemical bond detection and the reconnection mark in the molecular structure label, the initial model is iterated with parameters to obtain the recognition model. An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the molecular structure recognition method as claimed in any one of claims 1 to 7 when executing the program. A non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the molecular structure recognition method as claimed in any one of claims 1 to 7.