CN111613267A - A CRISPR/Cas9 off-target prediction method based on attention mechanism - Google Patents

Abstract

The invention relates to a CRISPR/Cas9 off-target prediction method based on an attention mechanism, which relates to the field of gene editing and comprises the following steps: constructing an off-target data set, encoding sgRNA-DNA sequence pairs, building an off-target prediction model, training model weights, and predicting whether the sgRNA-DNA sequence pairs contain off-target sites by using the trained model. The invention provides a novel CRISPR/Cas9 off-target prediction model based on an attention mechanism by analyzing the composition and the matching condition of a sgRNA-DNA sequence pair and the relation between the off-target problem, wherein the model takes a three-layer convolution neural network as a base framework, and the attention mechanism is introduced to extract the characteristics related to position information in the sgRNA-DNA sequence pair. The model can extract abundant characteristics related to the sgRNA-DNA miss-detection problem, can more accurately and comprehensively predict the CRISPR/Cas9 miss-detection problem compared with the existing prediction algorithm, and has better generalization performance.

Description

A CRISPR/Cas9 off-target prediction method based on attention mechanism

technical field

The invention relates to the fields of gene editing and deep learning, in particular to a CRISPR/Cas9 off-target prediction method based on an attention mechanism.

Background technique

The genome editing technology mediated by the CRISPR/Cas9 (clustered regularly interspaced short palindromic repeat/CRISPR-associa-tedprotein 9) system is the third-generation "genome site-directed editing technology", which can be used to modify specific genomic DNA targets. Compared with the previous two generations of zinc-finger nucleases (zinc-finger nucleases, ZFNs) and transcription activator-like effector nucleases (TALENs) technology, CRISPR/Cas9 technology has a wide range of applications, low cost and Ease of operation and so on. Its applications mainly include targeted mutagenesis of model organisms, knockout or knock-in of genes to verify gene function and epigenome editing, delivery of base editing enzymes to target sites, etc. In addition to this, the technology also has great potential for in vivo gene therapy. However, since the current delivery methods of CRISPR/Cas9 in vivo are not tissue-specific, CRISPR/Cas9 may undergo gene editing in non-target tissues and cause on-target side effects, which makes CRISPR/Cas9 gene therapy applications still facing a challenge. series of security challenges. Therefore, accurate prediction of off-target problems of CRISPR/Cas9 is of great significance for its application.

At present, there are many methods for predicting CRISPR/Cas9 off-target problems, which can be mainly divided into methods based on sequence alignment, methods based on hypothesis experience, sequencing methods and algorithms based on machine learning. Empirical and machine learning-based approaches. In recent years, deep learning has also been gradually used to predict the off-target problem of CRISPR/Cas9, and has achieved some results, but still faces the problem of low prediction accuracy. Since the application of deep learning can automatically extract the features of sgRNA-DNA sequence pairs, the operation is relatively simple and the performance of the algorithm has a large room for improvement. Therefore, further research can be done to improve the efficiency of CRISPR/Cas9 off-target prediction by applying deep learning.

SUMMARY OF THE INVENTION

The purpose of the present invention is to overcome the defects of the existing CRISPR/Cas9 off-target prediction methods, and propose a CRISPR/Cas9 off-target prediction method based on an attention mechanism. Since the input of the off-target prediction method is the sgRNA-DNA sequence pair, the algorithm mainly judges whether off-target phenomenon will occur by paying attention to the matching of the bases in the sgRNA sequence and the DNA sequence at each position, which is closely related to the position information, and the attention The mechanism can effectively promote the extraction of internal local location features from sequences. Therefore, this method takes the convolutional neural network as the base framework, and introduces the attention mechanism to strengthen the extraction of the location features of the sgRNA-DNA sequence, which can effectively improve the accuracy and recall rate of off-target prediction, and has high generalization. performance, which has important implications for the safe application of CRISPR/Cas9.

For achieving the above object, the technical scheme provided by the present invention is:

A CRISPR/Cas9 off-target prediction method based on attention mechanism mainly includes the following steps:

1) Construct datasets for model training and testing that contain sgRNA, DNA and their off-target tags, including the published HEK293T and K562t datasets;

2) Use a specific encoding method to encode the sgRNA-DNA sequence pair in the sample data set, so that it can be used as the input of the neural network;

3) Build a convolutional neural network model based on an attention mechanism, including a one-dimensional convolutional layer and an independent attention module;

4) Selecting an optimizer and a loss function, setting the parameters of the optimizer, the learning rate and the number of iterations, and using the self-service sampling method to train the model in step 3) to obtain the weight of the trained model;

5) Input the sgRNA-DNA sequence pair to be tested into the model trained in step 4) to obtain the off-target prediction value of the sgRNA-DNA sequence pair.

Further, the data set in described step 1) is constituted as follows:

In HEK293T, the number of sgRNA-DNA sequence pairs with off-target sites (positive samples) and sgRNA-DNA without off-target sites (negative samples) were 536 and 132,378, respectively; while in dataset K562t, the number of negative samples There are 20199, and there are 120 positive samples. The sum of the number of samples in the two datasets is 153233, and the ratio of positive and negative samples is approximately 1:233, where positive samples are marked with number 1 and negative samples are marked with number 0.

Further, described step 2) comprises the following steps:

21) Define a dictionary of base pairs, which contains the mapping of all 16 different base pairs synthesized in two groups of four different bases A, T, C, and G and their corresponding values, specifically: { AA:0, AT:1, AC:2, AG:3, TA:4, TT:5, TC:6, TG:7, CA:8, CT:9, CC:10, GG:11, GA: 12, GT: 13, GC: 14, GG: 15};

22) look up the numerical value corresponding to the nucleotide pair of each position in the sgRNA-DNA sequence pair (23bp) in the dictionary in step 21), and complete the numerical encoding of the sgRNA-DNA sequence pair;

23) For the coding sequence x ^1×23 obtained in step 22), use the Embedding layer in the Keras framework to further encode the word vector. The Embedding layer is actually a neural network containing a hidden layer, and the hidden layer vector is h ^{1× N} , the specific encoding method is as follows:

x ^1×23 ·W ^23×N =h ^1×N

Among them, W ^23×N is the weight matrix between the input layer and the hidden layer, N is the custom encoding size, and the final word vector encoding result is W ^23×N .

Further, described step 3) comprises the steps:

31) One-dimensional convolution is performed on the coding sequence obtained in step 23), the kernel size is 5, and the number of kernels is 20 to obtain the C1 layer;

For the C1 layer, normalize it to get the B1 layer;

For the B1 layer, perform a one-dimensional convolution with a kernel size of 5 and a kernel number of 40 to obtain the C2 layer;

For the C2 layer, normalize it to get the B2 layer;

For the B2 layer, perform a one-dimensional convolution with a kernel size of 5 and a kernel number of 80 to obtain the C3 layer;

For the C3 layer, normalize it to get the B3 layer;

For layer B1, perform linear scale transformation to make it consistent with the size of layer B3 to obtain layer C11;

32) Input the C11 and C3 layers obtained in step 31) into the attention module, strengthen the extraction of local location features within the sgRNA-DNA sequence pair, and obtain the output A1 layer of the attention module;

33) The feature map of the A1 layer in step 32) is passed through a fully connected layer with a size of 40 to obtain the E1 layer;

For the E1 layer, perform a random deactivation operation with a parameter of 0.2 to obtain the D1 layer;

For the D1 layer, pass it through the fully connected layer of size 20 to get the E2 layer;

For the E2 layer, perform a random deactivation operation with a parameter of 0.2 to obtain the D2 layer;

34) The feature mapping of the D2 layer in step 33) is the feature of the sgRNA-DNA sequence pair finally extracted by the model;

35) For the D2 layer, pass it through a fully connected layer with a size of 2 and an activation function of softmax to obtain a one-dimensional vector containing two elements, the two elements represent the probability of missing the target and the probability of hitting the target correctly.

Further, the self-service sampling method in described step 4) comprises the steps:

41) The negative samples in the training set are randomly divided into N equal parts according to the method of each 256 pieces of data, N=the total number of negative samples/256;

42) Randomly extract 256 samples from the positive samples in the training set, and combine them with a negative sample in (1) to form a small training set;

43) Repeating step 42) N times, a total of N balanced small training sets can be obtained;

44) In one iteration, a small training set is used each time for training, which needs to be divided into N times.

Further, the step 32) includes the following steps:

321) Calculate the attention score:

score _i =U·(W ₁ q+W ₂ k _i )

Among them, U, W ₁ and W ₂ are all randomly initialized matrices, q is the vector of a column in the output matrix of the C3 layer, _{ki is the vector of a column in the output matrix of the C11 layer, and score i is} the value between _ki and q Attention score, which represents the correlation between the two;

322) Calculate the output of the attention module:

where q' is the output of the attention module after being processed by the attention algorithm.

Compared with the prior art, the present invention has the following advantages and beneficial technical effects:

1. The present invention uses a convolutional neural network as the base framework, and introduces an attention mechanism on this basis, which can strengthen the extraction of features related to position information within the sgRNA-DNA sequence pair, and improve the accuracy of CRISPR/Cas9 off-target prediction rate and recall rate;

2. When predicting the off-target problem of CRISPR/Cas9, the present invention achieves a good balance between the accuracy rate and the recall rate;

3. The model structure of the present invention is relatively simple and has better generalization performance;

4. The present invention uses the convolutional neural network as the base frame, which has low time complexity and high practical performance.

Description of drawings

FIG. 1 is a flow chart of an implementation example of the method of the present invention.

Fig. 2 is a model structure diagram of the method of the present invention.

Detailed ways

In order to facilitate the understanding and implementation of the present invention by those of ordinary skill in the art, the present invention will be further described below with reference to the embodiments. It should be understood that the embodiments described herein are only used to illustrate and explain the present invention, but not to limit the present invention.

The present invention adopts the published HEK293T and K562t data sets, and the numbers of positive samples and negative samples in HEK293T are 536 and 132,378 respectively; while in the data set K562t, there are 20,199 negative samples and 120 positive samples. The sum of the number of samples in the two datasets is 153233, and the ratio of positive and negative samples is about 1:233.

The invention proposes a CRISPR/Cas9 off-target prediction method based on an attention mechanism. Before using a neural network model to predict whether an unknown sgRNA-DNA sequence pair contains an off-target site, the constructed neural network model needs to be trained. Use the trained model to predict off-target sgRNA-DNA sequence pairs. Refer to Figure 1 for the main process, including the following steps:

1) Preprocess the labels of the dataset, mark the samples containing off-target sites as positive samples, and use the number 1 as the label; the samples that do not contain off-target sites are recorded as negative samples, with the number 0 as the label.

2) Using a specific encoding method to encode the sgRNA-DNA sequence pair in the sample data set, with reference to Figure 2, the specific steps are as follows:

21) Define a dictionary of base pairs, which contains the mapping of all 16 different base pairs synthesized in two groups of four different bases A, T, C, and G and their corresponding values, specifically: { AA:0, AT:1, AC:2, AG:3, TA:4, TT:5, TC:6, TG:7, CA:8, CT:9, CC:10, GG:11, GA: 12, GT: 13, GC: 14, GG: 15};

22) look up the numerical value corresponding to the nucleotide pair of each position in the sgRNA-DNA sequence pair (23bp) in the dictionary in step 21), and complete the numerical encoding of the sgRNA-DNA sequence pair;

23) For the coding sequence x ^1×23 obtained in step 22), use the Embedding layer in the Keras framework to further encode the word vector. The Embedding layer is actually a neural network containing a hidden layer, and the hidden layer vector is h ^{1× N} , the specific encoding method is as follows:

x ^1×23 ·W ^23×N =h ^1×N

Among them, W ^23×N is the weight matrix between the input layer and the hidden layer, N is the custom encoding size, and the final word vector encoding result is W ^23×N .

3) For each data set, it is divided into training set and test set according to the ratio of 8:2, and the training set is used to train the weight of the model.

4) Input the encoded sgRNA-DNA sequence pair into the off-target prediction model for feature extraction, referring to Figure 2, including the following steps:

41) After the output of the Embedding layer is subjected to a one-dimensional convolution with a kernel number of 20 and a kernel size of 5, the first feature map C1 layer is obtained;

42) Normalize the output of the C1 layer to obtain the B1 layer;

43) After the output of the B1 layer is subjected to a one-dimensional convolution with a kernel number of 40 and a kernel size of 5, the second feature map C2 layer is obtained;

44) Normalize the output of the C2 layer to obtain the B2 layer;

45) After the output of the B2 layer is subjected to a one-dimensional convolution with a kernel number of 80 and a kernel size of 5, the third feature map C3 layer is obtained;

46) Normalize the output of the C3 layer to obtain the B3 layer;

47) Perform a linear scale transformation on the output of the B1 layer to make the dimension consistent with the B3 layer to obtain the C11 layer;

48) Calculate the attention mechanism on the outputs of the C11 layer and the B3 layer to obtain the output A1 layer of the attention module;

49) Flatten the output of the A1 layer to make it a one-dimensional vector,;

50) Integrate the feature vector in step 49) through two fully connected layers successively, each fully connected layer includes a Dense layer and a Dropout layer, and the sizes of the front and rear Dense layers are 40, 20, and Dropout layers respectively. The parameters are all 0.2, and finally the feature map D2 layer extracted from the entire model is obtained.

5) Fitting off-target tags of sgRNA-DNA sequence pairs using the extracted feature maps to train the model.

Follow the above steps to build a training model and conduct experiments. In the model training process, set the optimizer of the model to Adam, the initial learning rate is 0.003, the decline rate is 0.2, the batch in training is 512, and the number of iterations is 30. After the training is completed, on the divided test set After the test is completed, the average performance of the method on the test set is 11% higher than that of the existing optimal prediction algorithm. It can be seen that the method of the present invention can improve the overall performance of CRISPR/Cas9 off-target prediction, and has a relatively high performance. High generalization ability.

To sum up, the present invention focuses on the CRISPR/Cas9 off-target prediction algorithm based on the attention mechanism, aiming at the CRISPR/Cas9 off-target problem. This method uses a three-layer convolutional neural network as the base framework, and introduces an attention mechanism to enhance the extraction of location-related features of sgRNA-DNA sequence pairs. On the one hand, this method improves the prediction performance of CRISPR/Cas9 off-target problems, and on the other hand, it has high generalization performance and high practicability.

The above descriptions are only preferred embodiments of the present application, and the present invention is not limited to the above embodiments. It can be understood that other improvements and changes directly derived or imagined by those skilled in the art without departing from the spirit and concept of the present invention should be considered to be included within the protection scope of the present invention.

Claims (5)

1. An attention-based CRISPR/Cas9 off-target prediction method, which is characterized by comprising the following steps:

1) constructing a data set containing sgRNA, DNA and off-target labels thereof for model training and testing, wherein the data set comprises two published data sets of HEK293T and K562 t;

2) encoding the sgRNA-DNA sequence pairs in the sample data set by using a specific encoding method so that the sgRNA-DNA sequence pairs can be used as input of a neural network;

3) building a convolutional neural network model based on an attention mechanism, wherein the convolutional neural network model comprises a one-dimensional convolutional layer and an independent attention module;

4) selecting an optimizer and a loss function, setting parameters of the optimizer, a learning rate and iteration times, and training the model in the step 3) by adopting a self-service sampling method to obtain the weight of the trained model;

5) inputting the sgRNA-DNA sequence pairs to be detected into the trained model in the step 4) to obtain the off-target prediction value of the sgRNA-DNA sequence pairs.

2. The CRISPR/Cas9 off-target prediction method based on the attention mechanism as claimed in claim 1, wherein the coding method selected in the step 2) is word vector coding, comprising the following steps:

21) defining a base pair dictionary, wherein the dictionary comprises mapping of all 16 different base pairs formed by pairwise combination of A, T, C, G four different base pairs and corresponding numerical values, and specifically comprises the following steps: { AA:0, AT:1, AC:2, AG:3, TA:4, TT:5, TC:6, TG:7, CA:8, CT:9, CC:10, GG:11, GA:12, GT:13, GC:14, GG:15 };

22) searching a dictionary in the step 21) for a numerical value corresponding to the nucleotide pair at each position in the sgRNA-DNA sequence pair (23bp) to complete numerical value coding of the sgRNA-DNA sequence pair;

23) for the coding sequence x obtained in step 22)^1×23The method utilizes an Embedding layer in a Keras framework to further carry out word vector coding, wherein the Embedding layer is actually a neural network comprising a hidden layer, and the hidden layer vector is h^1×NThe specific encoding method is as follows:

x^1×23·W^23×N＝h^1×N

wherein W^23×NIs a weight matrix between the input layer and the hidden layer, N is a self-defined coding size, and the finally obtained word vector coding result is W^23×N。

3. The attention mechanism-based CRISPR/Cas9 off-target prediction method according to claim 1, wherein the model building in the step 3) comprises the following steps:

31) performing one-dimensional convolution on the coding sequence obtained in the step 23), wherein the kernel size is 5, and the kernel number is 20, so as to obtain C1 layers; for the C1 layer, carrying out normalization treatment on the C1 layer to obtain a B1 layer;

for the B1 layer, performing one-dimensional convolution with the kernel size of 5 and the kernel number of 40 to obtain a C2 layer;

for the C2 layer, carrying out normalization treatment on the C2 layer to obtain a B2 layer;

for the B2 layer, performing one-dimensional convolution with the kernel size of 5 and the kernel number of 80 to obtain a C3 layer;

for the C3 layer, carrying out normalization treatment on the C3 layer to obtain a B3 layer;

for the B1 layer, performing linear scale transformation to make the B1 layer consistent with the size of the B3 layer, and obtaining a C11 layer;

32) inputting the C11 and B3 layers obtained in the step 31) into an attention module, enhancing the extraction of the internal local position characteristics of the sgRNA-DNA sequence, and obtaining an output A1 layer of the attention module;

33) mapping the characteristics of the A1 layer in the step 32) through a full connection layer with the size of 40 to obtain an E1 layer;

for the E1 layer, carrying out random inactivation operation with the parameter of 0.2 to obtain a D1 layer;

for the D1 layer, it was passed through a fully connected layer of size 20, resulting in an E2 layer;

for the E2 layer, carrying out random inactivation operation with the parameter of 0.2 to obtain a D2 layer;

34) the feature mapping of the layer D2 in the step 33) is the features of the sgRNA-DNA sequence pairs finally extracted by the model;

35) for layer D2, it is passed through a fully connected layer of size 2 and activation function softmax to obtain a one-dimensional vector containing two elements representing miss probability and correct hit probability, respectively.

4. The method for predicting CRISPR/Cas9 miss based on attention mechanism according to claim 1, wherein the training model of the self-help sampling method in the step 4) comprises the following steps:

41) dividing the negative samples in the training set into N equal parts randomly according to the method of each 256 pieces of data, wherein N is the total number of the negative samples/256;

42) randomly drawing 256 samples from positive samples in the training set, and combining the 256 samples with one negative sample in the step (1) to form a small training set;

43) repeating the step 42) N times to obtain N balanced small training sets;

44) in one iteration, each time, a small training set is used for training, and the training needs to be carried out in N times.

5. The attention mechanism-based CRISPR/Cas9 off-target prediction method according to claim 1, the attention module implementation in step 32) comprising the steps of:

321) calculating the attention score:

score_i＝U·(W₁q+W₂k_i)

wherein U, W₁And W₂All are randomly initialized matrixes, q is a vector of a certain column in an output matrix of the B3 layer, and k_iIs a vector of a certain column in the C11 layer output matrix, score_iIs k_iAnd q, characterizing the degree of correlation between the two;

322) compute attention module output:

wherein q' is the output of the attention module obtained after processing by the attention algorithm.

Images