CN116132081A - Software defined network DDOS attack cooperative defense method based on ensemble learning - Google Patents

Abstract

The invention relates to the technical field of intrusion detection, in particular to a software defined network DDOS attack cooperative defense method based on ensemble learning, which firstly provides a Bagging integration-based feature selection algorithm, and avoids the condition that a single feature selection algorithm ignores the inter-feature connection and tends to select redundant features; secondly, introducing paired diversity measurement for selective integration, selecting a superior heterogeneous integration model, and adopting a layering ten-fold cross validation method to avoid overfitting; and finally, integrating a base classifier model by adopting a weighted voting mechanism based on a Bagging integration algorithm, embedding the model into an SDN controller, setting a detection time interval and realizing real-time monitoring.

Description

Collaborative defense method for DDOS attacks in software-defined networks based on ensemble learning

Technical Field

The present invention relates to the technical field of intrusion detection, and in particular to a software defined network DDOS attack collaborative defense method based on ensemble learning.

Background Art

In recent years, with the rapid development of the Internet, the scale of the network has become increasingly large, and the pressure on traditional networks has increased. Software Defined Networking (SDN) uses a programmable method to achieve the separation of forwarding and control, which can effectively manage network device resources, and embodies the controller-centric management model. It plays an important role in the development of cloud computing networks. However, this model will bring corresponding attack opportunities to various malicious applications. Although OpenFlow can provide some flow-based security detection methods, its underlying assumption is that the application software in the SDN is not maliciously attacked or the northbound interface is not damaged. This will play a certain security protection role for SDN, but it is helpless against DDOS attacks. Based on the fine-grained control of data flows by SDN, the controller can be used to conveniently perform packet filtering, rate limiting and attack tracing in DDOS attacks. However, the traditional classification algorithm uses a relatively single feature dimensionality reduction algorithm for feature dimensionality reduction, without considering the connection between features, and this method only uses a single algorithm to evaluate the model, resulting in weak model stability and redundant features that waste computing resources. Secondly, only a single classification algorithm is used as support in the design of the detection algorithm, or only the accuracy value is used as the weighting indicator of the integrated classifier, which leads to biased classification results.

Summary of the invention

The purpose of the present invention is to provide a software-defined network DDOS attack collaborative defense method based on ensemble learning, which avoids the situation that the single feature selection algorithm in the existing defense method ignores the connection between features, and adopts a layered ten-fold cross-validation method to avoid the overfitting problem.

To achieve the above object, the present invention provides a software defined network DDOS attack collaborative defense method based on ensemble learning, comprising the following steps:

Start the SDN controller to collect traffic data and store the traffic data in a CSV file;

Put the collected data into the Double-Bagging detection model for training;

The integrated model generated by training is embedded in the SDN controller, and the DDOS attack detection module is started.

Among them, the traffic data includes data of normal traffic and attack traffic, and is composed of the speed of the IP source, the traffic count, the speed of the traffic table item and the traffic comparison value. The values of the speed of the IP source, the traffic count, the speed of the traffic table item and the traffic comparison value are lower in normal traffic than in attack traffic.

The establishment process of the Double-Bagging detection model includes the following steps:

Perform data preprocessing and feature dimension reduction, and obtain the optimal feature subset through the feature subset selection voting mechanism based on the bagging ensemble algorithm;

The optimal feature subset is input into the base classifier for training, and a high-quality heterogeneous ensemble learning base classifier model is selected;

A weighted voting mechanism based on the Bagging ensemble algorithm is used to select the ensemble of base classifier models.

Among them, feature dimensionality reduction adopts a dimensionality reduction algorithm that combines filtering method and embedding method, and selects two feature selection algorithms in the filtering method, the chi-square verification and mutual information algorithm, and the extreme random tree algorithm in the embedding method to calculate the ranking of feature contributions.

The process of obtaining the optimal feature subset is to determine the number of results of each feature selection algorithm by setting a threshold, generate the result feature subset according to the feature contribution ranking, and use a voting strategy to select the feature subset with the largest number of occurrences as the optimal feature subset.

In the process of inputting the optimal feature subset into the base classifier for training and selecting a high-quality heterogeneous ensemble learning base classifier model, the diversity metric in ensemble learning is introduced to evaluate the combination effect between base classifiers, and the Bayesian optimization parameters are used to complete the first base classifier filtering by calculating the accuracy and area under the curve of the model.

The weight of the weighted voting mechanism is determined by the accuracy and the area under the curve value, and the area under the curve value is obtained by calculating the receiver operating characteristic curve of the base classifier.

Among them, after starting the DDOS attack detection module, when normal traffic is generated, the Double-Bagging model algorithm predicts it as normal traffic. When attack traffic is generated, it is immediately detected as a DDOS attack and the port it enters is blocked.

The present invention provides a software defined network DDOS attack collaborative defense method based on ensemble learning. Firstly, a feature selection algorithm based on bagging ensemble is proposed to avoid the situation that a single feature selection algorithm ignores the connection between features and tends to select redundant features. Secondly, a pairwise diversity metric is introduced for selective integration to select a relatively excellent heterogeneous integration model, and a layered ten-fold cross validation method is used to avoid overfitting. Finally, a weighted voting mechanism based on bagging ensemble algorithm is used to integrate base classifier models, and the model is embedded in an SDN controller, and a detection time interval is set to realize real-time monitoring.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

FIG1 is a flow chart of a software defined network DDOS attack collaborative defense method based on ensemble learning according to the present invention.

FIG2 is a flowchart of a specific implementation of the software-defined network DDOS attack collaborative defense method based on ensemble learning of the present invention.

Fig. 3 is a flow chart of the Double-Bagging algorithm of the present invention;

FIG4 is a flow chart of the feature selection voting mechanism based on Bagging of the present invention.

DETAILED DESCRIPTION

Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

The following are some nouns and terminology descriptions of the present invention:

Software Defined Networking (SDN);

Receiver operating characteristic (ROC);

Area Under Curve (AUC).

Referring to FIG. 1 , the present invention provides a software defined network DDOS attack collaborative defense method based on ensemble learning, comprising the following steps:

S1: Start the SDN controller to collect traffic data and store the traffic data in a CSV file;

S2: Put the collected data into the Double-Bagging detection model for training;

S3: Embed the integrated model generated by training into the SDN controller and start the DDOS attack detection module.

The overall design of the collaborative defense method for DDOS attacks in software-defined networks based on ensemble learning is shown in Figure 2. The design process of the collaborative defense model includes three aspects: traffic collection module, algorithm model selection module, and detection and defense mechanism design. The model is embedded in the Ryu controller, and the detection time interval is set to achieve real-time monitoring. The detailed implementation process is as follows:

S1: Start the SDN controller to collect traffic data and store the traffic data in a CSV file, wherein normal traffic data is collected first and then attack traffic data is collected for use in Double-Bagging model algorithm training;

The characteristics and parameters that need to be monitored and collected are as follows:

(1) Speed of IP sources: This characteristic gives the total number of TP sources entering the network within a specific time interval. Abbreviated as SSIP, it is defined as formula 1:

Where SumIPsrc is the total number of IP sources entering each flow, and T is the sampling time interval. The detection system monitors traffic and collects data every T seconds, and saves the number of source IPs in that time period. The controller needs to have enough normal traffic and attack traffic data for the machine learning algorithm to predict attacks. For normal attacks, SSIP is usually low, while for attacks, the count is usually high.

(2) Traffic count: Each traffic entering the network has a specific traffic count. Normal traffic is less than DDOS attack traffic.

(3) Flow table speed: The total number of flow table entries in the network switches in a certain time interval, Flow _N. Abbreviated as SFE, it is defined as Formula 2:

This is a very relevant feature for attack traffic detection, since in case of DDOS attack the number of flow table entries in a fixed time interval can increase significantly compared to the rate value of flow table entries for normal traffic.

(4) Traffic comparison value: refers to the total number of traffic entries flowing into the switch in the T time period, Flow _N , that is, the number of interactive IPs divided by the total traffic. It is abbreviated as RPF and is defined as Formula 3:

Where srcIP is the total number of collaborative IPs in the network flow. In the case of normal traffic, the IP source of the i-th flow is the same as the destination IP of the j-th flow, and the IP source of the j-th flow is the same as the destination IP of the i-th flow. This indicates an interactive flow, which is not the case when it is a DDOS attack traffic. When under attack, the flow table entries with a time T to reach the target host increase rapidly, and the target host cannot respond. Therefore, when the DDOS attack starts, the attack traffic decreases suddenly. Dividing the total number of collaborative flows by the total traffic makes this detection parameter scalable to networks under different operating conditions.

S2: Put the collected data into the Double-Bagging detection model for training;

The steps of establishing the Double-Bagging detection model are shown in Figure 3. First, a voting mechanism based on Bagging is proposed to select the optimal feature subset. Secondly, the ten-fold cross-validation method is used to detect anomalies in the data set and select the base classifier model. Finally, a weighted voting mechanism based on the Bagging integration algorithm is used to integrate the base classifier models. For the weighting of the base classifier, it is proposed to use the combination of accuracy and AUC value as the weight to integrate the classifier, which effectively solves the problem of uneven weighting.

Step S2.1, perform data preprocessing and feature dimensionality reduction. First, perform data standardization, and then perform feature dimensionality reduction. The present invention proposes a dimensionality reduction algorithm that combines filtering method and embedding method, which can avoid feature selection bias as much as possible and make up for the shortcomings of each single feature selection algorithm. The framework diagram is shown in Figure 4. First, the feature contribution (weight) is calculated for the two feature selection algorithms in the filtering method, the chi-square verification and the mutual information algorithm, and the extreme random tree algorithm in the embedding method.

The feature contribution calculation algorithm of chi-square verification is shown in formula 4:

Where c represents the degrees of freedom, O represents the observed value, and E represents the expected value.

The feature contribution calculation algorithm of the mutual information algorithm is shown in Formula 5:

Among them, p(x) represents the probability of X=xi occurring, p(y) represents the probability of Y=yi occurring, and p(x,y) represents the probability of X=xi and Y=yi occurring at the same time, that is, the joint probability.

The extreme random tree feature contribution calculation algorithm is shown in Formula 6:

k represents k categories, and Pk represents the sample weight of category k.

The feature contributions (weights) of the three algorithms are then ranked. The larger the weight, the more important the feature. Each feature selection algorithm assigns weights to each feature according to its own algorithm criteria and generates its own feature weight distribution. According to the feature weight ranking of the three feature selection algorithms, the voting mechanism based on the bagging ensemble algorithm is used to select the optimal feature subset. This mechanism determines the number of results of each method through a set threshold, generates multiple result feature subsets for the feature contribution ranking of the three methods, and finally uses the voting strategy to select the feature subset with the most occurrences as the optimal feature subset.

Step S2.2: Input the optimal feature subset obtained in step S2.1 into the base classifier for training, introduce the diversity metric in ensemble learning to evaluate the combination effect between base classifiers, and select a high-quality heterogeneous ensemble learning base classifier model. For the base classifier model to be selected, Bayesian optimization parameters are used to complete the first base classifier filtering by calculating the accuracy (ACC) and area under the curve (AUC) of the model.

The calculation formula of model ACC is shown in Equation 7:

In Formula 7, TP represents the number of samples whose predicted values are consistent with the true values and are both positive; FP represents the number of samples whose predicted values are positive and the true values are negative; TN represents the number of samples whose predicted values are consistent with the true values and are both negative; FN represents the number of samples whose predicted values are negative and the true values are positive.

AUC uses the receiver operating characteristic (ROC) curve, and its calculation method is shown in formula 8. AUC of 1 corresponds to an ideal classifier, and its expression formula is as follows:

In Formula 8, M _p and M _n represent the number of positive and negative samples respectively, i represents the sorting number of the positive sample, and M*N represents the number of cases where one sample is randomly selected from each of the positive and negative samples.

In order to obtain the best prediction effect, it is also necessary to select base learners with relatively large differences for the first base classifier filtering. For the selective integration of models, the pairwise diversity metric Dis, Q statistics and double failure metric DF based on ensemble learning are introduced.

The inconsistency measure Dis focuses on the samples with different classification results of the two classifiers. The more samples with different classification results, the higher the degree of difference between the two classifiers. On the contrary, the less diversity, the value range is [0,1]. The calculation of the inconsistency measure Dis is shown in formula 9. The total number of samples is represented by N.

The Q statistic is related to the diversity between classifiers, and its value range is [-1,1]. If two classifiers are relatively independent, it means that the classification methods are completely unrelated, that is, their Q statistic value is 0. The Q statistic is calculated as follows:

As shown in formula 10.

The double failure metric DF is mainly aimed at samples that are misclassified by two classifiers, and its value range is [0,1]. The more samples the two classifiers misclassify, the more likely the two classifiers are to make mistakes on the same samples. The calculation of the double failure metric DF is shown in Equation 11. The total number of samples is represented by N.

The present invention performs metric calculations on single classifiers in the target base classifier group in pairs, tending to select a base classifier combination whose inconsistency metric Dis and Q statistics tend to 1, and whose DF value tends to 0, so as to select the most excellent quality ensemble learning base classifier model. In addition, in order to avoid overfitting of the model, a ten-fold cross-validation is used, and the training set is divided into 10 sub-samples. A single sub-sample is retained as data for verifying the model, and the other (10-1) samples are used for training. The cross-validation is repeated 10 times, and each sub-sample is verified once. The results of 10 times are averaged or other combined methods are used to finally obtain a single estimate, thereby the fitting of the model.

Step S2.3: Use a weighted voting mechanism based on the Bagging ensemble algorithm to perform the integration of the base classifier model selected in step S2.2. The system obtains the ROC curve of the base classifier and calculates the AUC value based on the ROC curve, and then uses the accuracy and AUC value as the weights of the weighted voting mechanism. The inter-classifier ensemble weighting function is shown in Formula 12:

In Formula 12, U _aoc,i represents the area not covered by the ROC curve of the i-th base learner (U _aoc,i = 1-AUC) and are the maximum and minimum values of the area not covered by the ROC curve of all base learners, respectively. e _i , e _b , and _ew represent the accuracy of the i-th classifier, the accuracy of the classifier with the lowest accuracy in the set, and the accuracy of the classifier with the highest accuracy in the set, respectively.

S3: Embed the integrated model generated by training into the SDN controller and start the DDOS attack detection module.

The integrated model generated in step S2.3 is embedded into the SDN controller, and the DDOS attack detection module is started. When normal traffic is generated, the Double-Bagging model algorithm predicts it as normal traffic. When attack traffic is generated, it is immediately detected as a DDOS attack and the port it enters is blocked. If a port is blocked, the controller still allows normal traffic from other ports to pass. The controller unblocks the port after a period of time and restarts detection. If the attack is still active, it will detect and block the port again. The blocking will continue as long as the attack continues.

What is disclosed above is only a preferred embodiment of the present invention, and it certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims (8)

1. The software defined network DDOS attack cooperative defense method based on the ensemble learning is characterized by comprising the following steps:

starting an SDN controller to collect flow data, and storing the flow data in a CSV file;

putting the collected data into a Double-Bagging detection model for training;

embedding the integrated model generated by training into an SDN controller, and starting a DDOS attack detection module.

2. The software defined network DDOS attack collaborative defense method based on ensemble learning according to claim 1,

the flow data comprise data of normal flow and attack flow, and consist of the speed of an IP source, the flow count, the speed of a flow meter item and a flow comparison value, wherein the speed of the IP source, the flow count, the speed of the flow meter item and the flow comparison value are lower in the normal flow than in the attack flow.

3. The software defined network DDOS attack collaborative defense method based on ensemble learning according to claim 1,

the establishing process of the Double-Bagging detection model comprises the following steps:

performing data preprocessing and feature dimension reduction, and acquiring an optimal feature subset through a feature subset selection voting mechanism based on a bagging integration algorithm;

the optimal feature subset is input into a base classifier for training, and a high-quality heterogeneous integrated learning base classifier model is selected;

and integrating the selection base classifier model by adopting a weighted voting mechanism based on a Bagging integration algorithm.

4. The software defined network DDOS attack collaborative defense method based on ensemble learning according to claim 3,

feature dimension reduction adopts dimension reduction algorithm combining filtering method and embedding method, and selects two feature selection algorithms in filtering method, namely chi-square verification and mutual information algorithm and limit random tree algorithm in embedding method to calculate the ranking of feature contribution.

5. The software defined network DDOS attack collaborative defense method based on ensemble learning according to claim 3,

and acquiring an optimal feature subset, namely determining the number of results of each feature selection algorithm through a set threshold value, generating a feature subset of the results according to feature contribution ordering, and selecting the feature subset with the largest occurrence number as the optimal feature subset by adopting a voting strategy.

6. The software defined network DDOS attack collaborative defense method based on ensemble learning according to claim 3,

and in the process of inputting the optimal feature subset into the base classifier for training and selecting a high-quality heterogeneous integrated learning base classifier model, introducing a diversity measure in integrated learning to evaluate the combination effect among the base classifiers, adopting Bayesian optimization parameters, and completing the first filtering of the base classifier through calculating the accuracy of the model and the area under the curve.

7. The software defined network DDOS attack collaborative defense method based on ensemble learning according to claim 3,

the weight of the weighted voting mechanism is determined by the accuracy and the area value under the curve, and the area value under the curve is obtained through calculation of the working characteristic curve of the subject of the base classifier.

8. The software defined network DDOS attack collaborative defense method based on ensemble learning according to claim 1,

after the DDOS attack detection module is started, when normal traffic is generated, the Double-Bagging model algorithm predicts the normal traffic as the normal traffic, and when attack traffic is generated, the attack traffic is immediately detected as the DDOS attack and the port into which the attack traffic enters is blocked.

Images