CN107294929B

CN107294929B - Rule matching and management method and device

Info

Publication number: CN107294929B
Application number: CN201610206641.1A
Authority: CN
Inventors: 张耀
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba China Co Ltd
Priority date: 2016-04-05
Filing date: 2016-04-05
Publication date: 2021-05-18
Anticipated expiration: 2036-04-05
Also published as: CN107294929A

Abstract

The application discloses a rule matching and managing method and a device, comprising the following steps: the client matches the local file according to the local rule of the client, determines the file which is consistent with the local rule of the client and the corresponding matching rule, and sends the file which is consistent with the matching and the identification of the corresponding matching rule to the server, so that the server acquires the attribute of the matching rule corresponding to the identification of the matching rule according to the identification of the matching rule reported by the client; and filtering the file reported by the client according to the attribute of the matching rule. The workload of regular matching of the server is reduced by reducing the number of files reported to the server, and the matching efficiency of the server is improved.

Description

Rule matching and management method and device

Technical Field

The application belongs to the technical field of internet security, and particularly relates to a rule matching and managing method and device.

Background

With the rapid development of cloud computing, more and more entrepreneurs build their own servers on the cloud, and due to the uneven security capability of the personnel using the cloud servers, a hacker can very easily implant a backdoor program (the backdoor program generally refers to malicious codes which bypass security control and obtain access rights to a program or a system) on the cloud server, for example, WebShell, so that the cloud server is invaded, and serious economic loss is caused.

Large cloud server providers and professional cloud server security manufacturers can provide corresponding checking and killing products to scan and detect the backdoor programs, but due to the fact that the number of the backdoor programs needing to be scanned and detected is increased, detection throughput, matching time complexity and matching hit rate of the servers are greatly challenged.

In order to improve the detection efficiency of the backdoor program such as malicious codes, the timeliness, effectiveness, stability and optimization of the rules used by the detection system based on rule matching must be ensured, so that a management method of the matching rules is urgently needed to greatly improve the detection efficiency and hit rate of the backdoor program such as malicious codes and reduce the time complexity of matching.

Disclosure of Invention

In view of this, the present application provides a method and an apparatus for rule matching and management, which can solve the problems of low hit rate and high time complexity in the matching process of the existing rule.

In order to solve the above technical problem, a first aspect of the present application provides a rule matching method, executed by a client, including:

the method comprises the steps that a client matches a local file according to a client local rule, and the file which is consistent with the client local rule in matching and a corresponding matching rule are determined, wherein the client local rule comprises a plurality of rules, the matching rule is the client local rule which is consistent with the file in matching, and the local file is the file local to the client;

sending the files with consistent matching and the corresponding matching rule identification to a server side, so that the server side obtains the attribute of the matching rule corresponding to the matching rule identification according to the matching rule identification reported by the client side; and filtering the file reported by the client according to the attribute of the matching rule.

Optionally, the client obtains a new client rule from the server, and updates the client local rule according to the new client rule, where the new client rule includes a client rule reconfigured by the server.

Optionally, the client local rule includes a plurality of rules, each rule includes a file feature collected in advance, and the method further includes:

and matching the local files according to the file characteristics included in each rule, and if the characteristics of the local files are matched with the file characteristics included in one rule, determining that the rules are matched with the files.

The second aspect of the present application further provides a rule matching method, executed by a server, including:

the server receives a file which is consistent in matching and sent by a client and an identifier of a corresponding matching rule, wherein the matching rule is a client local rule which is consistent in matching with the file;

acquiring the attribute of the matching rule corresponding to the identification of the matching rule according to the identification of the matching rule reported by the client;

and filtering the file reported by the client according to the attribute of the matching rule.

Optionally, the attribute comprises an effective state of the rule and/or an effective time of the rule;

according to the attribute of the matching rule, filtering the file reported by the client, wherein the filtering comprises the following steps:

if the effective state of the attributes of the matching rules is an invalid state, the matching rules are invalid rules, and files matched with the invalid rules in the files reported by the client are discarded;

and if the effective state of the attributes of the matching rule is an effective state and the current time is not within the effective time, the matching rule is an invalid rule, and the files matched with the invalid rule in the files reported by the client are discarded.

Optionally, the filtering the file reported by the client according to the attribute of the matching rule includes:

and if the attribute of the matching rule is in a gray state, acquiring the matching hit rate of the matching rule, and when the matching hit rate of the matching rule is greater than a preset hit rate threshold value, discarding files matched with the matching rule in the files reported by the client, wherein the matching hit rate is the matching consistency of the same rule or the matching hit frequency within a certain time.

Optionally, after filtering the file reported by the client, the method includes:

the server side performs rule matching on the rest files one by one from high priority to low priority according to the priority of each rule in the local rules of the server side, wherein the rest files are the files left after the files reported by the client side are filtered;

determining files which are matched and consistent with the server local rule in the residual files and the corresponding matched and hit server local rule;

and updating the matching hit times of the matched and hit server local rule.

Optionally, after updating the number of matching hits of the server local rule that is hit by matching, the method further includes:

determining the matching hit rate of each rule according to the number of times of matching hit of each rule in the local rules of the server, wherein the matching hit rate is the number of times of matching hit or the same rule is matched and consistent within a certain time;

and dynamically adjusting the priority of each rule in the local rules of the server from high to low according to the matching hit rate.

A third aspect of the present application provides a rule management method, executed by a server, including:

when the server triggers the update of the local current effective rule of the server, allocating a storage space for the new effective rule, and initializing the new effective rule to the storage space;

and switching the current effective rule to the new effective rule, and performing rule matching according to the new effective rule.

Optionally, the method further comprises:

when the server triggers partial updating of the current effective rule, backing up the current effective rule;

partially updating the current validation rule of the backup;

and switching the current effective rule to the partially updated current effective rule.

Optionally, the method further comprises:

and backing up the current validation rule and the corresponding version number to an old validation rule base.

Optionally, the method further comprises:

when the server triggers the rollback of an old validation rule, acquiring the old validation rule corresponding to the version number to be rolled back from the old validation rule base according to the version number to be rolled back;

and switching the current effective rule to the rolled-back old effective rule.

Optionally, the method further comprises:

locking the rule matching of the server before the rule switching;

and after the rule switching, unlocking the rule matching of the server.

The fourth aspect of the present invention further provides a rule management method, executed by a server, including:

when a server triggers optimization of a local current effective rule of the server, obtaining a matching hit rate of each rule in the current effective rule, wherein the matching hit rate is the matching consistency of the same rule or the matching hit frequency within a certain time;

setting the priority of each rule in the current effective rule from high to low according to the matching hit rate to obtain an optimized effective rule;

and switching the current effective rule to the optimized effective rule.

Optionally, the method further comprises:

and when the server side matches the rules, the rules are matched one by one from high priority to low priority according to the priority of each rule in the optimized effective rules.

Optionally, the method further comprises:

the current validation rule comprises a plurality of rules, and the method further comprises:

and determining and recording the matching hit rate of each rule according to the matching hit times of each rule when each rule in the current effective rules is matched with the file within the preset time.

Optionally, the method further comprises:

and in a preset time, if the fact that the match hit rate high-low sequence change of each rule in the current effective rule is smaller than a preset change threshold value is determined, the optimization of the current effective rule is stopped to be triggered.

A fifth aspect of the present application provides a rule matching apparatus, located at a client, including:

the matching module is used for matching the local file according to the client local rule and determining the file which is consistent with the client local rule in matching and the corresponding matching rule, wherein the client local rule comprises a plurality of rules, and the matching rule is the client local rule which is consistent with the file in matching;

the sending module is used for sending the files which are matched consistently and the corresponding matching rule identification to the server side so that the server side can obtain the attribute of the matching rule corresponding to the matching rule identification according to the matching rule identification reported by the client side; and filtering the file reported by the client according to the attribute of the matching rule.

Optionally, the apparatus further comprises:

and the rule updating module is used for acquiring a new client rule from the server and updating the client local rule according to the new client rule, wherein the new client rule comprises a client rule reconfigured by the server.

Optionally, the client local rule includes a plurality of rules, each rule includes file features collected in advance, and the matching module is specifically configured to:

A sixth aspect of the present application provides a rule matching apparatus, located at a server, including:

the receiving module is used for receiving the files which are matched and consistent and the identifications of the corresponding matching rules, which are sent by the client, wherein the matching rules are client local rules which are matched and consistent with the files;

the acquisition module is used for acquiring the attribute of the matching rule corresponding to the identification of the matching rule according to the identification of the matching rule reported by the client;

and the filtering module is used for filtering the file reported by the client according to the attribute of the matching rule.

the filtration module is specifically configured to:

Optionally, the attribute includes a gray state of a rule, and according to the attribute of the matching rule, the filtering module is specifically configured to:

Optionally, the apparatus further comprises:

the matching module is used for carrying out rule matching on the residual files one by one from high priority to low priority according to the priority of each rule in the local rules of the server, wherein the residual files are the files which are residual after the files reported by the client are filtered; determining files which are matched and consistent with the server local rule in the residual files and the corresponding matched and hit server local rule;

and the matching hit frequency updating module is used for updating the matching hit frequency of the matched and hit server local rule.

Optionally, the apparatus further comprises:

the determining module is used for determining the matching hit rate of each rule according to the matching hit frequency of each rule in the local rules of the server, wherein the matching hit rate is the matching consistency of the same rule or the matching hit frequency within a certain time;

and the priority adjusting module is used for dynamically adjusting the priority of each rule in the local rules of the server from high to low according to the matching hit rate.

A seventh aspect of the present application provides a rule management apparatus, located at a server, including:

the rule updating module is used for distributing a storage space for the new effective rule and initializing the new effective rule to the storage space when the updating of the local current effective rule of the server is triggered;

and the switching module is used for switching the current validation rule to the new validation rule in English and matching the rules according to the new validation rule.

Optionally, the apparatus further comprises:

the backup module is used for backing up the current effective rule when the rule updating module triggers partial updating of the current effective rule;

the rule updating module is also used for partially updating the current effective rule of the backup;

the switching module is further configured to switch the current validation rule to the partially updated current validation rule.

Optionally, the backup module is further configured to backup the current validation rule and the corresponding version number into an old validation rule base.

Optionally, the apparatus further comprises:

the rollback module is used for acquiring an old validation rule corresponding to the version number to be rolled back from the old validation rule base according to the version number to be rolled back when the rollback of the old validation rule is triggered;

the switching module is further configured to switch the current validation rule to the rolled-back old validation rule.

Optionally, the apparatus further comprises:

the locking module is used for locking the rule matching of the server before the rule switching;

and the unlocking module is used for unlocking the rule matching of the server after the rule switching.

An eighth aspect of the present application provides a rule management apparatus, located at a server, including:

the system comprises an obtaining module, a judging module and a judging module, wherein the obtaining module is used for obtaining the matching hit rate of each rule in the current effective rules when the optimizing module triggers the optimization of the local current effective rules of the server, and the matching hit rate is the matching consistency of the same rule or the matching hit frequency within a certain time;

the optimization module is used for setting the priority of each rule in the current effective rule from high to low according to the matching hit rate to obtain the optimized effective rule;

and the switching module is used for switching the current effective rule to the optimized effective rule.

Optionally, the apparatus further comprises:

and the matching module is used for matching the rules one by one from high priority to low priority according to the priority of each rule in the effective rules optimized by the optimization module when the rules are matched.

Optionally, the current validation rule includes a plurality of rules, and the apparatus further includes:

and the determining module is used for determining and recording the matching hit rate of each rule according to the matching hit times of each rule when each rule in the current effective rule is matched with the file within the preset time.

Optionally, the determining module is further configured to stop triggering the optimization of the current validation rule if it is determined that the match hit rate of each rule in the current validation rule is higher than a preset change threshold in a preset time.

In the embodiment of the invention, the priority of each rule in the local rules of the server is set from high to low according to the matching hit rate, so that the rules with high matching hit rate are matched firstly when the rules are matched, the detection result of the matched file can be accelerated, the time complexity of rule matching can be reduced, and the matching efficiency can be increased.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

fig. 1 is a schematic flow chart of a rule matching method according to an embodiment of the present invention;

FIG. 2 is a schematic flow chart illustrating a rule matching method according to an embodiment of the present invention;

FIG. 3 is a flowchart illustrating a rule management method according to an embodiment of the present invention;

FIG. 4 is a flowchart illustrating a rule management method according to an embodiment of the present invention;

FIG. 5 is a flowchart illustrating a rule management method according to an embodiment of the present invention;

FIG. 6 is a flowchart illustrating a rule management method according to an embodiment of the present invention;

FIG. 7 is a flowchart illustrating a server configuration rule according to an embodiment of the present invention;

FIG. 8 is a schematic diagram illustrating a rule matching process between a client and a server according to an embodiment of the present invention;

fig. 9 is a system architecture diagram for filtering a file reported by a client according to an embodiment of the present invention;

fig. 10 is a schematic flow chart illustrating filtering of a file reported by a client according to an embodiment of the present invention;

FIG. 11 is a functional diagram of server rule generation according to an embodiment of the present invention;

FIG. 12 is a flowchart illustrating the generation of rules at the server side according to an embodiment of the present invention;

FIG. 13 is a flowchart illustrating a rule change of a server according to an embodiment of the present invention;

FIG. 14 is a functional diagram illustrating a server rule rollback according to an embodiment of the present invention;

FIG. 15 is a flowchart illustrating a regular rollback at a server in accordance with an embodiment of the present invention;

FIG. 16 is a functional diagram illustrating rule matching at a server according to an embodiment of the present invention;

FIG. 17 is a diagram illustrating a server rule matching process according to an embodiment of the present invention;

FIG. 18 is a schematic diagram of a detection process for detecting county city according to an embodiment of the present invention;

FIG. 19 is a schematic diagram of rule optimization according to an embodiment of the present invention;

FIG. 20 is a schematic structural diagram of a rule matching apparatus according to an embodiment of the present invention;

FIG. 21 is a schematic structural diagram of a rule matching apparatus according to an embodiment of the present invention;

FIG. 22 is a schematic structural diagram of a rule management apparatus according to an embodiment of the present invention;

fig. 23 is a schematic structural diagram of a rule management apparatus according to an embodiment of the present invention.

Detailed Description

Embodiments of the present application will be described in detail with reference to the drawings and examples, so that how to implement technical means to solve technical problems and achieve technical effects of the present application can be fully understood and implemented.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape or disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transient media), such as modulated data signals and carrier waves.

As used in the specification and in the claims, certain terms are used to refer to particular components. As one skilled in the art will appreciate, manufacturers may refer to a component by different names. This specification and claims do not intend to distinguish between components that differ in name but not function. In the following description and in the claims, the terms "include" and "comprise" are used in an open-ended fashion, and thus should be interpreted to mean "include, but not limited to. "substantially" means within an acceptable error range, and a person skilled in the art can solve the technical problem within a certain error range to substantially achieve the technical effect. Furthermore, the term "coupled" is intended to encompass any direct or indirect electrical coupling. Thus, if a first device couples to a second device, that connection may be through a direct electrical coupling or through an indirect electrical coupling via other devices and couplings. The description which follows is a preferred embodiment of the present application, but is made for the purpose of illustrating the general principles of the application and not for the purpose of limiting the scope of the application. The protection scope of the present application shall be subject to the definitions of the appended claims.

It is also noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a good or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such good or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a commodity or system that includes the element.

Fig. 1 is a schematic flowchart of a rule matching method according to an embodiment of the present invention, which is executed at a client, and as shown in fig. 1, the method includes:

101. the client matches the local file according to the local rule of the client, and determines the file which is consistent with the local rule of the client and the corresponding matching rule;

specifically, the local file refers to a file local to the client, and the file may be a malicious file or a non-malicious file, and the matching rule is a client local rule matching the file; the client local rule comprises a plurality of rules, wherein each rule comprises file characteristics (such as malicious file characteristics) collected in advance, and the specific matching process comprises the following steps:

and matching the local files according to the file characteristics included in each rule, and if the characteristics of the local files are matched with the file characteristics included in one rule, determining that the rules are matched with the files. For example, assuming that the client local rule includes N rules (rule 1, rule 2 … rule N), when matching rule 1 with the local file, assuming that rule 1 includes m characteristic points of the trojan virus, by extracting k characteristic points of the local file, comparing the m characteristic points of the trojan virus in rule 1 with the k characteristic points of the local file, and if N characteristic points greater than or equal to a preset value are consistent, it may be determined that rule 1 is consistent with the local file.

102. Sending the files with consistent matching and the corresponding matching rule identification to a server side, so that the server side obtains the attribute of the matching rule corresponding to the matching rule identification according to the matching rule identification reported by the client side; and filtering the file reported by the client according to the attribute of the matching rule.

Specifically, after determining files which are matched with local rules of the client and corresponding matching rules, the client acquires the identifiers of the matching rules and reports all the files which are matched with the local rules and the identifiers of the corresponding matching rules to the server;

correspondingly, the server side acquires the attribute of the matching rule corresponding to the identification of the matching rule according to the identification of the matching rule reported by the client side; and filtering the file reported by the client according to the attribute of the matching rule.

It should be noted that the client local rule in the embodiment of the present invention is also set by the server, and the client periodically obtains the client local rule set by the server from the server. The server sets corresponding attributes when setting each client rule, wherein the attributes of the rules include but are not limited to information such as the identification of the rules, the effective state of the rules, the gray state of the rules, the effective time of the rules and/or the priority of the rules. Therefore, after the client reports the identifier of the matching rule, the server can obtain the attribute of the corresponding matching rule according to the identifier of the matching rule;

further, the filtering, by the server, the file reported by the client according to the attribute of the matching rule includes:

if the effective state of the attributes of the matching rule is an effective state and the current time is not within the effective time, the matching rule is an invalid rule, and the files matched with the invalid rule in the files reported by the client are discarded;

It should be noted that, in the embodiment of the present invention, the client may obtain a new client rule from the server periodically to update the client local rule, so as to ensure timeliness and validity of the client local rule.

According to the method and the device, the client local file is matched for the first time through the client local rule, only the file which is consistent with the rule matching is reported to the server, the workload of rule matching of the server is reduced by reducing the number of the files which are reported to the server, and the matching efficiency of the server is further improved.

Fig. 2 is a schematic flowchart of a rule matching method according to an embodiment of the present invention, which is executed at a server, and as shown in fig. 2, the method includes:

201. the server receives the files which are consistent in matching and the corresponding matching rule identification sent by the client;

the matching rule is a local client rule matching with the file, and the client sends the file matching with the file and the identifier of the corresponding matching rule to the server, which may refer to the related description of step 101 in the embodiment of fig. 1 and is not described again;

202. acquiring the attribute of the matching rule corresponding to the identification of the matching rule according to the identification of the matching rule reported by the client;

203. and filtering the file reported by the client according to the attribute of the matching rule.

The method specifically comprises the following steps:

Optionally, the step 203 further includes, after filtering the file reported by the client:

204. according to the priority of each rule in the local rules of the server, performing rule matching on the rest files one by one from high priority to low priority;

the residual files are files which are left after the files reported by the client are filtered;

205. determining files which are matched and consistent with the server local rule in the rest files and the corresponding server local rule which is matched and hit;

206. updating the matching hit times of the matched and hit server local rule;

207. determining the matching hit rate of each rule according to the number of times of matching hit of each rule in the local rules of the server;

the matching hit rate is the matching consistency of the same rule or the number of matching hits within a certain time;

208. and dynamically adjusting the priority of each rule in the local rules of the server from high to low according to the matching hit rate.

In practical application, in order to improve the rule matching throughput (or detection efficiency) of the server, the embodiment of the present invention may adopt a thread pool (multiple detection threads) parallel detection manner, that is, files and rules reported by the client are submitted to each detection thread for matching at the same time. After the detection of each detection thread is completed, a matching hit is given or all rules are matched and ended (i.e. matching is not hit), if a certain rule is consistent with file matching, the rule is matched and hit, and the hit frequency of the rule is correspondingly updated (i.e. the matching hit rate of the rule is recorded).

FIG. 3 is a flowchart illustrating a rule management method according to an embodiment of the present invention; the method is executed at a server side, and as shown in fig. 3, the method includes:

301. when the server triggers the update of the local current effective rule of the server, allocating a storage space for the new effective rule, and initializing the new effective rule to the storage space;

it should be noted that, in step 301, updating the current rule in effect of the server is to regenerate a new rule.

In order to perform rule rollback operation subsequently, in the embodiment of the present invention, the current validation rule may be backed up to the old validation rule base as the old validation rule, and the version number of each old validation rule needs to be recorded during backup.

302. And switching the current effective rule to the new effective rule, and performing rule matching according to the new effective rule.

It should be noted that, in order to avoid a detection error occurring when rule matching is performed during rule switching, the rule matching of the server is locked before the rule switching; and after the rule switching, unlocking the rule matching of the server. The timeliness and the effectiveness of the local rules of the server are guaranteed, and therefore the accuracy of rule matching of the server is improved.

FIG. 4 is a flowchart illustrating a rule management method according to an embodiment of the present invention; as shown in fig. 4, includes:

401. when the server triggers partial updating of the local current effective rule of the server, backing up the current effective rule;

402. partially updating the current validation rule of the backup;

403. switching the current validation rule to the partially updated current validation rule

Because the server rule has a complex structure and the rule is stored in the database, if the rule is changed with a small amplitude, the regeneration mode is not adopted, and because the resource consumption is very high if the rule is changed with a small amplitude and the regeneration mode shown in fig. 3 is adopted, in the embodiment of the invention, a method for dynamically configuring the rule is adopted for the case that the rule is changed with a small amplitude. I.e. a copy (preferably a deep backup) of the current validation rules is made and the copy (i.e. the current validation rules of the backup) is subsequently updated.

Similarly, in order to perform rule rollback operation subsequently, in the embodiment of the present invention, the current validation rule may be backed up to the old validation rule base as the old validation rule, and the version number of each old validation rule needs to be recorded during backup.

It should be noted that, in order to avoid a detection error occurring when rule matching is performed during rule switching, the rule matching of the server is locked before the rule switching; and after the rule switching, unlocking the rule matching of the server.

FIG. 5 is a flowchart illustrating a rule management method according to an embodiment of the present invention; the method is executed at a server side, and as shown in fig. 5, the method includes:

501. when the server triggers the rollback of an old validation rule, acquiring the old validation rule corresponding to the version number to be rolled back from the old validation rule base according to the version number to be rolled back;

502. and switching the current effective rule to the rolled-back old effective rule.

If a serious problem occurs after a rule is on line and the whole rollback is needed, a version number needs to be selected from an old effective rule base for backup to perform the whole rollback. Since the rollback operation needs to be fast, the old validation rules that are usually backed up need to reside in memory to achieve efficient rollback.

FIG. 6 is a flowchart illustrating a rule management method according to an embodiment of the present invention; as shown in fig. 6, includes:

601. when a server triggers the server to locally optimize the current effective rule, the matching hit rate of each rule in the current effective rule is obtained;

specifically, the embodiment of the present invention may be set within a preset time, and when each rule in the current validation rule is matched with a file, according to the number of times of matching hit of each rule, the matching hit rate of the rule may be recorded, that is, the matching hit rate is the number of times of matching hit or matching match of the same rule within a certain time;

602. Setting the priority of each rule in the current effective rule from high to low according to the matching hit rate to obtain an optimized effective rule;

the optimized effective rules are sorted from high to low according to the matching hit rate, when the server side matches the rules, the rules are matched one by one from high to low according to the priority of each rule in the optimized effective rules, and the rules with high matching hit rate are matched first, so that the detection result of the matched file can be accelerated, the time complexity of rule matching can be reduced, and the detection efficiency can be improved.

603. And switching the current effective rule to the optimized effective rule.

It should be noted that, in order to avoid a detection error occurring when rule matching is performed during rule switching, the rule matching of the server is locked before the rule switching; and after the rule switching, unlocking the rule matching of the server. Meanwhile, the server side updates the local rules of the server side according to the optimized validation rules, and the timeliness, effectiveness and optimization of the local rules of the server side are guaranteed, so that when the server side performs rule matching, the rules are matched one by one from high priority to low priority according to the priority of each rule in the optimized validation rules, the rules with high matching hit rate are matched first, the detection result of a matched file can be accelerated, therefore, the time complexity of rule matching can be reduced, and the detection efficiency can be increased.

It should be noted that frequent rule sequence changes may reduce detection efficiency, and in order to overcome the problem caused by frequent rule sequence changes, in the embodiment of the present invention, in a preset time, if it is determined that the match hit rate of each rule in the current validation rule is higher than a preset change threshold, the triggering of the optimization of the current validation rule may be stopped. That is, the adaptive priority of the optimized validation rule according to the embodiment of the present invention may also be configured to stop the exchange of rule sequences after the rule sequence is stabilized (when the rule on line has reached a stable and efficient state).

The method according to the above embodiment of the present invention is described in detail by the following specific implementation manner:

the rules of the invention are divided into client rules and server rules, wherein the client rules mainly perform preliminary matching judgment on the files and are used for controlling the sample size of the reported files. The server-side rule further carries out accurate matching judgment on the file samples reported by the client-side.

When each rule is configured, the server configures corresponding attributes (such as an effective state, a formal online state, a gray state, a priority level, and the like). Fig. 7 is a schematic flow chart of a server configuration rule according to an embodiment of the present invention, as shown in fig. 7, when an operator configures each rule, the server prompts a real-time situation of rule matching of a current server, and then reasonably sets attributes of the rule, such as a priority of the configuration rule (default is used when not configured, and the server is adjusted according to an actual situation), or quickly drops/rolls back some rules, or configures a gray rule, and when configuring a gray rule, a matching hit rate threshold of the gray rule needs to be configured, for example, when the matching hit rate of the gray rule is greater than a preset hit rate threshold, a file matching the gray rule in a reported file is discarded.

Fig. 8 is a schematic diagram of a rule matching process between a client and a server according to an embodiment of the present invention, as shown in fig. 8, the client reports a file sample to the server, the server filters the file sample reported by the client, and selects different types of rules according to the file sample type, and then selects a corresponding rule according to a real-time rule matching state (e.g., matching according to priority until a matching hit or irregular matching result is obtained), counts a latest rule matching condition, triggers a change in rule priority (the server selects a rule to match the sample file reported by the client, records a matching hit rate of the rule, and the server dynamically adjusts the rule priority according to the matching hit rate of the rule).

Fig. 9 is a system architecture diagram illustrating filtering of a file reported by a client according to an embodiment of the present invention, and fig. 10 is a schematic flow chart illustrating filtering of a file reported by a client according to an embodiment of the present invention, where as shown in fig. 9 and fig. 10, a set of looser rules (denoted by CRx) is maintained by a client, and a main function is to perform preliminary screening on a reported file and avoid reporting of a full sample. Firstly, the client will primarily screen out suspicious files according to the rule (CRx), and report the rule hit by matching to the server, and the reported information includes (CRx, file information). The information reported by the client firstly reaches a Filter (a Filter module) of the server, the Filter module can judge whether the rule CRx is effective under the control of a controller (a processor) of the server according to the judgment that whether the rule CRx is abandoned, whether the rule CRx is in a gray state and the hit frequency exceeds a threshold value, whether the rule CRx is in the gray state and the hit frequency exceeds the threshold value in a short time. If the CRx is invalid or the CRx is a gray rule and the hit frequency exceeds a threshold value, the Filter module discards the reported file sample.

The policy of the controller may be set by an operator, when a certain rule CRx of the client needs to be offline, but the rule file of the client cannot be updated in time, a sample reported by hitting the rule can be configured in the controller for interception and discarding, and the Filter module discards the file sample conforming to the policy.

For the gray rule of the client, the controller provides different strategies, and the controller counts the total matching hit times of one gray rule or the matching hit times within a period of time. According to the strategy of the controller for the gray level rule, when the statistical number exceeds a threshold value, the interception is automatically carried out, and an operator is informed to check the gray level state of the rule.

Wherein, the rule filtering policy can be configured as follows:

matchType: and in the hit mode, discarding the file if the hit frequency exceeds a threshold value, discarding the file if the hit frequency in a short time exceeds the threshold value, and discarding the file if the hit frequency in a short time exceeds the threshold value.

matchCount: the number of hits for the rule.

effectvetime: the validation time of the rule.

enable: whether the rule is in effect.

DeadTime the validity period of the rule, but beyond the validity period, the rule will not take effect.

The filtering policy of the rule according to the embodiment of the present invention is not limited to the above example, and the filtering policy of the rule may be configured in other manners.

Fig. 11 is a functional schematic diagram of the server rule generation according to the embodiment of the present invention, and fig. 12 is a flowchart of the server rule generation according to the embodiment of the present invention, as shown in fig. 11 and fig. 12, the management of the server rule (expressed by SRx) mainly requires timeliness, stability, and efficiency.

When the server side triggers the rule updating, a memory space is redistributed, all the rules of each version are read from the database, and the new rules are initialized to the new memory space. After the initialization process is complete, the old rules are backed up. Locking the rule manager, suspending the matching work, switching the rules, taking the new rules into effect, releasing the lock on the rule manager, and restarting the detection task.

Fig. 13 is a schematic flowchart of a change of a rule of a server according to an embodiment of the present invention, as shown in fig. 13, the rule of the server has a complex structure, and the rule is stored in a database, and if there is a small change, a regeneration method is not adopted, and regeneration is costly, so a method for dynamically configuring the rule is needed. When the rules need to be changed temporarily, a currently effective rule is copied (must be deeply copied) from the current rule database, and then when the copy is updated, the rule manager is locked, the matching work is suspended, the rules are switched, the new rule is effective, the lock on the rule manager is released, and the detection task is restarted.

Fig. 14 is a functional schematic diagram of the rule rollback of the server according to the embodiment of the present invention, and fig. 15 is a schematic diagram of a flow of the rule rollback of the server according to the embodiment of the present invention, as shown in fig. 14 and 15, when a rule is on-line and a serious problem occurs and an entire rollback is required, a version needs to be selected from the backed-up rules to perform the entire rollback. Avoiding a large number of false alarms. Since the rollback operation needs to be fast, the backup rule needs to reside in a memory, and efficient rollback is achieved.

In the embodiment of the present invention, the fields of the SRx rule of the server mainly include:

SR1	enable	matchCount	effectiveTime	isGray
					SR1	enable	matchCount	effectiveTime	isGray

enable whether rules are in effect

matchCount, number of rule hits

effectvetime rule effective time

isGray whether it is a gray rule

Fig. 16 is a functional schematic diagram of rule matching of a server according to an embodiment of the present invention, and fig. 17 is a schematic diagram of a rule matching process of a server according to an embodiment of the present invention, and as shown in fig. 16 and fig. 17, a method for matching file samples of a rule matching file one by one is used until a match is hit or no match is made to the rule.

The embodiment of the invention can dynamically adjust the matching sequence of the rules by adopting a rule priority mode, and places the rule bit with high hit rate at the front position, thereby further reducing the time complexity of matching.

When a certain rule is hit in each sample, the rule manager updates the hit times of the certain rule, and the rule sequence is adjusted after the rule adjustment strategy is triggered.

In order to further improve the detection throughput rate of the server, the CheckManager maintains a thread pool, and the detection uses a multi-thread parallel detection mode, and the CheckManager acquires a file sample from a cache of the file sample reported by the client for detection, and simultaneously acquires a rule from the rule manager, and submits the rule to the detection thread pool for matching detection.

The result manager receives the detection result, and determines whether re-detection is required according to the detection result (e.g., determine whether the current rule version number changes, and if so, re-detection is required). Meanwhile, the rule manager is informed to update the hit state, and finally, the detection result is placed in a result cache to wait for subsequent operation. The rule manager is responsible for operations such as updating, backup and rollback of the rule, and is locked when the rule manager is updated, and the CheckManager cannot acquire the rule and blocks the rule. Meanwhile, the rule manager maintains the hit times of each rule, and provides a gray scale method like filtering a Filter module in a client sample (which is not described herein again).

Note that the CheckManager is responsible for obtaining the rules in the rules manager. Fig. 18 is a schematic diagram of a detection flow of detecting threads according to an embodiment of the present invention, as shown in fig. 18, if operations such as adjusting or deleting and modifying a rule occurrence sequence in a rule manager occur during a detection process of detecting threads, a detection failure of the detection threads may occur (rule duplicate detection, rule missing detection). The detection thread pushes the detection result to the result manager. The result manager needs to judge that the rule version changes, and if the rule version changes, the result which is not successfully detected needs to be handed over to the sample cache again to wait for re-detection.

It should be noted that fig. 19 is a schematic diagram of rule optimization according to the embodiment of the present invention, and as shown in fig. 19, in order to avoid the problem that the detection efficiency is reduced due to frequent change of the rule sequence, the embodiment of the present invention may be configured such that the server triggers a task of rule optimization once every 6 hours. And carrying out priority sequencing according to the hit times of the rules, and reorganizing the rule list. The rules are sorted from high to low according to the hit times in the organized rule table, so that the hit rate of the rules can be greatly improved during detection, the time complexity is reduced, and the detection throughput is increased.

Since the rules tend to stabilize after taking effect on the line for a period of time, the rule adaptive priority adjustment of the present invention may also be configured to stop the exchange of rule sequences after stabilization (until the next change of the rule), at which point the on-line rules have reached a stable and efficient state.

FIG. 20 is a schematic structural diagram of a rule matching apparatus according to an embodiment of the present invention; located at the client, as shown in fig. 20, includes:

the matching module 21 is configured to match a local file according to a client local rule, and determine a file matching the client local rule in a consistent manner and a corresponding matching rule, where the client local rule includes multiple rules, and the matching rule is a client local rule matching the file in a consistent manner;

a sending module 22, configured to send the file with the consistent matching and the identifier of the corresponding matching rule to the server, so that the server obtains, according to the identifier of the matching rule reported by the client, an attribute of the matching rule corresponding to the identifier of the matching rule; and filtering the file reported by the client according to the attribute of the matching rule.

Wherein, the device still includes:

and the rule updating module 23 is configured to acquire a new client rule from the server, and update the client local rule according to the new client rule, where the new client rule includes a client rule reconfigured by the server.

Wherein, the matching module 21 is specifically configured to:

The apparatus shown in fig. 20 may execute the method described in the embodiment shown in fig. 1, and implementation principles and technical effects are not described again, and reference may be made to the related description in the above embodiment.

FIG. 21 is a schematic structural diagram of a rule matching apparatus according to an embodiment of the present invention; located at the server side, as shown in fig. 21, includes:

the receiving module 31 is configured to receive a file sent by a client and matching the file in a consistent manner and an identifier of a corresponding matching rule, where the matching rule is a client local rule matching the file in a consistent manner;

an obtaining module 32, configured to obtain, according to the identifier of the matching rule reported by the client, an attribute of the matching rule corresponding to the identifier of the matching rule;

and a filtering module 33, configured to filter the file reported by the client according to the attribute of the matching rule.

Wherein the attribute comprises an effective state of the rule and/or an effective time of the rule;

the filtering module 33 is specifically configured to:

Wherein the attribute includes a gray state of a rule, and according to the attribute of the matching rule, the filtering module 33 is specifically configured to:

Wherein, the device still includes:

the matching module 34 is configured to perform rule matching on the remaining files one by one from a high priority to a low priority according to a priority of each rule in the local rules of the server, where the remaining files are files remaining after the file reported by the client is filtered; determining files which are matched and consistent with the server local rule in the residual files and the corresponding matched and hit server local rule;

and a matching hit number updating module 35, configured to update the number of matching hits of the server local rule that is hit by matching.

Wherein, the device still includes:

a determining module 36, configured to determine a matching hit rate of each rule according to the number of times that each rule in the server local rule is matched and hit, where the matching hit rate is the number of times that the same rule is matched and consistent or matched and hit within a certain time;

and the priority adjusting module 37 is configured to dynamically adjust the priority of each rule in the server local rule from high to low according to the matching hit rate.

The apparatus shown in fig. 21 may execute the method described in the embodiment shown in fig. 2, and implementation principles and technical effects are not described again, and reference may be made to the related description in the above embodiment.

FIG. 22 is a schematic structural diagram of a rule management apparatus according to an embodiment of the present invention; located at the server side, as shown in fig. 22, includes:

a rule updating module 41, configured to, when updating of a local current validation rule of a server is triggered, allocate a storage space for the new validation rule, and initialize the new validation rule to the storage space;

and the switching module 42 switches the current validation rule to the new validation rule in english, and performs rule matching according to the new validation rule.

Wherein, the device still includes:

a backup module 43, configured to backup the current validation rule when the rule update module triggers a partial update of the current validation rule;

the rule updating module 41 is further configured to partially update the current validation rule of the backup;

the switching module 42 is further configured to switch the current validation rule to the partially updated current validation rule.

The backup module 43 is further configured to backup the current validation rule and the corresponding version number into an old validation rule base.

Wherein, the device still includes:

the rollback module 44 is configured to, when rollback of an old validation rule is triggered, obtain, according to the version number to be rolled back, the old validation rule corresponding to the version number to be rolled back from the old validation rule base;

the switching module 42 is further configured to switch the current validation rule to the rolled-back old validation rule.

Wherein, the device still includes:

a locking module 45, configured to lock the rule matching of the server before the rule switching;

and the unlocking module 46 is used for unlocking the rule matching of the server after the rule switching.

The apparatus shown in fig. 22 may execute the method described in any one of the embodiments in fig. 3 to fig. 5, and implementation principles and technical effects are not described again, and reference may be made to the related description in the above embodiments.

Fig. 23 is a schematic structural diagram of a rule management apparatus according to an embodiment of the present invention, which is located at a server, and as shown in fig. 23, the rule management apparatus includes:

the obtaining module 51 is configured to obtain a matching hit rate of each rule in the current validation rule when the optimizing module 52 triggers optimization of a local current validation rule of a server, where the matching hit rate is a number of times that a same rule is matched and consistent or matched and hit within a certain time;

the optimization module 52 is configured to set the priority of each rule in the current validation rule from high to low according to the matching hit rate, so as to obtain an optimized validation rule;

a switching module 53, configured to switch the current validation rule to the optimized validation rule.

Wherein, the device still includes:

and the matching module 54 is configured to perform rule matching one by one from higher priority to lower priority according to the priority of each rule in the rule that is optimized by the optimization module when the rule is matched.

Wherein the current validation rule includes a plurality of rules, the apparatus further comprising:

and the determining module 55 is configured to determine and record a matching hit rate of each rule according to the number of times that each rule is matched and hit when each rule in the current validation rules is matched with a file within a preset time.

The determining module 55 is further configured to, within a preset time, stop triggering the optimization of the current validation rule if it is determined that the match hit rate of each rule in the current validation rule has a high-low order change smaller than a preset change threshold.

The apparatus shown in fig. 23 may execute the method described in the embodiment shown in fig. 6, and implementation principles and technical effects are not described again, and reference may be made to the related description in the above embodiment.

The foregoing description shows and describes several preferred embodiments of the invention, but as aforementioned, it is to be understood that the invention is not limited to the forms disclosed herein, but is not to be construed as excluding other embodiments and is capable of use in various other combinations, modifications, and environments and is capable of changes within the scope of the inventive concept as expressed herein, commensurate with the above teachings, or the skill or knowledge of the relevant art. And that modifications and variations may be effected by those skilled in the art without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

1. A rule matching method is executed at a client side and is characterized by comprising the following steps:

the client matches a local file according to a client local rule, and determines a file which is consistent with the client local rule in matching and a corresponding matching rule, wherein the client local rule comprises a plurality of rules, and the matching rule is the client local rule which is consistent with the file in matching;

sending the files which are matched consistently and the marks of the corresponding matching rules to a server, so that the server acquires the attributes of the matching rules corresponding to the marks of the matching rules according to the marks of the matching rules reported by the client, filters the files reported by the client according to the attributes of the matching rules, and performs rule matching on the rest files one by one from high priority to low priority according to the priority of each rule in the local rules of the server, wherein the rest files are the files left after the files reported by the client are filtered; determining files which are matched and consistent with the server local rule in the residual files and the corresponding matched and hit server local rule; determining the matching hit rate of each rule according to the number of times of matching hit of each rule in the local rules of the server, wherein the matching hit rate is the number of times of matching hit or the same rule is matched and consistent within a certain time; and dynamically adjusting the priority of each rule in the local rules of the server side from high to low according to the matching hit rate, wherein the attributes of the matching rules include but are not limited to the identification of the rules, the effective state of the rules, the gray state of the rules, the effective time of the rules and the priority of the rules.

2. The method of claim 1, further comprising:

and the client acquires a new client rule from the server and updates the local rule of the client according to the new client rule, wherein the new client rule comprises the client rule reconfigured by the server.

3. The method of claim 1 or 2, wherein the client-local rule comprises a plurality of rules, each rule comprising pre-collected file features, the method further comprising:

4. A rule matching method is executed at a server side and is characterized by comprising the following steps:

acquiring the attribute of the matching rule corresponding to the identifier of the matching rule according to the identifier of the matching rule reported by the client, wherein the attribute of the matching rule comprises but is not limited to the identifier of the rule, the effective state of the rule, the gray state of the rule, the effective time of the rule and the priority of the rule;

filtering the files reported by the client according to the attributes of the matching rules;

after filtering the file reported by the client, the method comprises the following steps:

5. The method of claim 4, wherein filtering the file reported by the client according to the attribute of the matching rule comprises:

6. The method of claim 4, wherein filtering the file reported by the client according to the attribute of the matching rule comprises:

7. The method according to any one of claims 4-6, wherein filtering the file reported by the client comprises:

and updating the matching hit times of the matched and hit server local rule.

8. A rule management method is executed at a server side and is characterized by comprising the following steps:

switching the current validation rule to the new validation rule, and filtering a corresponding file reported by a client according to the attribute of the new validation rule, wherein the attribute of the new validation rule includes but is not limited to a rule identifier, a rule validation state, a rule gray state, a rule validation time and a rule priority;

further comprising:

according to the priority of each rule in the local rules of the server, performing rule matching on the rest files one by one from high priority to low priority, wherein the rest files are the files left after the files reported by the client are filtered;

9. The method of claim 8, further comprising:

partially updating the current validation rule of the backup;

10. The method of claim 8 or 9, further comprising:

11. The method of claim 10, further comprising:

and switching the current effective rule to the rolled-back old effective rule.

12. The method of claim 8, 9 or 11, further comprising:

locking the rule matching of the server before the rule switching;

and after the rule switching, unlocking the rule matching of the server.

13. A rule management method is executed at a server side and is characterized by comprising the following steps:

switching the current validation rule to the optimized validation rule, and filtering a corresponding file reported by a client according to the attribute of the optimized validation rule, wherein the attribute of the optimized validation rule comprises but is not limited to a rule identifier, a rule validation state, a rule gray state, a rule validation time and a rule priority;

wherein, still include:

14. The method of claim 13, further comprising:

15. The method of claim 14, wherein the current validation rule includes a plurality of rules, the method further comprising:

16. The method of claim 15, further comprising:

17. A rule matching apparatus at a client, comprising:

the sending module is used for sending the files which are matched consistently and the corresponding matching rule identification to the server side so that the server side can obtain the attribute of the matching rule corresponding to the matching rule identification according to the matching rule identification reported by the client side; filtering the files reported by the client according to the attributes of the matching rules, and performing rule matching on the rest files one by one from high priority to low priority according to the priority of each rule in the local rules of the server, wherein the rest files are the files left after the files reported by the client are filtered; determining files which are matched and consistent with the server local rule in the residual files and the corresponding matched and hit server local rule; determining the matching hit rate of each rule according to the number of times of matching hit of each rule in the local rules of the server, wherein the matching hit rate is the number of times of matching hit or the same rule is matched and consistent within a certain time; and dynamically adjusting the priority of each rule in the local rules of the server side from high to low according to the matching hit rate, wherein the attributes of the matching rules include but are not limited to the identification of the rules, the effective state of the rules, the gray state of the rules, the effective time of the rules and the priority of the rules.

18. The apparatus of claim 17, further comprising:

19. The apparatus according to claim 17 or 18, wherein the client-side local rule includes a plurality of rules, each rule includes a file feature collected in advance, and the matching module is specifically configured to:

20. A rule matching apparatus at a server, comprising:

the acquisition module is used for acquiring the attribute of the matching rule corresponding to the identifier of the matching rule according to the identifier of the matching rule reported by the client, wherein the attribute of the matching rule comprises but is not limited to the identifier of the rule, the effective state of the rule, the gray state of the rule, the effective time of the rule and the priority of the rule;

the filtering module is used for filtering the files reported by the client according to the attributes of the matching rules;

further comprising:

21. The apparatus of claim 20, wherein the filtration module is specifically configured to:

22. The apparatus of claim 20, wherein, according to the attributes of the matching rule, the filtering module is specifically configured to:

23. The apparatus of any one of claims 20-22, further comprising:

24. A rule management apparatus at a server, comprising:

the rule updating module is used for distributing a storage space for a new effective rule and initializing the new effective rule to the storage space when the updating of the local current effective rule of the server is triggered;

a switching module, configured to switch the current validation rule to the new validation rule, and filter, according to an attribute of the new validation rule, a corresponding file reported by a client, where the attribute of the new validation rule includes, but is not limited to, a rule identifier, a rule validation state, a rule gray state, a rule validation time, and a rule priority;

the device is also used for carrying out rule matching on the residual files one by one from high priority to low priority according to the priority of each rule in the local rules of the server, wherein the residual files are the files which are residual after the files reported by the client are filtered; determining files which are matched and consistent with the server local rule in the residual files and the corresponding matched and hit server local rule; determining the matching hit rate of each rule according to the number of times of matching hit of each rule in the local rules of the server, wherein the matching hit rate is the number of times of matching hit or the same rule is matched and consistent within a certain time; and dynamically adjusting the priority of each rule in the local rules of the server from high to low according to the matching hit rate.

25. The apparatus of claim 24, further comprising:

26. The apparatus of claim 25, further comprising:

the backup module is further used for backing up the current validation rule and the corresponding version number to an old validation rule base.

27. The apparatus of claim 26, further comprising:

28. The apparatus of claim 24, 25 or 27, further comprising:

29. A rule management apparatus at a server, comprising:

a switching module, configured to switch the current validation rule to the optimized validation rule, and filter, according to an attribute of the optimized validation rule, a corresponding file reported by a client, where the attribute of the optimized validation rule includes, but is not limited to, a rule identifier, a rule validation state, a rule gray state, a rule validation time, and a rule priority;

30. The apparatus of claim 29, further comprising:

31. The apparatus of claim 30, wherein the current validation rule comprises a plurality of rules, further comprising:

32. The apparatus of claim 31, further comprising:

the determining module is further configured to stop triggering the optimization of the current validation rule if it is determined that the match hit rate high-low order change of each rule in the current validation rule is smaller than a preset change threshold value within a preset time.