CN108173676A - A data intelligent filtering method - Google Patents
A data intelligent filtering method Download PDFInfo
- Publication number
- CN108173676A CN108173676A CN201711327019.7A CN201711327019A CN108173676A CN 108173676 A CN108173676 A CN 108173676A CN 201711327019 A CN201711327019 A CN 201711327019A CN 108173676 A CN108173676 A CN 108173676A
- Authority
- CN
- China
- Prior art keywords
- data
- message
- communication
- user
- filtering
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001914 filtration Methods 0.000 title claims abstract description 37
- 238000000034 method Methods 0.000 title claims abstract description 18
- 238000004891 communication Methods 0.000 claims abstract description 33
- 239000000284 extract Substances 0.000 claims abstract description 3
- 230000002452 interceptive effect Effects 0.000 claims 1
- 230000003993 interaction Effects 0.000 abstract description 7
- 230000006399 behavior Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0604—Management of faults, events, alarms or notifications using filtering, e.g. reduction of information by using priority, element types, position or time
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/02—Network architectures or network communication protocols for network security for separating internal from external traffic, e.g. firewalls
- H04L63/0227—Filtering policies
- H04L63/0236—Filtering by address, protocol, port number or service, e.g. IP-address or URL
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/10—Network architectures or network communication protocols for network security for controlling access to devices or network resources
- H04L63/101—Access control lists [ACL]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/22—Parsing or analysis of headers
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Information Transfer Between Computers (AREA)
Abstract
本发明公开了一种数据智能过滤方法，包括如下步骤：获取用户与终端设备交互过程中产生的通信数据，提取所述通信数据的数据特征，判断所述数据特征是否满足数据过滤条件，若所述数据特征满足数据过滤条件，则对产生的所述通信数据进行过滤；接收用户发送的数据过滤规则；通过通讯芯片接收数据报文；根据所述数据过滤规则对所述数据报文进行过滤；及将过滤后的数据报文发送至终端设备。本发明获取用户与终端设备交互产生的通信数据并提取该通信数据中的数据特征，判断该数据特征是否满足数据过滤条件，若满足，则对通信数据进行过滤，能够对用户与终端设备的交互数据进行过滤，使据能够更加准确的反映用户的真实行为。The invention discloses a data intelligent filtering method, which includes the following steps: acquiring communication data generated during the interaction between a user and a terminal device, extracting the data features of the communication data, and judging whether the data features meet the data filtering conditions, and if the If the data feature meets the data filtering condition, then the generated communication data is filtered; the data filtering rule sent by the user is received; the data message is received through the communication chip; and the data message is filtered according to the data filtering rule; And send the filtered data message to the terminal device. The present invention acquires the communication data generated by the interaction between the user and the terminal equipment, extracts the data features in the communication data, judges whether the data features meet the data filtering conditions, and if so, filters the communication data, and can analyze the interaction between the user and the terminal equipment. The data is filtered so that the data can more accurately reflect the real behavior of the user.
Description
技术领域technical field
本发明涉及一种方法，具体是一种数据智能过滤方法。The invention relates to a method, in particular to a data intelligent filtering method.
背景技术Background technique
用户在经由终端设备(比如台式计算机或者移动终端设备)浏览互联网时所发生的上网行为(例如，在浏览器地址栏中键入页面地址或者点击页面中的超链接)会产生大量通信数据When users browse the Internet via terminal devices (such as desktop computers or mobile terminal devices), surfing behaviors (for example, typing a page address in the browser address bar or clicking a hyperlink on a page) will generate a large amount of communication data
当需要对用户产生的这些通信数据进行分析时，由于产生的数据量较大，且存在大量“脏数据”(即，与用户真实的上网行为，例如，用户手动行为无关的数据)较多。这样会导致对数据的处理速度较慢，并影响数据分析的准确性，以至于影响了多个层面的数据质量，而且需要的存储空间较大。When it is necessary to analyze the communication data generated by the user, due to the large amount of data generated, there is a large amount of "dirty data" (that is, data that is not related to the user's real online behavior, such as the user's manual behavior). This will result in a slow data processing speed and affect the accuracy of data analysis, so that the data quality at multiple levels is affected, and a large storage space is required.
发明内容Contents of the invention
本发明的目的在于提供一种数据智能过滤方法，以解决上述背景技术中提出的问题。The purpose of the present invention is to provide a data intelligent filtering method to solve the problems raised in the above-mentioned background technology.
为实现上述目的，本发明提供如下技术方案：To achieve the above object, the present invention provides the following technical solutions:
一种数据智能过滤方法，包括如下步骤：获取用户与终端设备交互过程中产生的通信数据，提取所述通信数据的数据特征，判断所述数据特征是否满足数据过滤条件，若所述数据特征满足数据过滤条件，则对产生的所述通信数据进行过滤；接收用户发送的数据过滤规则；通过通讯芯片接收数据报文；根据所述数据过滤规则对所述数据报文进行过滤；及将过滤后的数据报文发送至终端设备。A data intelligent filtering method, comprising the steps of: acquiring communication data generated during the interaction between a user and a terminal device, extracting data features of the communication data, and judging whether the data features meet data filtering conditions, and if the data features meet According to the data filtering condition, the generated communication data is filtered; the data filtering rule sent by the user is received; the data message is received through the communication chip; the data message is filtered according to the data filtering rule; and the filtered The datagram is sent to the terminal equipment.
作为本发明进一步的方案：所述数据特征包括通信数据中的文件类型信息。As a further solution of the present invention: the data feature includes file type information in the communication data.
作为本发明进一步的方案：所述文件类型信息包括内容类型标识和/或文件类型标识。As a further solution of the present invention: the file type information includes a content type identifier and/or a file type identifier.
作为本发明进一步的方案：所述数据过滤规则包括以下一种或者多种：应用的标识、协议类型、应用服务器的IP地址、端口号以及报文内容。As a further solution of the present invention: the data filtering rules include one or more of the following: application identification, protocol type, IP address of the application server, port number, and message content.
作为本发明进一步的方案：接收所述用户发送的黑名单及白名单，所述黑名单及白名单根据所述数据过滤规则设置的。As a further solution of the present invention: receiving the blacklist and whitelist sent by the user, the blacklist and whitelist are set according to the data filtering rules.
作为本发明进一步的方案：接收所述用户发送的黑名单及白名单，所述黑名单及白名单根据所述数据过滤规则设置的。As a further solution of the present invention: receiving the blacklist and whitelist sent by the user, the blacklist and whitelist are set according to the data filtering rules.
作为本发明进一步的方案：所述内容类型标识是由所述通信数据的http包头解析而来，所述文件类型标识是由所述通信数据的http包体解析而来。As a further solution of the present invention: the content type identifier is parsed from the http packet header of the communication data, and the file type identifier is parsed from the http packet body of the communication data.
作为本发明再进一步的方案：当所述数据报文的信息特征不在黑名单及白名单时，提示用户是否接收所述数据报文。As a further solution of the present invention: when the information feature of the data message is not in the black list or the white list, the user is prompted whether to receive the data message.
与现有技术相比，本发明的有益效果是：本发明获取用户与终端设备交互产生的通信数据并提取该通信数据中的数据特征，判断该数据特征是否满足数据过滤条件，若满足，则对通信数据进行过滤，能够对用户与终端设备的交互数据进行过滤，使据能够更加准确的反映用户的真实行为。Compared with the prior art, the beneficial effect of the present invention is: the present invention acquires the communication data generated by the interaction between the user and the terminal device and extracts the data features in the communication data, and judges whether the data features meet the data filtering conditions, and if so, then Filtering the communication data can filter the interaction data between the user and the terminal device, so that the data can more accurately reflect the real behavior of the user.
具体实施方式Detailed ways
下面对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。The following clearly and completely describes the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.
本发明实施例中，一种数据智能过滤方法，包括如下步骤：获取用户与终端设备交互过程中产生的通信数据，提取所述通信数据的数据特征，判断所述数据特征是否满足数据过滤条件，若所述数据特征满足数据过滤条件，则对产生的所述通信数据进行过滤；接收用户发送的数据过滤规则；通过通讯芯片接收数据报文；根据所述数据过滤规则对所述数据报文进行过滤；及将过滤后的数据报文发送至终端设备。所述数据特征包括通信数据中的文件类型信息。所述文件类型信息包括内容类型标识和/或文件类型标识。所述数据过滤规则包括以下一种或者多种：应用的标识、协议类型、应用服务器的IP地址、端口号以及报文内容。接收所述用户发送的黑名单及白名单，所述黑名单及白名单根据所述数据过滤规则设置的。接收所述用户发送的黑名单及白名单，所述黑名单及白名单根据所述数据过滤规则设置的。所述内容类型标识是由所述通信数据的http包头解析而来，所述文件类型标识是由所述通信数据的http包体解析而来。当所述数据报文的信息特征不在黑名单及白名单时，提示用户是否接收所述数据报文。In an embodiment of the present invention, a data intelligent filtering method includes the following steps: acquiring communication data generated during the interaction between a user and a terminal device, extracting data features of the communication data, and judging whether the data features meet data filtering conditions, If the data feature satisfies the data filtering condition, then filter the generated communication data; receive the data filtering rule sent by the user; receive the data message through the communication chip; and process the data message according to the data filtering rule filtering; and sending the filtered data message to the terminal device. The data feature includes file type information in the communication data. The file type information includes a content type identifier and/or a file type identifier. The data filtering rules include one or more of the following: application identification, protocol type, IP address of the application server, port number, and message content. Receive the blacklist and whitelist sent by the user, the blacklist and whitelist are set according to the data filtering rules. Receive the blacklist and whitelist sent by the user, the blacklist and whitelist are set according to the data filtering rules. The content type identifier is parsed from the http packet header of the communication data, and the file type identifier is parsed from the http packet body of the communication data. When the information feature of the data packet is not in the blacklist or the whitelist, prompt the user whether to receive the data packet.
在本发明的一个实施例中，当所述终端设备接收到所述数据报文后，并根据数据过滤规则对所述数据报文进行过滤。所述电子设备获取所述数据报文的信息特征，所述信息特征包括，但不限于：应用的标识(应用名称和版本号)、协议类型(Type)、应用服务器的IP地址、端口号以及报文内容(字符长度：Len、字符值：Data)等。In an embodiment of the present invention, after the terminal device receives the data message, it filters the data message according to a data filtering rule. The electronic device acquires information features of the data message, and the information features include, but are not limited to: application identification (application name and version number), protocol type (Type), IP address of the application server, port number, and Message content (character length: Len, character value: Data), etc.
当所述数据报文的信息特征在所述黑名单时，所述电子设备将所述数据报文舍弃。当所述数据报文的信息特征在所述白名单时，所述电子设备保留所述数据报文，控制所述终端设备将过滤后的数据报文发送所述至用户。When the information feature of the data packet is in the blacklist, the electronic device discards the data packet. When the information feature of the data packet is in the white list, the electronic device retains the data packet, and controls the terminal device to send the filtered data packet to the user.
当所述数据报文的信息特征不在所述黑名单及所述白名单时，所述电子设备提示用户是否接收所述数据报文。当用户选择接收所述数据报文时，所述电子设备控制所述第二处理设备将所述数据报文发送所述至用户。当所述数据报文被拒绝接收时(例如，没有接收到用户同意接收所述数据报文的信号)，所述电子设备将所述数据报文丢弃。When the information feature of the data packet is not in the blacklist or the whitelist, the electronic device prompts the user whether to receive the data packet. When the user chooses to receive the data packet, the electronic device controls the second processing device to send the data packet to the user. When the data packet is refused to be received (eg, no signal from the user agreeing to receive the data packet is received), the electronic device discards the data packet.
对于本领域技术人员而言，显然本发明不限于上述示范性实施例的细节，而且在不背离本发明的精神或基本特征的情况下，能够以其他的具体形式实现本发明。因此，无论从哪一点来看，均应将实施例看作是示范性的，而且是非限制性的，本发明的范围由所附权利要求而不是上述说明限定，因此旨在将落在权利要求的等同要件的含义和范围内的所有变化囊括在本发明内。此外，应当理解，虽然本说明书按照实施方式加以描述，但并非每个实施方式仅包含一个独立的技术方案，说明书的这种叙述方式仅仅是为清楚起见，本领域技术人员应当将说明书作为一个整体，各实施例中的技术方案也可以经适当组合，形成本领域技术人员可以理解的其他实施方式。It will be apparent to those skilled in the art that the invention is not limited to the details of the above-described exemplary embodiments, but that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Accordingly, the embodiments should be regarded in all points of view as exemplary and not restrictive, the scope of the invention being defined by the appended claims rather than the foregoing description, and it is therefore intended that the scope of the invention be defined by the appended claims rather than by the foregoing description. All changes within the meaning and range of equivalents of the elements are embraced in the present invention. In addition, it should be understood that although this specification is described according to implementation modes, not each implementation mode only contains an independent technical solution, and this description in the specification is only for clarity, and those skilled in the art should take the specification as a whole , the technical solutions in the various embodiments can also be properly combined to form other implementations that can be understood by those skilled in the art.
Claims (8)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711327019.7A CN108173676A (en) | 2017-12-13 | 2017-12-13 | A data intelligent filtering method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711327019.7A CN108173676A (en) | 2017-12-13 | 2017-12-13 | A data intelligent filtering method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108173676A true CN108173676A (en) | 2018-06-15 |
Family
ID=62525784
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711327019.7A Pending CN108173676A (en) | 2017-12-13 | 2017-12-13 | A data intelligent filtering method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108173676A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110688628A (en) * | 2019-10-09 | 2020-01-14 | 北京达佳互联信息技术有限公司 | Multimedia information processing method and device and server |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102413013A (en) * | 2011-11-21 | 2012-04-11 | 北京神州绿盟信息安全科技股份有限公司 | Network abnormal behavior detection method and device |
| US9413722B1 (en) * | 2015-04-17 | 2016-08-09 | Centripetal Networks, Inc. | Rule-based network-threat detection |
| CN106385450A (en) * | 2016-09-13 | 2017-02-08 | 宇龙计算机通信科技(深圳)有限公司 | Data filtering method and system |
| US20170099197A1 (en) * | 2015-10-02 | 2017-04-06 | Ixia | Network Traffic Pre-Classification Within VM Platforms In Virtual Processing Environments |
-
2017
- 2017-12-13 CN CN201711327019.7A patent/CN108173676A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102413013A (en) * | 2011-11-21 | 2012-04-11 | 北京神州绿盟信息安全科技股份有限公司 | Network abnormal behavior detection method and device |
| US9413722B1 (en) * | 2015-04-17 | 2016-08-09 | Centripetal Networks, Inc. | Rule-based network-threat detection |
| US20170099197A1 (en) * | 2015-10-02 | 2017-04-06 | Ixia | Network Traffic Pre-Classification Within VM Platforms In Virtual Processing Environments |
| CN106385450A (en) * | 2016-09-13 | 2017-02-08 | 宇龙计算机通信科技(深圳)有限公司 | Data filtering method and system |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110688628A (en) * | 2019-10-09 | 2020-01-14 | 北京达佳互联信息技术有限公司 | Multimedia information processing method and device and server |
| CN110688628B (en) * | 2019-10-09 | 2021-08-13 | 北京达佳互联信息技术有限公司 | Multimedia information processing method and device and server |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9185093B2 (en) | System and method for correlating network information with subscriber information in a mobile network environment | |
| CN110768933B (en) | Network flow application identification method, system and equipment and storage medium | |
| CN110708215B (en) | Deep packet inspection rule base generation method, device, network equipment and storage medium | |
| CN110149418A (en) | A kind of hidden tunnel detection method of DNS based on deep learning | |
| CN107222369A (en) | Recognition methods, device, switch and the storage medium of application program | |
| CN106416171A (en) | A feature information analysis method and device | |
| CN103916294A (en) | Identification method and device for protocol type | |
| CN108768921A (en) | A kind of malicious web pages discovery method and system of feature based detection | |
| WO2015021873A1 (en) | Method, platform server, and system of data pushing | |
| WO2015078122A1 (en) | Method and device for recognizing data stream | |
| CN106257887B (en) | A method for obtaining client hardware information based on WebSocket | |
| CN113949653A (en) | Encryption protocol identification method and system based on deep learning | |
| CN106789413B (en) | Method and device for detecting proxy internet surfing | |
| CN104184723B (en) | A kind of method for identifying application program, device and the network equipment | |
| CN114760267B (en) | Domain name blocking method, device, equipment, medium and program product | |
| CN114553730B (en) | Application identification method and device, electronic equipment and storage medium | |
| CN114979073B (en) | Address information acquisition system, method, electronic device and storage medium | |
| CN107592554A (en) | Live video retransmission method and device | |
| CN108173676A (en) | A data intelligent filtering method | |
| CN104410724A (en) | Method for realizing device type recognition in intelligent gateway based on HTTP protocol | |
| CN107070882A (en) | A kind of method of WIFI wireless datas real time parsing | |
| CN105516200A (en) | Cloud system security processing method and device | |
| CN111225038B (en) | Server access method and device | |
| CN105100246A (en) | Network flow management and control method based on downloaded resource name | |
| CN111245770B (en) | Method, apparatus and computer storage medium for user account management |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180615 |
|
| RJ01 | Rejection of invention patent application after publication |