WO2018136088A1

WO2018136088A1 - OTxIT NETWORK INSPECTION SYSTEM USING ANOMALY DETECTION BASED ON CLUSTER ANALYSIS

Info

Publication number: WO2018136088A1
Application number: PCT/US2017/014440
Authority: WO
Inventors: Takashi Isobe
Original assignee: Hitachi, Ltd.
Priority date: 2017-01-20
Filing date: 2017-01-20
Publication date: 2018-07-26

Abstract

Example implementations described herein are directed to network inspection systems (NIS) configured to provide a security solution covering operational technology (OT) networks. Example implementations can involve a Fast Fourier Transform (FFT) block and historical data generator to calculate bandwidth and sensor values cyclically across each connection. Example implementations also involve an OT protocol inspector and OT command/format dictionary to extract OT layer data over each connection. Example implementations further involve cluster analysis by using sensor or other data m addition to network attributes for each connection, and provide an interface to indicate anomalies related to such data across each connection.

Description

OTxIT NETWORK INSPECTION SYSTEM

USING ANOMALY DETECTION BASED ON CLUSTER ANALYSIS

BACKGROUND

Field

[0001] The present disclosure relates to cloud service platforms, and more specifically, for anomaly detection of Internet of Things (IoT) devices and Operational Technology (OT) devices.

[0002] Related art IoT systems may involve a cloud service platform, a network and OT devices, as illustrated in FIG. 1. IoT networks can involve multi-hierarchical networks. The network can be divided into multi-layer networks such as an information technology (IT) network, an OT maintenance network and an OT control network. A gateway may be utilized in the boundary between the IT network and the OT network. For examples, in the boundary of IT network and OT maintenance network, there is a field gateway to gather data concentrated at the domain gateway. In another example, in the boundary of OT maintenance network and OT control network, there is a domain gateway to gather raw data from OT devices. A cloud gateway is connected to the service platform, and gathers data from all OT areas.

[0003] In related art IoT systems, the physical platform can be configured to support multi-domain IoT services such as smart grid, predictive maintenance and optimized factory using virtual machine of service platform and virtual wide area network of IT network. On the other hand, the requirement for quality and security is different between services.

[0004] The cloud and IT network are protected by related art IT security technologies such as firewall, Intrusion Detection System (IDS) and Intrusion Prevention System (IPS), as illustrated in FIG. 2. These technologies implement security based on rules and signatures through assuming that the communications using Transmission Control Protocol/Internet Protocol (TCP/IP) from office applications pass through the boundary of network.

[0005] In IoT systems that include OT maintenance networks and OT control networks, the communications using domain-specific protocol from OT devices pass not only between OT network and IT network but also within each OT network. For example, cyclic domain- specific communications can be used in order to control and maintain OT devices.

[0006] Therefore, each gateway is configured to support various functions to maintain the quality and health of each related art IoT system. Such functions can include providing a function to visualize quality and the status of domain-specific and cyclic communications, to detect the anomaly of domain-specific and cyclic communications and to optimize domain- specific and cyclic communications.

[0007] Related art IT security system implementations such as firewall, IDS and IPS may not suitable for OT networks as illustrated in FIG. 3, as the security is implemented based on the rules and signatures assuming that the communications using TCP IP from office application pass through the boundary of network. OT networks utilize OT-specific protocols and cyclic communications to control OT devices over multi-layer networks, and have security implementations that involve OT domain specific protocols and cyclic communications over multi-hierarchical networks.

SUMMARY

[0008] Example implementations in the present disclosure extend the developments in technology in the form of the Network Inspection System (NIS), which facilitates high-speed extraction of network attributes through each connection, and anomaly detection using highspeed cluster analysis.

[0009] The NIS implementations can achieve real-time extraction of network attributes for many IoT devices in the way of wait free multi-threading. The wait free multi-threading achieves high-speed access between multi-threads and multi-tables, and achieves the performance of 10 Gigabits per seconds (Gbps) for a 2M connection at the interval of 10 seconds. Network attributes can include of packet header information (Media Access Control (MAC), IP, TCP, port, timestamp, byte counts, packet counts, etc.) and analysis information (Round Trip Time (RTT), loss ratio, throughput, etc.)

[0010] The NIS implementations can also achieve anomaly detection based on utilizing high-speed cluster analysis. The NIS implementations can extract an unexpected event with unusual network attributes using a density and grid base clustering algorithm. Such example implementations divide the area into multi-grids, and calculate the density by each grid and makes clusters by merging the grid next to the grid with larger density. If a new cluster appears in a blank grid or if the top of cluster moves into blank grid, then an anomaly is determined to occur. Cluster analysis is utilized due to low complexity, small calculation and intuitive visualization.

[0011] NTS was implemented for security and Quality of Service (QoS) monitoring, and has been enhanced for securing a critical infrastructure like railway and electric power. NIS currently implements security to defend multi-hierarchical networks including IT network and a part of OT maintenance network. However, these current our technologies have issues in the point of not covering whole OT networks because it doesn't consider OT/domain- specific protocol, cyclic and knowledge.

[0012] Example implementations described herein enhance NIS implementations to cover OT networks. Example implementations described herein can extend NIS implementations for microgrid, energy trading, or factory optimization implementations. In microgrid implementations, NIS implementations can be extended to the microgrid manager. In the case of energy trading implementations, NIS can be extended to the meter data management and trading system.

[0013] In the case of microgrid implementations, example implementations can logically detect anomaly using actual energy consumption data from microgrid using NIS implementations. Example implementations can confirm logical anomalies as actual anomalies using sensor data in combination with network attributes and OT protocol inspection from NIS implementations.

[0014] In energy trading implementations, NIS is available for facilitating energy trading systems in addition to microgrid systems. For factory optimizations, example implementations can extend NIS technology to facilitate edge analytics that are adaptable to the OT control network.

[0015] Aspects of the present disclosure includes a system configured to analyze packets communicated between first devices connected to an Informational Technology (IT) network and second devices connected to an Operational Technology (OT) network. The system can include a memory configured to store management information including first information extracted from header information of the packets, and second information derived from payload information of the packets, the payload information including one or more OT commands; and a processor, configured to, for receipt of the packets, determine, from the payload information of the packets, at least one of a command type of the OT command, transmission or reception sequence information, an identifier for a source device or target device, and data derived from the second device; and update the second information with at least one of the command type of the OT command, the transmission or reception sequence information, the identifier for the source or target device, and the data derived from the second device.

[0016] Aspects of the present disclosure further includes a computer program which can include instructions for analyzing packets communicated between first devices connected to an Informational Technology (IT) network and second devices connected to an Operational Technology (OT) network. The instructions can include managing management information including first information extracted from header information of the packets, and second information derived from payload information of the packets, the payload information including one or more OT commands; for receipt of the packets, determining, from the payload information of the packets, at least one of a command type of the OT command, transmission or reception sequence information, an identifier for a source device or target device, and data derived from the second device; and updating the second information with at least one of the command type of the OT command, the transmission or reception sequence information, the identifier for the source or target device, and the data derived from the second device. The computer program can be stored on a non-transitory computer readable medium and configured to be executed by one or more processors.

[0017] Aspects of the present disclosure further includes a method for analyzing packets communicated between first devices connected to an Informational Technology (IT) network and second devices connected to an Operational Technology (OT) network. The method can include managing management information including first information extracted from header information of the packets, and second information derived from payload information of the packets, the payload information including one or more OT commands; for receipt of the packets, determining, from the payload information of the packets, at least one of a command type of the OT command, transmission or reception sequence information, an identifier for a source device or target device, and data derived from the second device; and updating the second information with at least one of the command type of the OT command, the transmission or reception sequence information, the identifier for the source or target device, and the data derived from the second device.

[0018] Aspects of the present disclosure further includes a system for analyzing packets communicated between first devices connected to an Informational Technology (IT) network and second devices connected to an Operational Technology (OT) network. The system can include means for managing management information including first information extracted from header information of the packets, and second information derived from payload information of the packets, the payload information including one or more OT commands; for receipt of the packets, means for determining, from the payload information of the packets, at least one of a command type of the OT command, transmission or reception sequence information, an identifier for a source device or target device, and data derived from the second device; and means for updating the second information with at least one of the command type of the OT command, the transmission or reception sequence information, the identifier for the source or target device, and the data derived from the second device.

BRIEF DESCRIPTION OF DRAWINGS

[0019] FIG. 1 illustrates an example related art IoT system configuration.

[0020] FIG. 2 illustrates an example quality and security breakdown of a related art IoT system configuration.

[0021] FIG. 3 illustrates an example of security implementations for IT systems and OT systems.

[0022] FIG. 4 illustrates example functions for extending NIS implementations to facilitate various functions, in accordance with an example implementation.

[0023] FIG. 5 illustrates an example configuration of a system, in accordance with an example implementation.

[0024] FIG. 6 illustrates a NIS and Attribute Table, in accordance with an example implementation.

[0025] FIG. 7A illustrates an example OT Command / Format Dictionary Table, in accordance with an example implementation. [0026] FIG. 7B illustrates an example flow diagram for the OT Protocol Inspector, in accordance with an example implementation.

[0027] FIG. 8A illustrates an example FFT range table, in accordance with an example implementation.

[0028] FIG. 8B illustrates a flow chart related with the FFT calculation, in accordance with an example implementation.

[0029] FIG. 9 illustrates a configuration of function blocks, in accordance with an example implementation.

[0030] FIG. 10 illustrates an example of the Initial Value Table for Division and Normalization Table, in accordance with an example implementation.

[0031] FIG. 1 1 illustrates an example Density Table, in accordance with an example implementation.

[0032] FIG. 12 illustrates a density-and-grid-base cluster analysis, in accordance with an example implementation.

[0033] FIG. 13 illustrates an example flow for Clustering, in accordance with an example implementation.

[0034] FIG. 14 illustrates an example detection of anomaly using the density-and-grid- base cluster analysis, in accordance with an example implementation.

[0035] FIGS. 15(a) to 15(e) illustrate example anomaly detection as specified for the OT protocol and cyclic communication by each TCP/IP connection, in accordance with an example implementation.

[0036] FIG. 16 illustrates example anomaly detection over each connection from using cluster analysis, in accordance with an example implementation.

[0037] FIG. 17 illustrates example anomaly detection over each connection shown over a map, in accordance with an example implementation.

[0038] FIG. 18 illustrates an example computing environment with an example computer device suitable for use in example implementations. DETAILED DESCRIPTION

[0039] The following detailed description provides further details of the figures and example implementations of the present application. Reference numerals and descriptions of redundant elements between figures are omitted for clarity. Terms used throughout the description are provided as examples and are not intended to be limiting. For example, the use of the term "automatic" may involve fully automatic or semi-automatic implementations involving user or administrator control over certain aspects of the implementation, depending on the desired implementation of one of ordinary skill in the art practicing implementations of the present application. Selection can be conducted by a user through a user interface or other input means, or can be implemented through a desired algorithm. Example implementations as described herein can be utilized either singularly or in combination and the functionality of the example implementations can be implemented through any means according to the desired implementations.

[0040] In example implementations, NIS implementations are applied to achieve anomaly detection over OT networks. FIG. 4 illustrates example functions for extending NIS implementations to facilitate various functions. Such functions include a fast fourier transform (FFT) block and historical data generator to calculate the cyclic of bandwidth and sensor data by each connection, and an OT protocol inspector and OT command/format dictionary to extract OT layer data from each connection. Additional functions can include cluster analysis using sensor/cyclic/Global Positioning Satellite (GPS) data in addition to network (NW) attributes by each connection, and a graphic user interface (GUI) showing anomalies related to OT sensor/cyclic/location data by each connection over the map.

[0041] FIG. 5 illustrates an example configuration of a system, in accordance with an example implementation. The example configuration includes Mirror/Tap 102-1, 102-2, NIS 103-1, 103-2, Attribute Table 104-1 , 104-2, Cluster Analysis 100-1, 100-2, GUI 105 and Set Up Host 106. The system as illustrated in FIG. 5 is connected with networks 120-1, 120-2 and 120-3, and analyzes the communications between hosts 1 10-1, 1 10-2 and hosts 1 10-3, 1 10-4. In an example implementation, the hosts 110-3, 1 10-4 can be in the form of an OT device such as a sensor, PLC (Programmable Logic Controller), SCADA (Supervisory Control And Data Acquisition), a camera, a device aggregating a sensor, or other similar devices depending on the desired implementation. Further, the hosts 110-1, 110-2 can be in the form of a personal computer (PC) or a server which collects and analyzes the data derived from the hosts 110-3, 1 10-4, or other similar devices depending on the desired implementation. In an example implementation, host 1 10-4 may be included into a moving apparatus such as a car. Moreover, host 1 10-3, NIS 103-2, Attribute Table 104-2, Cluster Analysis 100-2, Mirror/Tap 102-2, and OT Control Network 120-3 may be included into moving device like a car. Depending on the desired implementation, Mirror/Tap 102-1, 102-2 may be in the configuration of a domain or field gateway configured to facilitate receipt of data from networks such as IT Network 120-1, OT Maintenance Network 120-2, and OT Control Network 120-3. Further Mirror/Tap 102-1, 102-2 can be configured to copy a packet communicated between hosts 1 10-1, 1 10-2 and hosts 1 10-3, 1 10-4 and send the copied data to NIS 103. The packet can include a header and payload, and the payload can include OT command.

[0042] FIG. 6 illustrates a NIS 103 and Attribute Table 104, in accordance with an example implementation. Example functionality of the NIS is described, for example, in US Patent Application No. 15/205,699, hereinafter incorporated by reference in its entirety for all purposes.

[0043] Attribute Table 104 can contain historical information for each connection 104- 10, and live connections, depending on the desired implementation. Historical information 104-10 can include a plurality of entries each made up of a plurality of feature amounts of a connection session. Historical information 104-10 can include header information 104-1, header analysis information 140-2, OT cyclic control information 104-3, OT protocol information 104-4, OT sensor/machine information 104-5, and OT data/place information 104-6. Features of header information 104-1 can include device identifier (ID), internet protocol (IP) address, sequence information, acknowledgement information, and timestamp. The device ID (e.g., hostl, host3) identifies the devices for a session. The IP address information (e.g., IP1, IP3) indicates the IP addresses of each of devices connected for the session. The port information (e.g., portl, port3) indicate the port numbers of the devices connected via the session. The sequence information (e.g., seql, seq3) indicate the transmission sequence numbers of the devices connected through the session. The acknowledgement information (e.g., ackl, ack3) indicates the reception sequence numbers of the devices connected via the session. The packet information (e.g., pktl, pkt3) indicates the transmission packet counts of the devices connected via the session. The bit information (e.g., bitl, bit3) indicates the transmission bit numbers of the devices connected via the session. Timestamp information (e.g., timestampl, timestamp3) indicates the timestamp in which a packet is received.

[0044] Features of header analysis information 104-2 can include bandwidth (BW) information, average transmission bandwidth (ave) information, packet loss rate information (rate), and round trip delay information (rrt). The bandwidth information (e.g., BW1, BW3) indicates the most recent transmission bandwidths of the devices connected via the session. The average transmission bandwidth information (e.g., avel, ave3) indicates the average transmission bandwidths of the devices connected via the session. The packet loss rate information (e.g., lossl, loss3) indicates the packet loss rates of the devices connected via the session. The round trip delay information (e.g., rrtl, rrt3) indicate the round-trip delay times of the devices connected via the session.

[0045] Features of OT cyclic control information 104-3 can include bandwidth frequency and sensor frequency. Such features are derived from execution of a fast Fourier transform on sensor information and bandwidth information as described in FIGS. 7A, 7B, 8A and 8B.

[0046] Features of OT protocol information 104-4 can include communication information (com), reply information (reply), sequence information (seq), and acknowledgement information (ack).

[0047] Features of OT sensor/machine information 104-5 can include sensor number (sensor#), machine number (machine#), and global positioning satellite information (GPS#) associated with the machine or sensor.

[0048] Features of OT Data/Place information 104-6 can include data associated with the OT device, the longitude of the OT device, and the latitude of the OT device.

[0049] In the example implementation of FIG. 6, NIS 103 received packets at TCP/IP header inspector from Mirror/Tap 102.

[0050] TCP/IP Header Inspector 103-1 extracts the header information from packet header 104-1, and records them into header info area 104-1 of Attribute Table 104. Analysis thread of NIS 103 analyzes using the header information, and records the result into Header Analysis Info Area 104-2 of Attribute Table 104. [0051] OT Protocol Inspector 103-6 receives the information of packet payload, which can include one or more OT commands, from TCP/IP Header Inspector 103-1, and extracts the information about OT protocol information 104-4, OT sensor/machine information 104-5 and OT data/place information 104-6 based on OT Command / Format Dictionary Table 103- 7.

[0052] FIG. 7A illustrates an example OT Command / Format Dictionary Table 103-7, in accordance with an example implementation. The OT Command / Format Dictionary Table can includes information involving the command, reply, sequence number (seq), acknowledge number (ack), source/target number of sensor/GPS/machine and data. Sequence number shows the number of command. Acknowledge number shows the number of replies. OT Protocol Inspector 103-6 is configured to derive the command, reply, seq, ack, source/target number and data from received packet payload information based on the bit range information defined in the OT Command / Format Dictionary Table 103-7. Further, OT Protocol Inspector 103-6 stores the derived information into attribute table 104 as explained below in FIG. 7B.

[0053] FIG. 7B illustrates an example flow diagram for the OT Protocol Inspector 103-6, in accordance with an example implementation. At 701 , the OT protocol inspector 103-6 determines the type of command or reply that is received, and records the command or reply into the OT Protocol Information Area 104-4 of Attribute Table 104. At 702, the OT Protocol Inspector 103-6 checks the sequence number of command or the acknowledge number of the reply, and records it into OT protocol Information Area 104-4 of Attribute Table 104. The sequence number is the ID for the exact operation sequence used to operate the command and reply. At 703, the OT Protocol Inspector determines the source or target devices included in the received packet, and determines the corresponding data at 704. For example, if the data is sensor/machine data, then the number of sensors or machines is recorded into the area of OT Sensor/Machine Information 104-5 of Attribute Table 104, and the data of sensor or machine is recorded into the area of OT Data/Place Information 104-6 of Attribute Table 104. In another example, if the data is from a GPS sensor, the number of GPS sensors is recorded into the area of OT Sensor/Machine Information 104-5 of Attribute Table 104, and the data of longitude and latitude is recorded into OT Data/Place Information 104-6 of Attribute Table 104. GPS 103-8 of NIS 103 may also store latitude and longitude data into the area of OT Data/Place Information 104-6 of Attribute Table 104. Through this flow, the OT Protocol Inspector 103-6 is configured to extract OT data from OT devices and store the OT data into the OT data table (e.g., OT cyclic control information 104-3, OT Protocol information 104-4, OT Sensor/Machine information 104-5, OT Data/Place information 104-6).

[0054] From the execution of FIG. 7B, the Fast Fourier Transform (FFT) Thread 103-5 calculates the cyclic information by using FFT or other cumulative value functions (e.g. Laplace transform) at each time range (e.g., month, day, hour, minute) on historical sensor data or bandwidth data based on the desired implementation, and extracting the characteristic frequency of OT cyclic communications.

[0055] FIG. 8 A illustrates an example FFT range table 103-3, in accordance with an example implementation. The FFT Range Table records the number of sensors and the interval time used for the FFT calculation based on the condition of each connection. The FFT range in the FFT range table 103-3 indicates the interval time for calculating FFT. The values recorded in this table may be updated by Set Up Host 106. Threshold indicates a threshold value for a value determined from an FFT calculation to be indicative of an anomaly. The threshold can be set by a domain operator, or by other methods depending on the desired implementation.

[0056] FIG. 8B illustrates a flow chart related with the FFT calculation, in accordance with an example implementation. At 801, the Historical Data Generator 103-4 first reads the FFT Range matching header information from FFT Range Table 103-3, and then, extracts historical bandwidth (BW) or sensor data from Attribute Table 104. In this flow, the Historical Data Generator 103-4 compares the incoming connection with TCP/IP data from previous connections. At 802, the FFT thread 103-5 calculates the cyclic control information using FFT for the historical sensor data or bandwidth data, and extracts the characteristic frequency of cyclic communications whose amplitude is more than preset threshold at 803. In this flow, the FFT thread 103-5 conducts the flow of 802 based on the identifier of the sensor or OT device, and the corresponding threshold set for the sensor or OT device. The extracted characteristic frequency is recorded into the area of OT Cyclic Control Information 104-3 of Attribute Table 104 at 804. The data stored from NIS 103 to Attribute Table 104 is analyzed to detect anomalies at Cluster Analysis 102.

[0057] FIG. 9 illustrates a configuration of function blocks, in accordance with an example implementation. Receiver 1025 of Cluster Analysis 102 receives data including network attributes and OT sensor/cyclic/command information at the preset interval over each connection, and Normal izer 1024 normalizes the data by using Initial Value Table for Division 1081 and Normalization Table 1028. Then, Quantization 1023 quantizes the data using Initial Value Table for Division 1081 , Normalization Table 1028 and Density Table 1027. After Density Table 1027 stores the density data by each grid, Clustering 1022 classifies the each grid into clusters, conducts a density-and-grid-base cluster analysis and outputs the result to Sender 1021. Sender 1021 sends the result to GUI 105.

[0058] FIG. 10 illustrates an example of the Initial Value Table for Division 1081 and Normalization Table 1028, in accordance with an example implementation. Initial Value Table for Division 1081 stores the number of grids for each parameter, the quantum size of density, the number of parameters, the type of parameters, the maximum value of each parameter, the minimum value of each parameter and the division interval of each parameter. Normalization Table 1028 stores the normalized value of each parameter by each connection. Normalizer 1024 analyzes the input data from Attribute Table 104, and stores the maximum and minimum value of each parameter into Initial Value Table for Division 1081. Then, Normalizer 1024 calculates the division interval of each parameter from dividing the differentiation between the maximum and minimum value of each parameter by the number of grids. Furthermore, Normalizer 1024 normalizes the value by dividing the offset value of each parameter by the division interval, and stores the normalized value into Normalization Table 1028 for each connection. In the example of FIG. 10, Parameter 1 is the Destination IP (DIP) address and the maximum value of the DIP is 192.0.20.150 and minimum value of the DIP is 192.0.2.3. Parameter2 is the data sensed by sensor #4 and the maximum value of the sensed data is 777 and minimum value of the sensed data is 0. The parameters (Parameter 1 and Parameter2) can be determined in advance, and any attribute within the attribute table 104 can be a parameter. Further, combinations of attributes within the attribute table 104 can be a parameter. For example, cluster analysis 100 can calculate an rapid acceleration (e.g. increase) of the value of the sensed data based on the items in the attribute table 104, and the rapid acceleration of the value of the sensed data can be a parameter.

[0059] FIG. 11 illustrates an example Density Table 1027, in accordance with an example implementation. Density Table 1027 stores density, quantum density and cluster number by each combination of multi-normalized parameters. Quantization 1023 counts up the density of Density Table 1027 according to the combination of normalized parameters of Normalization Table 1028 by each connection. The normalized parameters can be indicative of the location on the grid for cluster analysis. For example, the connection of number 1, which has the combination of the parameter 1 of 0 and parameter2 of 6 in Normalization Table 1028, counts up the density in the combination of the normalized parameter 1 of 0 and the normalized parameter2 of 6 in Density Table 1027. Furthermore, Quantization 1023 calculates the quantum density using the quantum size of density written in Initial Value Table for Division 1081. The value of quantum density is retrieved from dividing the density by the quantum size of density. The density or the quantum density is used for the density- and-gr id-base cluster analysis by Clustering 1022.

[0060] FIG. 12 illustrates a density-and-grid-base cluster analysis, in accordance with an example implementation. As the first step, the scatter plot 2201 between parameter 1 and parameter2 is divided into multiple grids 2202 by a division interval. After the grids are created, the density is calculated by each grid 2203. The result is stored into Density Table 1027. At the second step, Clustering 1022 conducts density-and-grid-base cluster analysis by using Density Table 1027. The density can be representative of the number of connections associated with the devices for parameter 1 and parameter2.

[0061] FIG. 13 illustrates an example flow for Clustering 1022, in accordance with an example implementation. After calculating the density of each grid, Clustering 1022 determines the grid with largest density at 2301. In the example of FIG. 12, the largest density grid is identified at 2204. At 2302, the grid is set as an independent cluster. The cluster number of 1 is assigned to the grid. Next, Clustering 1022 finds the grid with next largest density at 2303. If the grid with next largest density exists at 2304, Clustering 1022 judges if the neighbor with the larger density exists at 2305. If yes, the grid is merged with neighbor as shown at 2307 and as illustrated at 2205 in FIG. 12. If no, the grid is set as an independent cluster as shown in FIG. 12 at 2206. The flow from 2303 to 2307 are looped, and all grids are classified to multi-clusters. In the case of FIG. 12, the grids included within the cluster at 2210 are set to 1 , and the grids included within the cluster at 2209 are set to 2.

[0062] FIG. 14 illustrates an example detection of anomaly using the density-and-grid- base cluster analysis, in accordance with an example implementation. Specifically, FIG. 14 illustrates an example of clustering in a normal situation when no anomaly is directed at 2401, 2402, 2403 and 2404, with clustering as shown at 2409. FIG. 14 also illustrates an example of an anomalous cluster detection at 2405, 2406, 2407, at 2408 with anomalous clusters detected based on the clusters of 2412, 2413 and 241 1.

[0063] In the example of FIG. 14, there are two examples of anomaly detection. In a first example of anomaly detection, the cluster analysis detects the appearance of new cluster as shown at 2412 in comparison to the previous or normal cluster of 2410. If the new cluster appears in a blank grid, Clustering 1022 determines the cluster to be an anomaly. In another example of anomaly detection, there is a cluster shift to the top to the grid as shown in 241 1 in comparison to the previous or normal cluster of 2410. If the top of the cluster moves into a blank grid, Clustering 1022 also detects the shift as an anomaly.

[0064] Through the example implementations, the user can detect anomalies occurring from OT sensor/cyclic/protocol information at high speed while specifying that place over a map by each TCP/IP connection, as shown at FIGS. 15 to 17. The example implementations can facilitate anomaly detection over the scatter plot between OT attributes, or between OT attribute and IT attribute. Cluster analysis can be utilized to detect anomalies based on place/location/network/per cycle, and so on.

[0065] FIGS. 15(a) to 15(e) illustrate example anomaly detection as specified for the OT protocol and cyclic communication by each TCP/IP connection, in accordance with an example implementation. As shown in FIG. 15(a), 15(b) and 15(e), OT parameters can be compared with other OT parameters to determine data clusters that are indicative of anomalies. In additional example implementations as shown in FIGS. 15(c) and 15(d), OT parameters can be compared with IT parameters to determine data clusters that are indicative of anomalies. As shown in FIG. 15(e), there can be an OT sensor# having a rapid acceleration (e.g. increase) in the values of the OT sensors (e.g. a programmable logic controller (PLC)). In such an example, when the rapid acceleration of the sensor values occurs beyond a threshold, the behavior can be indicative of an anomaly.

[0066] FIG. 16 illustrates example anomaly detection over each connection from using cluster analysis, in accordance with an example implementation. As shown in FIG. 16, a dashboard 1601 can be provided to compare OT parameters with a desired OT or IT parameter, as described with respect to FIGS. 15(a) to 15(d). A dashboard 1602 can also be provided to indicate when clusters are generated or when new clusters are detected for selected OT or IT parameters. Another dashboard 1603 can also be provided to indicate connections from both OT and IT devices, which can include information such as wide area network (WAN) internet protocol (IP) address, local area network (LAN) IP address, WAN port, LAN port, WAN round trip delay (RRT), OT sequence number, OT sensor data, OT frequency, and loss in WAN connection.

[0067] FIG. 17 illustrates example anomaly detection over each connection shown over a map, in accordance with an example implementation. In the example of FIG. 17, there is an example table 1700 which logs when clusters are formed, when new clusters are generated, or when a cluster moves, and also provides additional parameters for context. Such parameters can include the time, the associated alert level based on the cluster, the WAN port range, the OT frequency range, the latitude and longitude information of the OT device, and so on depending on the desired implementation. A map of each connected device can be provided as shown at 1701, in comparison to the physical locations of devices over the desired map (e.g., buildings, street level, etc.) The cluster UI as illustrated in FIGS. 15(a) to 15(d) can also be utilized in the dashboard of FIG. 17, as illustrated at 1702. An additional UI can be provided to indicate the number of connections for a given IT or OT device over time, as illustrated at 1703.

[0068] Through the example implementations, high-speed anomaly detection can be facilitated over each TCP/IP connection over the scatter plot between OT data/attributes, or between OT data/attributes and IT data/attributes, while specifying the locations of such devices over a map as illustrated in FIGS. 15 to 17.

[0069] FIG. 18 illustrates an example computing environment with an example computer device suitable for use in example implementations. For example, the example computer device outlined below can be utilized as at least one of a hosts 101, Mirror/Tap 102, an NIS 103, Attribute Table 104, Cluster Analysis 100 and GUI 105 as described in FIG. 5 to facilitate the functions therein. Computer device 1805 in computing environment 1800 can include one or more processing units, cores, or processors 1810, memory 1815 (e.g., RAM, ROM, and/or the like), internal storage 1820 (e.g., magnetic, optical, solid state storage, and/or organic), and/or I/O interface 1825, any of which can be coupled on a communication mechanism or bus 1830 for communicating information or embedded in the computer device 1805. [0070] Computer device 1805 can be communicatively coupled to input/user interface 1835 and output device/interface 1840. Either one or both of input/user interface 1835 and output device/interface 1840 can be a wired or wireless interface and can be detachable. Input/user interface 1835 may include any device, component, sensor, or interface, physical or virtual, that can be used to provide input (e.g., buttons, touch-screen interface, keyboard, a pointing/cursor control, microphone, camera, braille, motion sensor, optical reader, and/or the like). Output device/interface 1840 may include a display, television, monitor, printer, speaker, braille, or the like. In some example implementations, input/user interface 1835 and output device/interface 1840 can be embedded with or physically coupled to the computer device 1805. In other example implementations, other computer devices may function as or provide the functions of input/user interface 1835 and output device/interface 1840 for a computer device 1805.

[0071] Examples of computer device 1805 may include, but are not limited to, highly mobile devices (e.g., smartphones, devices in vehicles and other machines, devices carried by humans and animals, and the like), mobile devices (e.g., tablets, notebooks, laptops, personal computers, portable televisions, radios, and the like), and devices not designed for mobility (e.g., desktop computers, other computers, information kiosks, televisions with one or more processors embedded therein and/or coupled thereto, radios, and the like).

[0072] Computer device 1805 can be communicatively coupled (e.g., via I/O interface 1825) to external storage 1845 and network 1850 for communicating with any number of networked components, devices, and systems, including one or more computer devices of the same or different configuration. Computer device 1805 or any connected computer device can be functioning as, providing services of, or referred to as a server, client, thin server, general machine, special-purpose machine, or another label.

[0073] I/O interface 1825 can include, but is not limited to, wired and/or wireless interfaces using any communication or I/O protocols or standards (e.g., Ethernet, 802.1 lx, Universal System Bus, WiMax, modem, a cellular network protocol, and the like) for communicating information to and/or from at least all the connected components, devices, and network in computing environment 1800. Network 1850 can be any network or combination of networks (e.g., the Internet, local area network, wide area network, a telephonic network, a cellular network, satellite network, and the like). [0074] Computer device 1805 can use and/or communicate using computer-usable or computer-readable media, including transitory media and non-transitory media. Transitory media include transmission media (e.g., metal cables, fiber optics), signals, carrier waves, and the like. Non-transitory media include magnetic media (e.g., disks and tapes), optical media (e.g., CD ROM, digital video disks, Blu-ray disks), solid state media (e.g., RAM, ROM, flash memory, solid-state storage), and other non-volatile storage or memory.

[0075] Computer device 1805 can be used to implement techniques, methods, applications, processes, or computer-executable instructions in some example computing environments. Computer-executable instructions can be retrieved from transitory media, and stored on and retrieved from non-transitory media. The executable instructions can originate from one or more of any programming, scripting, and machine languages (e.g., C, C++, C#, Java, Visual Basic, Python, Perl, JavaScript, and others).

[0076] Processors) 1810 can execute under any operating system (OS) (not shown), in a native or virtual environment. One or more applications can be deployed that include logic unit 1860, application programming interface (API) unit 1865, input unit 1870, output unit 1875, and inter-unit communication mechanism 1895 for the different units to communicate with each other, with the OS, and with other applications (not shown). The described units and elements can be varied in design, function, configuration, or implementation and are not limited to the descriptions provided.

[0077] In some example implementations, when information or an execution instruction is received by API unit 1865, it may be communicated to one or more other units (e.g., logic unit 1860, input unit 1870, output unit 1875). In some instances, logic unit 1860 may be configured to control the information flow among the units and direct the services provided by API unit 1865, input unit 1870, output unit 1875, in some example implementations described above. For example, the flow of one or more processes or implementations may be controlled by logic unit 1860 alone or in conjunction with API unit 1865. The input unit 1870 may be configured to obtain input for the calculations described in the example implementations, and the output unit 1875 may be configured to provide output based on the calculations described in example implementations.

[0078] Memory 1815 can be configured to store management information that can include first information extracted from header information of the packets, and second information derived from payload information of the packets, the payload information including one or more OT commands as illustrated in FIG. 6. Memory may also be configured to a command dictionary that can include bit range information corresponding to each of the at least one of a command type of the OT command, the transmission or reception sequence information, the identifier for a source device or target device, and the data derived from the second device as illustrated in FIG. 7A.

[0079] Processors) 1810 can be configured to, for receipt of the packets, determine, from the payload information of the packets, at least one of a command type of the OT command, transmission or reception sequence information, an identifier for a source device or target device, and data derived from the second device; and update the second information with at least one of the command type of the OT command, the transmission or reception sequence information, the identifier for the source or target device, and the data derived from the second device as illustrated in the flows of FIG. 6 and FIG. 7B.

[0080] For each connection indicated by at least a portion of the first information, processor(s) 1810 can be configured to conduct a FFT operation on data extracted from at least one of the first information and the second information for the each connection to determine a characteristic frequency having an amplitude exceeding a threshold; and store the characteristic frequency exceeding the threshold in the memory as a part of the second information as illustrated in the flow of FIG. 8B.

[0081] Processors) 1810 can be configured to conduct normalization of at least two types of data, the at least two types of data including a first type of data extracted from the second information and a second type of data extracted from at least one of the first information and the second information, based on a division interval associated with each type of the at least two types of data; quantize the normalized data into grids, the grids divided by using the at least two types of data, based on the number of connections; and conduct cluster analysis on the grids of the normalized data as illustrated in the flow of FIG. 9. Processor(s) 1810 is also configured to quantize the normalized data into the grids through a determination of density of the normalized data, and conduct cluster analysis on the grids of normalized data based on a determination of a cluster number for each of the grids from the density of the normalized data as illustrated, for example, in the flows of FIGS. 12, 13 and 14. [0082] Processors) 1810 is configured to conduct anomaly detection based on the cluster analysis indicative of at least one of a formation of a new cluster and a movement of an existing cluster, and for the cluster analysis indicative of at least one of the formation of a new cluster and the movement of the existing cluster, designate a cluster associated with the at least one of the formation of the new cluster and the movement of the existing cluster as an anomaly as illustrated, for example, in the flows of FIGS. 12, 13 and 14.

[0083] Processors) 1810 is configured to facilitate a graphical user interface (GUI) indicative of Transmission Control Protocol/Internet Protocol (TCP/IP) connections associated with the grids of the normalized data, wherein the GUI is configured to display clusters based on the grids, the clusters displaying a comparison of the first type of data with the second type of data as illustrated in FIGS. 16 and 17.

[0084] Processor(s) 1810 can be configured to determine the at least one of the command type of the OT command, the transmission or reception sequence information, the identifier for a source device or target device, and the data derived from the second device through a matching of the payload information of the packets with the command dictionary as described in FIG. 6 through the OT Command/Format Dictionary 103-7 and the OT Protocol Inspector 103-6.

[0085] Processors) 1810 can be configured to generate a plurality of clusters from the grids, by setting a grid having a largest density from the grids as a first cluster; for each subsequent ones of the grids having a next highest density and a larger density neighboring grid than the each subsequent ones of the grids, merging the each subsequent ones of the grids into the larger density neighboring grid; and for each subsequent ones of the grids having the next highest density and not having the larger density neighboring grid than the each subsequent ones of the grids, generating another cluster for the each subsequent ones of the grids as illustrated in FIG. 12 and FIG. 13. Processors) 1810 can then be configured to conduct the cluster analysis based on the generated plurality of clusters as illustrated in FIG. 14.

[0086] Some portions of the detailed description are presented in terms of algorithms and symbolic representations of operations within a computer. These algorithmic descriptions and symbolic representations are the means used by those skilled in the data processing arts to convey the essence of their innovations to others skilled in the art. An algorithm is a series of defined steps leading to a desired end state or result. In example implementations, the steps carried out require physical manipulations of tangible quantities for achieving a tangible result.

[0087] Unless specifically stated otherwise, as apparent from the discussion, it is appreciated that throughout the description, discussions utilizing terms such as "processing," "computing," "calculating," "determining," "displaying," or the like, can include the actions and processes of a computer system or other information processing device that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system's memories or registers or other information storage, transmission or display devices.

[0088] Example implementations may also relate to an apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, or it may include one or more general-purpose computers selectively activated or reconfigured by one or more computer programs. Such computer programs may be stored in a computer readable medium, such as a computer-readable storage medium or a computer-readable signal medium. A computer-readable storage medium may involve tangible mediums such as, but not limited to optical disks, magnetic disks, read-only memories, random access memories, solid state devices and drives, or any other types of tangible or non-transitory media suitable for storing electronic information. A computer readable signal medium may include mediums such as carrier waves. The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. Computer programs can involve pure software implementations that involve instructions that perform the operations of the desired implementation.

[0089] Various general-purpose systems may be used with programs and modules in accordance with the examples herein, or it may prove convenient to construct a more specialized apparatus to perform desired method steps. In addition, the example implementations are not described with reference to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the example implementations as described herein. The instructions of the programming language(s) may be executed by one or more processing devices, e.g., central processing units (CPUs), processors, or controllers. [0090] As is known in the art, the operations described above can be performed by hardware, software, or some combination of software and hardware. Various aspects of the example implementations may be implemented using circuits and logic devices (hardware), while other aspects may be implemented using instructions stored on a machine-readable medium (software), which if executed by a processor, would cause the processor to perform a method to carry out implementations of the present application. Further, some example implementations of the present application may be performed solely in hardware, whereas other example implementations may be performed solely in software. Moreover, the various functions described can be performed in a single unit, or can be spread across a number of components in any number of ways. When performed by software, the methods may be executed by a processor, such as a general purpose computer, based on instructions stored on a computer-readable medium. If desired, the instructions can be stored on the medium in a compressed and/or encrypted format.

[0091] Moreover, other implementations of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the teachings of the present application. Various aspects and/or components of the described example implementations may be used singly or in any combination. It is intended that the specification and example implementations be considered as examples only, with the true scope and spirit of the present application being indicated by the following claims.

Claims

CLAIMS What is claimed is:

1. A system configured to analyze packets communicated between first devices connected to an Informational Technology (IT) network and second devices connected to an Operational Technology (OT) network, the system comprising: a memory configured to store management information comprising first information extracted from header information of the packets, and second information derived from payload information of the packets, the payload information comprising one or more OT commands; a processor, configured to: for receipt of the packets: determine, from the payload information of the packets, at least one of a command type of the OT command, transmission or reception sequence information, an identifier for a source device or target device, and data derived from the second device; and update the second information with at least one of the command type of the OT command, the transmission or reception sequence information, the identifier for the source or target device, and the data derived from the second device.

2. The system of claim 1, wherein the processor is configured to: for each connection indicated by at least a portion of the first information, conduct a fast Fourier transform (FFT) operation on data extracted from at least one of the first information and the second information for the each connection to determine a characteristic frequency having an amplitude exceeding a threshold; and store the characteristic frequency exceeding the threshold in the memory as a part of the second information.

3. The system of claim 1, wherein the processor is configured to: conduct normalization of at least two types of data, the at least two types of data comprising a first type of data extracted from the second information and a second type of data extracted from at least one of the first information and the second information, based on a division interval associated with each type of the at least two types of data; quantize the normalized data into grids, the grids divided by using the at least two types of data, based on the number of connections; and conduct cluster analysis on the grids of the normalized data.

4. The system of claim 3, wherein the processor is configured to quantize the normalized data into the grids through a determination of density of the normalized data, and wherein the processor is configured to conduct cluster analysis on the grids of normalized data based on a determination of a cluster number for each of the grids from the density of the normalized data.

5. The system of claim 3, wherein the processor is configured to conduct anomaly detection based on the cluster analysis indicative of at least one of a formation of a new cluster and a movement of an existing cluster, and for the cluster analysis indicative of at least one of the formation of a new cluster and the movement of the existing cluster, designate a cluster associated with the at least one of the formation of the new cluster and the movement of the existing cluster as an anomaly.

6. The system of claim 3, wherein the processor is configured to facilitate a graphical user interface (GUI) indicative of Transmission Control Protocol/Internet Protocol (TCP/IP) connections associated with the grids of the normalized data, wherein the GUI is configured to display clusters based on the grids, the clusters displaying a comparison of the first type of data with the second type of data.

7. The system of claim 1, wherein the memory is configured to store a command dictionary comprising bit range information corresponding to each of the at least one of a command type of the OT command, the transmission or reception sequence information, the identifier for a source device or target device, and the data derived from the second device; wherein the processor is configured to determine the at least one of the command type of the OT command, the transmission or reception sequence information, the identifier for a source device or target device, and the data derived from the second device through a matching of the payload information of the packets with the command dictionary.

8. The system of claim 3, wherein the processor is configured to generate a plurality of clusters from the grids, by: setting a grid having a largest density from the grids as a first cluster; for each subsequent ones of the grids having a next highest density and a larger density neighboring grid than the each subsequent ones of the grids, merging the each subsequent ones of the grids into the larger density neighboring grid; and for each subsequent ones of the grids having the next highest density and not having the larger density neighboring grid than the each subsequent ones of the grids, generating another cluster for the each subsequent ones of the grids; wherein the processor is configured to conduct the cluster analysis based on the generated plurality of clusters.

9. A non-transitory computer readable medium storing instructions for analyzing packets communicated between first devices connected to an Informational Technology (IT) network and second devices connected to an Operational Technology (OT) network, the instructions comprising: managing management information comprising first information extracted from header information of the packets, and second information derived from payload information of the packets, the payload information comprising one or more OT commands; for receipt of the packets: determining, from the payload information of the packets, at least one of a command type of the OT command, transmission or reception sequence information, an identifier for a source device or target device, and data derived from the second device; and updating the second information with at least one of the command type of the OT command, the transmission or reception sequence information, the identifier for the source or target device, and the data derived from the second device.

10. The non-transitory computer readable medium of claim 9, wherein the instructions further comprise: for each connection indicated by at least a portion of the first information, conducting a fast Fourier transform (FFT) operation on data extracted from at least one of the first information and the second information for the each connection to determine a characteristic frequency having an amplitude exceeding a threshold; and storing the characteristic frequency exceeding the threshold in the memory as a part of the second information.

1 1. The non-transitory computer readable medium of claim 9, wherein the instructions further comprise: conducting normalization of at least two types of data, the at least two types of data comprising a first type of data extracted from the second information and a second type of data extracted from at least one of the first information and the second information, based on a division interval associated with each type of the at least two types of data; quantizing the normalized data into grids, the grids divided by using the at least two types of data, based on the number of connections; and conducting cluster analysis on the grids of the normalized data.

12. The non-transitory computer readable medium of claim 1 1, wherein the quantizing the normalized data into the grids is based on a determination of density of the normalized data, and wherein the conducting cluster analysis on the grids of normalized data based on a determination of a cluster number for each of the grids from the density of the normalized data.

13. The non-transitory computer readable medium of claim 1 1, wherein the conducting the anomaly detection is based on the cluster analysis being indicative of at least one of a formation of a new cluster and a movement of an existing cluster, and for the cluster analysis being indicative of at least one of the formation of a new cluster and the movement of the existing cluster, the instructions further comprise designating a cluster associated with the at least one of the formation of the new cluster and the movement of the existing cluster as an anomaly.

14. The non-transitory computer readable medium of claim 9, the instructions further comprising managing a command dictionary comprising bit range information corresponding to each of the at least one of a command type of the OT command, the transmission or reception sequence information, the identifier for a source device or target device, and the data derived from the second device; wherein the determining the at least one of the command type of the OT command, the transmission or reception sequence information, the identifier for a source device or target device, and the data derived from the second device comprises matching the payload information of the packets with the command dictionary.

15. The non-transitory computer readable medium of claim 1 1 , the instructions further comprising generating a plurality of clusters from the grids, the generating the plurality of clusters from the grids comprising: setting a grid having a largest density from the grids as a first cluster; for each subsequent ones of the grids having a next highest density and a larger density neighboring grid than the each subsequent ones of the grids, merging the each subsequent ones of the grids into the larger density neighboring grid; and for each subsequent ones of the grids having the next highest density and not having the larger density neighboring grid than the each subsequent ones of the grids, generating another cluster for the each subsequent ones of the grids;

wherein the conducting cluster analysis is based on the generated plurality of clusters.