CN106951431A - A kind of cluster monitoring collecting method - Google Patents

Abstract

The invention discloses a kind of cluster monitoring collecting method, including：Business module pushes cluster monitoring data to monitoring module, and the monitoring data is preserved in monitoring module in the form of the data structure KVData based on key-value pair；Clustered node Redis is initialized, and cluster monitoring data carry out landing storage using Hash table, while using the index of set storage non-zero；Two kinds of storage modes of Hash table and set based on Redis, statistical module uses unified monitoring data statistical method, carries out the statistics of cluster monitoring data in single measurement period；With n minutes frequencies once, the non-zero in KVData is gathered, monitoring data is reported into data center.This method has acquisition mode unified, first counts and reports afterwards, memory space needed for cluster monitoring data center is small, and monitored item extension is convenient, the characteristics of cluster monitoring data acquisition reports efficiency high, acquisition module and low business module degree of coupling.

Description

A cluster monitoring data collection method

technical field

The invention relates to the technical field of IT cluster monitoring, in particular to a cluster monitoring data collection method.

Background technique

With the rapid development of Internet technology, people's lifestyles are constantly changing, becoming more and more closely connected with the Internet, and people enjoy the convenience brought by Internet services all the time. The daily use of Internet services makes its service quality and stability more important. In order to provide stable services that support high concurrency, Internet service providers usually use distributed cluster deployment services, and cluster monitoring services are an important guarantee for improving the stability of Internet service clusters. However, it is impossible for any cluster service to be 100% reliable. Cluster monitoring can help development and maintenance personnel quickly locate, troubleshoot, and restore services. The carrier of the cluster monitoring service is data. The monitoring service is an automatic abnormality detection and discovery service. The data is the object of its detection and an important basis for judging the operation status of the service.

In the existing cluster services, generally the number of nodes is large, there are many monitoring items, and different modules have different data collection formats, and the cost of data management and analysis is high. And each node usually reports data at a fixed frequency, the amount of data collected by the monitoring system is huge, and it is likely to become a bottleneck of the cluster monitoring service. In order to fully and accurately reflect the running status of the cluster service program, the first task is to efficiently collect and report the cluster monitoring data. Therefore, it is necessary to set up a reasonable data storage structure and an efficient data statistics method, adopt a unified storage format for all monitoring items, aggregate the monitoring data before reporting the data, reduce the back-end load, and efficiently collect the statistical data and reporting to realize the efficient collection of cluster monitoring data.

Contents of the invention

The object of the present invention is to provide a cluster monitoring data collection method in order to solve the above-mentioned defects in the prior art.

The purpose of the present invention can be achieved by taking the following technical solutions:

A cluster monitoring data collection method, said method comprising the following steps:

S1. The business module pushes cluster monitoring data to the monitoring module. The cluster monitoring data is stored in the monitoring module in the form of a data structure KVData based on key-value pairs, where the key corresponds to a single monitoring item of a single node, and the value corresponds to a specific monitoring item. The value of , the cluster monitoring data is addressed by node ID and single node monitoring item Item;

S2, the cluster node Redis is initialized, the cluster monitoring data is stored using the hash table provided by Redis, and the index of non-zero monitoring data is stored in the collection provided by Redis;

S3. Based on the hash table and set storage methods provided by Redis, the statistics module adopts a unified monitoring data statistics method to perform statistics on cluster monitoring data in a single statistical cycle;

S4. The statistical data reporting module traverses the collection at a frequency of n minutes to obtain the index of non-zero data, and according to the obtained index set, find out the statistical value of the monitoring data corresponding to the non-zero data index in the hash table of Redis, Report the data.

Further, the step S2 specifically includes:

S2.1. Initialize the hash table. Each cluster node creates a hash table in its Redis. The hash table is named as its node ID. Different monitoring items are used as different fields of the hash table. The hash table The value of the field in the field stores the data of the corresponding monitoring item;

S2.2. Initialize the collection. When the cluster node is initialized, each cluster node creates an empty collection in its Redis, where the Key of the collection is set to the node ID, and the collection is used to store the index of non-zero monitoring data.

Further, the step S3 specifically includes:

S3.1. Obtain the timestamp clock, wherein the timestamp clock records the starting point of the statistical cycle. If the timestamp clock is 0, set the timestamp clock as the current time of the system. If the timestamp clock is not 0, skip this step;

S3.2, generate a transaction;

S3.3. Generate a command for accumulating monitoring data to the hash table monitoring item;

S3.4. Generate a command to write a non-zero monitoring data index into the set;

S3.5. Execute the command to collect the cluster monitoring data in a single statistical period.

Further, the step S4 specifically includes:

S4.1, generate a random number Rdm_value, set the lock overtime;

S4.2, set the lock lock_redis_ID;

S4.3. Obtain all members in the set and save the non-zero monitoring item index;

S4.4. Traverse the non-zero monitoring items in the hash table according to the index, read the data and report the data;

S4.5, Redis generates transactions;

S4.6, Redis generates a command to clear the hash table data;

S4.7, Redis generates a command to clear the collection;

S4.8, execute the order;

S4.9, release the lock lock_redis_ID;

S4.10. Set the timestamp clock to 0;

S4.11. Circularly report data to the data center at a frequency of n minutes.

Further, the key of the key-value pair-based data structure KVData is composed of a node ID and a single-node monitoring item Item connected through a string, and a comma is used between the two substrings to logically separate the two data, The key of the KVData string type is finally saved in the form of "ID, Item".

Further, the monitoring items include general performance indicators of each module under the cluster RPC call framework, and the performance indicators include: interface call time-consuming, call success rate, data reporting volume, specific monitoring data and exceptions of different business logics data.

Further, in the step S3, the statistics module aggregates the monitoring data of a single node according to the node ID and the monitoring item, so as to reduce the space required for storing the monitoring data and reduce the amount of subsequent monitoring data to be reported.

Further, the command to accumulate monitoring data to the hash table monitoring item is "HINCRBY node ID monitoring item name monitoring data value";

The command to write the non-zero monitoring data index to the set is "SADD node ID monitoring item name";

In the step S3.5, if the Redis has been locked, put the cluster monitoring data into the waiting queue.

Further, the command to obtain all members in the set is "SMEMBERS node ID";

The command that the Redis generates to clear the hash table data is "SMEMBERS node ID";

The command generated by Redis to clear the set is "HSET node ID monitoring item name 0".

Further, the value of n is 2.

Compared with the prior art, the present invention has the following advantages and effects:

(1) Unify the data collection format KVData of monitoring items in different modules, which reduces the difficulty of data statistics and processing, and effectively improves the efficiency of data collection and reporting.

(2) Use the key-value pair data structure to manage monitoring data, which can smoothly increase monitoring items and has strong scalability.

(3) Use the memory database Redis to manage KVData, and the reading and writing speed is fast.

(4) Provide monitoring data statistics methods to avoid system performance loss caused by frequent reporting of large amounts of data by each node of the cluster, and achieve high efficiency of monitoring data reporting.

(5) During the statistical process, the Redis collection is used to store the non-zero monitoring item index to reduce the effective data search time.

(6) The data reported to the data center is aggregated data based on node IDs and monitoring items, which improves the efficiency of data reporting and reduces the load on the back-end monitoring data center.

Description of drawings

Fig. 1 is a logical step diagram of the cluster monitoring data acquisition method disclosed by the present invention;

Fig. 2 is the framework diagram of the cluster monitoring data acquisition system of the inventive method;

Fig. 3 is the flow chart of initialization of the single-node monitoring data acquisition service of the inventive method;

Fig. 4 is the flow chart of cluster single-point monitoring data statistics of the inventive method;

Fig. 5 is a flow chart of monitoring data reporting in the method of the present invention.

detailed description

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

Example

The present embodiment discloses a cluster monitoring data acquisition method. FIG. 1 is a logical step diagram of the cluster monitoring data acquisition method disclosed in the present invention. FIG. 2 is a cluster monitoring data acquisition system architecture diagram of the inventive method. The method specifically includes the following step:

S1. The business module pushes cluster monitoring data to the monitoring module. The cluster monitoring data is stored in the monitoring module in the form of a data structure KVData based on key-value pairs, where the key corresponds to a single monitoring item of a single node, and the value corresponds to a specific monitoring item. The value of the cluster monitoring data is addressed by node ID and single node monitoring item Item.

In a specific embodiment, the key of the key-value pair-based data structure KVData is composed of a node ID and a single-node monitoring item Item in the form of a string connection, and a comma is used between the two substrings to separate the two data in a logical manner. Separation is performed on the above, and the key of the KVData string type is finally saved in the form of "ID, Item".

The cluster monitoring data information is addressed through the node ID and the single point monitoring item Item.

The monitoring items include general performance indicators of each module under the cluster RPC call framework, and the performance indicators include: interface call time consumption, call success rate, data reporting volume, specific monitoring data and abnormal data of different business logics.

In this step S1, different monitoring items of all business modules adopt a unified storage format, and the use of key-value pair data structure has the characteristics of good scalability, and indexing monitoring data through node IDs and monitoring items speeds up monitoring data addressing speed.

The key point is to unify the storage structure of different monitoring items in each module.

S2, the cluster node Redis is initialized, the cluster monitoring data is stored using the hash table provided by Redis, and the index of non-zero monitoring data is stored in the collection provided by Redis;

The flow chart of this step is shown in accompanying drawing 3, specifically includes:

S2.1. When the cluster nodes are initialized, each cluster node creates a hash table in its Redis, where the hash table is named as its node ID, and different monitoring items are used as different fields of the hash table. The hash table The value of the field in the field stores the data of the corresponding monitoring item;

S2.2. When the cluster nodes are initialized, each cluster node creates an empty collection in its Redis, where the Key of the collection is set to the node ID, and the collection is used to store the index of non-zero monitoring data.

In a specific implementation, the cluster node Redis is initialized, and the cluster monitoring data is stored using the hash table provided by Redis, and at the same time, the index of the non-zero monitoring data is stored collectively. That is, the data in the set corresponds to the field name of the corresponding non-zero monitoring data in the hash table. The field names of the non-zero monitoring data can be obtained by traversing the elements in the collection.

The focus is on initializing hash tables and collections in Redis.

S3. Based on the two storage methods provided by Redis, the hash table and the set, the statistics module adopts a unified monitoring data statistics method to perform statistics on the cluster monitoring data in a single statistical cycle;

The flow chart of this step is shown in accompanying drawing 4, specifically includes:

S3.1. Obtain the timestamp clock. If the timestamp clock is 0, set the timestamp clock as the current system time. If the timestamp clock is not 0, skip this step;

S3.2, generate a transaction;

S3.3. Generate a command for accumulating monitoring data to the hash table monitoring item;

S3.4. Generate a command to write a non-zero monitoring data index into the set;

S3.5. Execute the command to collect the cluster monitoring data in a single statistical cycle.

In the step S3, the statistics module aggregates the monitoring data of a single node according to the node ID and the monitoring items, so as to reduce the space required for storing the monitoring data and reduce the amount of subsequent monitoring data to be reported.

In this step S3, the time stamp clock records the starting point of the statistics period.

In this step S3.3, the monitoring data is accumulated to the hash table monitoring item, and the command "HINCRBY node ID monitoring item name monitoring data value" is used.

In this step S3.4, a non-zero data index is written into the collection, and the command "SADD node ID monitoring item name" is used.

In this step S3.5, if the Redis has been locked, put the cluster monitoring data into the waiting queue.

The focus is on the statistics of different monitoring item data for each module of a single node.

S4. The statistical data reporting module traverses the collection at a frequency of n minutes to obtain the index of non-zero data, and according to the obtained index set, find out the statistical value of the monitoring data corresponding to the non-zero data index in the hash table of Redis, Report the data.

The flow chart of this step is shown in accompanying drawing 5, specifically includes:

S4.1, generate a random number Rdm_value, set the lock overtime;

S4.2, set the lock lock_redis_ID;

S4.3. Obtain all members in the set and save the non-zero monitoring item index;

S4.4. Traverse the non-zero monitoring items in the hash table according to the index, read the data and report the data;

S4.5, Redis generates transactions;

S4.6, Redis generates a command to clear the hash table data;

S4.7, Redis generates a command to clear the collection;

S4.8, execute the order;

S4.9, release the lock lock_redis_ID;

S4.10. Set the timestamp clock to 0;

S4.11. Circularly report data to the data center at a frequency of n minutes.

In a preferred embodiment, the value of n is 2, that is, the data reporting logic is cycled every 2 minutes. It should be emphasized that the above-mentioned values are not limited to this technical solution.

In this step S4.3, all members in the set are obtained, and the command "SMEMBERS node ID" is used.

In this step S4.6, the data in the hash table is cleared, and the command "HSET node ID monitoring item name 0" is used.

In this step S4.7, the set is cleared, and the command "SINTER node ID" is used.

In this step S4.10, the time stamp clock is set to 0, which is used to end the previous statistical cycle and start a new statistical cycle.

In the method for efficiently reporting cluster monitoring data, each time the single-point monitoring data of a cluster node is successfully reported, the original monitoring data is cleared to avoid repeated statistics.

The collection and reporting of the monitoring data of each node in the cluster is realized through the data reporting module. This method can reduce the coupling between the data reporting service and the business module by setting the CPU affinity for the data reporting module, so that the collection and reporting of the cluster monitoring data and Business modules do not affect each other.

The focus is on reporting the aggregated monitoring data to the data center.

To sum up, the cluster monitoring data collection method provides a unified key-value pair data structure KVData for different monitoring items of each node in the cluster, which is used to store monitoring data, and unified monitoring data collection methods. Tables and collections are used to maintain single-point monitoring data, aggregate monitoring data according to monitoring items, store statistical data on the ground, and report data according to policies, effectively improving the efficiency of cluster monitoring data collection and reporting.

The above-mentioned embodiment is a preferred embodiment of the present invention, but the embodiment of the present invention is not limited by the above-mentioned embodiment, and any other changes, modifications, substitutions, combinations, Simplifications should be equivalent replacement methods, and all are included in the protection scope of the present invention.

Claims (10)

1. a kind of cluster monitoring collecting method, it is characterised in that methods described comprises the following steps：

S1, business module push cluster monitoring data to monitoring module, and the cluster monitoring data are in monitoring module with based on key assignments To data structure KVData form preserve, key therein corresponds to the single monitored item of individual node, and value corresponds to specific The numerical value of monitored item, the cluster monitoring data are addressed by node ID and single node monitored item Item；

S2, clustered node Redis are initialized, and the cluster monitoring data carry out landing storage using the Redis Hash tables provided, Simultaneously using the index of the Redis set storage non-zero monitoring datas provided；

S3, two kinds of storage modes of the Hash table based on Redis offers and set, statistical module are counted using unified monitoring data Method, carries out the statistics of cluster monitoring data in single measurement period；

S4, statistics reporting module were traveled through with n minutes frequencies once to be gathered, the index of acquisition non-zero, and according to The indexed set gone out, non-zero is found in Redis Hash table and indexes corresponding monitoring data statistical value, and data are carried out Report.

2. a kind of cluster monitoring collecting method according to claim 1, it is characterised in that the step S2 is specifically wrapped Include：

S2.1, initialization Hash table, each clustered node create a Hash table in its Redis, and wherein Hash table is named For its node ID, different monitoring Item is as the different field of the Hash table, and the value of field then stores corresponding prison in Hash table Control the data of item；

S2.2, initialization set, when clustered node is initialized, each clustered node creates a null set in its Redis, The Key wherein gathered is set to the node ID, and the set is used for the index for storing non-zero monitoring data.

3. a kind of cluster monitoring collecting method according to claim 1, it is characterised in that the step S3 is specifically wrapped Include：

S3.1, acquisition timestamp clock, wherein the timestamp clock records the starting point of measurement period, if timestamp clock For 0, then timestamp clock is set to the current time in system, if timestamp clock is not 0, skips this step；

S3.2, generation affairs；

S3.3, the order for generating the monitoring data that added up toward Hash table monitored item；

S3.4, the order for generating past set write-in non-zero monitoring data index；

S3.5, the cluster monitoring data performed in order, the single measurement period of statistics.

4. a kind of cluster monitoring collecting method according to claim 1, it is characterised in that the step S4 is specifically wrapped Include：

S4.1, generation random number R dm_value, set lock time-out time；

S4.2, setting lock lock_redis_ID；

All members in S4.3, acquisition set, preserve non-zero monitoring entry index；

S4.4, according to index traversal Hash table in non-zero monitored item, read data carry out data report；

S4.5, Redis generate affairs；

S4.6, Redis generate the order for emptying Hash table data；

S4.7, Redis generate the order for emptying set；

S4.8, perform order；

S4.9, release lock lock_redis_ID；

S4.10, timestamp clock set to 0；

S4.11, with n minutes frequency cycle reported datas once to data center.

5. a kind of cluster monitoring collecting method according to claim 1, it is characterised in that described based on key-value pair Data structure KVData key is made up of node ID and single node monitored item the Item form connected by character string, two sub- words Two data are logically separated using comma in the middle of symbol string, the keys of KVData character string types it is final with " ID, Item " form is preserved.

6. a kind of cluster monitoring collecting method according to claim 1, it is characterised in that the monitored item includes every Individual module performance indications universal under cluster RPC invocation framenorts, the performance indications include：Interface interchange takes, called into Power, the data amount of reporting, the specific monitoring data of different business logic and abnormal data.

7. a kind of cluster monitoring collecting method according to claim 1, it is characterised in that counted in the step S3 Module, according to node ID and monitored item, is polymerize to single node monitoring data, is reduced and space and subtract needed for supervising data storage Few follow-up monitoring data amount of reporting.

8. a kind of cluster monitoring collecting method according to claim 3, it is characterised in that

The order toward the cumulative monitoring data of Hash table monitored item is " HINCRBY node IDs monitoring key name monitoring data value "；

The order toward set write-in non-zero monitoring data index is " SADD node IDs monitor key name "；

In the step S3.5, if Redis has been locked, cluster monitoring data are put into waiting list.

9. a kind of cluster monitoring collecting method according to claim 4, it is characterised in that

The order for obtaining all members in set is " SMEMBERS node IDs "；

The order that the Redis generations empty Hash table data is " SMEMBERS node IDs "；

The Redis generations empty the order of set for " HSET node IDs monitor key name 0 ".

10. a kind of cluster monitoring collecting method according to claim 1 or 4, it is characterised in that the n values are 2。