CN107122490A - The data processing method and system of aggregate function in a kind of Querying by group - Google Patents

Abstract

The present application discloses a data processing method and system for aggregation functions in group query, including the main CPU dividing the data file to be queried into data blocks, and distributing the data blocks to multiple computing nodes; each computing node CPU respectively The data block is transmitted to the FPGA, and the FPGA is correspondingly connected to the computing node CPU through the PCIe interface; the FPGA obtains the data block, and performs SQL group query on the data block according to the calculation algorithm of the aggregation function in the group query Calculate the aggregation function, and return the calculation result to the main CPU through the calculation node CPU; the calculation algorithm is realized by using Opencl language parallel optimization, and transplanted on the FPGA; the main CPU merges the calculation results to obtain the final query result. The present application improves the processing capability of computing nodes through FPGA parallel acceleration, thereby improving the computing speed of group query aggregation functions.

Description

Data processing method and system for aggregate function in group query

technical field

The present application relates to the field of big data processing, in particular to a data processing method and system for aggregate functions in group query.

Background technique

In recent years, with the development of information technology, big data processing technology, as a high-tech product of the new era, has been widely used in various industries.

Distributed processing technology is used in big data processing, which can connect computing nodes of different types of instruction sets and architectures in different locations using communication networks to form a distributed processing system. Under the unified control and management of the main control unit, each computing node coordinates to complete the data processing task.

Group query calculation is a common data processing task. It can divide the processing objects of the same category into a group for calculation, and obtain the target aggregation function value, making the information in the data more clear at a glance. As the most popular fast general-purpose computing platform in the field of big data processing, the Spark big data processing system can support external devices to use SQL statements to perform group query and computing operations.

With the continuous increase of data storage scale and calculation scale, the SQL group query calculation operation of the database is facing a huge challenge. In today's era of information explosion, data is gradually showing the characteristics of massive volume, multiple types, and low value density, which has greatly increased the processing volume and complexity of SQL group query calculation operations in the database. Compared with the explosive growth rate of data, the computing power of computing nodes in the prior art is limited, so the processing time is also greatly increased. Therefore, how to effectively increase the speed of group query computing operations to meet the development needs of the current society has become a technical problem to be solved urgently by those skilled in the art.

Contents of the invention

The purpose of this application is to provide a data processing method and system for aggregate functions in group query, so as to improve the speed of group query aggregate function calculation operations on the big data platform.

In order to solve the above technical problems, the application provides a data processing method for aggregate functions in group query, including:

The main CPU divides the data file to be queried into data blocks, and distributes the data blocks to multiple computing nodes;

Each computing node CPU transmits the data block to the FPGA respectively, and the FPGA and the computing node CPU are correspondingly connected through a PCIe interface;

The FPGA acquires the data block, performs the calculation of the SQL group query aggregation function on the data block according to the calculation algorithm of the aggregation function in the group query, and returns the calculation result to the main CPU through the computing node CPU; The code of the calculation algorithm of the aggregation function in the group query adopts Opencl language parallel optimization, and is transplanted and deployed on the FPGA;

The main CPU combines the calculation results to obtain the final query result.

Optionally, before the main CPU divides the data file to be queried into data blocks, it also includes:

The main CPU parses the query statement;

The data file to be queried is loaded according to the query statement.

Optionally, obtaining the data block by the FPGA includes:

The FPGA applies for space in the local memory;

Loading the data block from the DDR into the space results in a data column.

Optionally, the calculation of the SQL group query aggregation function on the data block by the FPGA according to the calculation algorithm of the aggregation function in the group query includes:

loading data in said data column in a data vector;

Judging whether the data in the data vector satisfies the query condition of the aggregate function in the SQL group query calculation; if so, execute the calculation operation of the SQL group query aggregate function, and rewrite the corresponding bit data in the result vector by the first flag bit is the second flag;

Combining the result vectors according to the query condition of the aggregation function to obtain a calculation result.

Optionally, the calculation of the SQL group query aggregation function on the data block by the FPGA according to the calculation algorithm of the aggregation function in the group query includes:

The SQL group query aggregate function is calculated on the data block in the manner of unroll loop expansion according to the calculation algorithm of the aggregate function in the group query.

The present application also provides a data processing system for aggregation functions in group query, including:

Main CPU: used to divide the data file to be queried into data blocks, and distribute the data blocks to multiple computing nodes; receive and merge the calculation results of each FPGA for SQL group query aggregation function calculation to obtain the final query result, Wherein, the FPGA is correspondingly connected to the computing node CPU through a PCIe interface;

Computing node CPU: used to transmit the allocated data block to the FPGA, and transmit the calculation result returned by the FPGA to the main CPU;

The FPGA: used to obtain the data block, perform the SQL group query aggregation function calculation on the data block according to the calculation algorithm of the aggregation function in the group query, and return the data to the main CPU through the computing node CPU. Calculation results; the code of the calculation algorithm of the aggregation function in the group query is realized by using Opencl language parallel optimization, and transplanted and deployed on the FPGA.

Optionally, the main CPU is also used for:

Analyzing the query statement; loading the data file to be queried according to the query statement.

Optionally, when the FPGA obtains the data block, it is specifically used for:

Applying for a space in the local memory; and loading the data block from the DDR into the space to obtain a data column.

Optionally, when the FPGA performs the calculation of the SQL group query aggregation function on the data block according to the calculation algorithm of the aggregation function in the group query, it is specifically used for:

Load the data in the data column in the data vector; judge whether the data in the data vector satisfies the query condition of the aggregate function in the SQL group query; if so, execute the calculation operation of the SQL group query aggregate function, and send the result The data of the corresponding bit of the vector is rewritten from the first flag bit to the second flag bit; and the result vector is combined according to the query condition of the aggregation function to obtain the calculation result.

Optionally, the FPGA is specifically used to:

The SQL group query aggregate function is calculated on the data block in the manner of unroll loop expansion according to the calculation algorithm of the aggregate function in the group query.

In the data processing method of the aggregation function in the group query provided by the present application, the main CPU divides the data file to be queried into data blocks, and distributes the data blocks to a plurality of computing nodes; The block is transmitted to the FPGA, and the FPGA is correspondingly connected to the computing node CPU through the PCIe interface; the FPGA obtains the data block, and performs SQL group query aggregation function on the data block according to the calculation algorithm of the aggregation function in the group query Calculate, and return the calculation result to the main CPU through the calculation node CPU; the code of the calculation algorithm of the aggregation function in the group query adopts Opencl language parallel optimization, and transplants and deploys on the FPGA; the main CPU The CPU combines the calculation results to obtain the final query result.

It can be seen that compared with the prior art, the data processing method of the aggregation function in the group query provided by the present application constitutes a heterogeneous parallel accelerated computing system by setting FPGAs correspondingly connected to the CPUs of each computing node using PCIe interfaces. The characteristics of hardware parallelism, using the Opencl high-level comprehensive language on the FPGA to carry out the parallel optimization of the group query aggregation function calculation algorithm, in order to improve the calculation speed of the group query calculation of the data after the block. Through each FPGA, the computing power and computing speed of a single computing node can be effectively improved, and then the query results of the entire data file to be queried can be quickly obtained, and the speed of group query computing can be improved.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings that need to be used in the description of the embodiments of the present application. Of course, the following drawings related to the embodiments of the application describe only a part of the embodiments of the application, and those of ordinary skill in the art can obtain other The accompanying drawings, and other obtained drawings also belong to the protection scope of the present application.

Fig. 1 is the hardware scenario schematic diagram of the data processing method of aggregation function in the group query provided by the present application;

Fig. 2 is the flow chart of the data processing method of aggregate function in the group query provided by the present application;

Fig. 3 is the flow chart that FPGA carries out SQL grouping query aggregate function calculation to data block in Fig. 2;

Fig. 4 is the flow chart that FPGA performs SQL grouping query aggregate function calculation to two data columns;

FIG. 5 is a structural block diagram of a data processing system for aggregate functions in group query provided by the present application.

detailed description

In order to describe the technical solutions in the embodiments of the present application more clearly and completely, the technical solutions in the embodiments of the present application will be introduced below in conjunction with the drawings in the embodiments of the present application. Apparently, the described embodiments are only some of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

Please refer to Fig. 1, Fig. 1 is the schematic diagram of the hardware scene of the data processing method of aggregation function in the group query provided by the present application, including main server 101, sub-server 102, FPGA (Field Programmable Gate Array, Field Programmable Gate Array) 103 .

The main CPU of the main server 101 can complete corresponding database operations by responding to the host-side program, and realize human-computer interaction. The CPU of the sub-server 102, that is, the CPU of the computing node, is correspondingly connected with the FGPA 103 through the PCIe interface to form a computing node, which has the ability to process data independently. The sub-servers 102 in different computing nodes are connected through a communication network, and are uniformly controlled and managed by the main CPU of the main server 101 .

Please refer to FIG. 2 . FIG. 2 is a flow chart of the data processing method of the aggregation function in the group query provided by the present application. The method specifically includes the following steps:

Step 201: The main CPU divides the data file to be queried into data blocks.

The main CPU is the control center that manages various data operation processes in the entire big data processing system. When performing a group query operation on the database, since the data scale is generally relatively large, the main CPU first divides the data file to be queried into data blocks, so as to facilitate subsequent query operations.

Specifically, the data file to be queried may be obtained by the main CPU after receiving the query statement and loading it from a designated storage area according to the query statement.

Step 202: the main CPU distributes the data blocks to multiple computing nodes.

The main CPU distributes each data block to a plurality of computing nodes through a communication network, so that each computing node processes each data block in parallel.

Step 203: Each computing node CPU transmits the data block to the FPGA respectively.

As mentioned above, in the data processing method of the aggregation function in the group query provided by this application, the FPGA and the CPU of the computing node in the computing node are correspondingly connected through the PCIe interface, so that data transmission and communication can be realized. After the computing node CPU receives the data block allocated by the main CPU, it can transmit the data block to the FPGA through the data interface, so that the FPGA can process the data block and perform group query operations.

Step 204: FPGA obtains the data block.

Specifically, when the FPGA obtains the data block transmitted by the computing node, it first applies for enough storage space in its local memory, and then transfers the computing node CPU from the DDR (Double Data Rate, double-rate synchronous dynamic random access memory) The incoming data blocks are loaded into the requested storage space to obtain the data columns, so that in the subsequent steps, the calculation of the aggregate function in the parallel grouping query can be performed on the data columns.

It is worth mentioning that storing data blocks in the local memory of the FPGA can reduce the access and use of the global memory in the computing node communication network, thereby improving the computing performance of the computing nodes and increasing the speed of group query calculations.

Step 205: The FPGA calculates the aggregation function of the SQL group query on the data block according to the calculation algorithm of the aggregation function in the group query.

Since FPGA can realize heterogeneous accelerated computing, the code of the calculation algorithm of the aggregation function in the group query can be optimized and implemented in parallel using the Opencl high-level synthesis language, and can be efficiently transplanted and deployed to each FPGA, so that the SQL grouping of data blocks can be performed using the FPGA Query the computation of aggregate functions. Under the framework of the distributed system, each FPGA calculates the SQL group query aggregation function for the acquired data block according to the calculation algorithm of the aggregation function in the group query, and obtains the calculation result of the data block. Using the advantages of FPGA hardware parallelism can greatly increase the speed of data processing of each computing node and realize heterogeneous parallel acceleration.

It should be noted that the calculation of the aggregate function in the SQL group query described here can specifically be summation by group, calculation of the arithmetic mean value by group, maximum value by group, etc., and corresponding aggregation functions can be used respectively. implementation, the embodiment of this application does not limit the specific content of SQL group query aggregate function calculation.

Step 206: the FPGA returns the calculation result to the main CPU through the computing node CPU.

The FPGA in each computing node transmits the calculation result obtained in step 205 to the corresponding computing node CPU through the PCIe interface, and the computing node CPU further returns the calculation result to the main CPU through the communication network.

Step 207: The main CPU merges the calculation results to obtain the final query result.

Under the framework of a distributed system, the calculation results received by the main CPU from each computing node correspond to the data blocks allocated to each computing node. Therefore, the main CPU can combine the calculation results to obtain the data file to be queried. The final query result.

It can be seen that the data processing method of the aggregation function in the group query provided by the present application utilizes FPGA to form a heterogeneous parallel accelerated computing system. The reason why FPGA is used for heterogeneous calculation of database group query aggregation function is because FPGA has the advantage of hardware parallelism, and compared with CPU, FPGA can perform more processing tasks in each clock cycle, providing fast response time to meet application requirements ; In addition, it also has the advantages of dynamic reconfigurability, good stability, low power consumption, etc., so it is very suitable as an acceleration device in heterogeneous computing. Using FPGA as a hardware acceleration device, taking advantage of its hardware parallelism to perform heterogeneous accelerated computing, can greatly improve the computing power of a single node in the big data processing framework, reduce the power consumption of its data center, and at the same time reduce the speed of computing clusters. Network transmission overhead caused by expansion. Therefore, by using the Opencl high-level comprehensive language to perform parallel optimization on the FPGA for the calculation algorithm of the group query aggregation function, and using the FPGA for calculation in each computing node, the group query aggregation function for the entire data file to be queried can be effectively improved. Computing speed.

Please refer to FIG. 3. FIG. 3 is a flow chart of the calculation of the SQL group query aggregation function performed by the FPGA on the data block in FIG. 2, which may include the following steps:

Step 301: Load the data in the data column in the data vector.

After the FPGA acquires the data block to obtain the data column, a data vector can be defined for each data column, and then the data in the data column is loaded into the data vector. The number of data bits of the data vector of each data column is consistent with the number of data bits of the data column.

Step 302: Determine whether the data in the data vector satisfies the query condition of the aggregation function in the SQL group query; if so, go to step 303.

Judging each data in each data vector: judging whether the bit data satisfies the query condition of the aggregation function in the SQL group query; if so, enter step 303 to complete the corresponding follow-up operation. Of course, the judgment of the FPGA for each data in each data vector is done in parallel, which can effectively improve the data processing speed.

Step 303: Execute the calculation operation of the SQL group query aggregation function, and rewrite the corresponding bit data in the result vector from the first flag bit to the second flag bit.

As mentioned above, the calculation of the aggregation function of the SQL group query can specifically be various contents, and can be realized through the corresponding aggregation function. According to the aggregation function in the SQL group query, the FPGA executes the calculation operation of the SQL group query aggregation function for the data judged to meet the query conditions in step 302 . At the same time, in order to distinguish and indicate whether the bit data meets the query condition, the corresponding bit data in the result vector can be rewritten from the first flag bit to the second flag bit.

Among them, the result vector, data vector and data column are in a one-to-one correspondence relationship, that is, the data in the result vector indicates whether the corresponding data in the corresponding data vector or data column meets the query condition of the aggregation function. In order to distinguish and express, all the data in the result vector can be initialized as the first flag by default; when it is judged that the data of the corresponding bit in the data vector meets the query conditions of the aggregation function, the data of the corresponding bit in the result vector can be rewritten as the second Flag bit. Specifically, the first flag bit can be set to 0, and the second flag bit can be set to 1; or the first flag bit can be set to 1, and the second flag bit can be set to 0, which is not limited in this application.

It should be noted that in this step, the calculation operation of the SQL group query aggregate function can be performed first, and the corresponding bit data in the result vector is rewritten from the first flag bit to the second flag bit, or the corresponding bit in the result vector can be first The data is rewritten from the first flag bit to the second flag bit, and then the calculation operation of the SQL group query aggregation function is executed. The embodiment of the present application does not limit the order of the two, and can be selected according to the specific situation and programming habits.

It is worth mentioning that, in the embodiment of the present application, when calculating the aggregation function of the SQL group query, it is preferable to process the data in the manner of unroll loop expansion, that is, the calculation algorithm of the aggregation function in the group query is preferably The code is written and implemented in the way of unroll loop expansion to further increase the workload of the FPGA in each clock cycle, thereby shortening the running time of the entire algorithm and further improving the calculation speed of the group query aggregation function.

Step 304: Combine the result vectors according to the query condition of the aggregation function to obtain the calculation result.

For the result vectors corresponding to the plurality of data columns obtained in step 303, the FPGA merges them according to the query condition of the aggregation function to obtain the calculation result of the data block of the computing node, so as to facilitate the completion of the subsequent steps.

Next, taking the case where there are two data columns to be queried as an example, the process of FPGA performing SQL group query calculation is introduced in detail. The flowchart is shown in Figure 4, including the following steps:

Step 401: Load the data of data column 1 in data vector 1; load the data of data column 2 in data vector 2.

Taking the case where there are two data columns to be queried as an example, the two data columns may be named data column 1 and data column 2 respectively, and the corresponding data vectors are named data vector 1 and data vector 2 respectively, And load the two data columns into their respective data vectors. It is easy to understand that data vector 1 has the same number of data bits as data column 1; data vector 2 has the same number of data bits as data column 2.

Step 402: Determine whether each bit of data in data vector 1 and data vector 2 satisfies the query condition of the aggregation function by unrolling; if so, go to step 403.

Judgment is performed on each bit of data in data vector 1 and data vector 2 in parallel, and if the query operation of the aggregation function is satisfied, then go to step 403 .

Moreover, in this embodiment, the judgment operation in step 402 and a series of operations performed on the judgment result in subsequent steps 403 and 404 are performed in an unroll loop, which can further increase the speed of data processing.

Step 403: Execute the calculation operation of the SQL group query aggregation function, and rewrite the corresponding bit data in the result vector 1 or the result vector 2 from 0 to 1.

It can also be understood that the result vector 1 has the same number of data bits as the data column 1; and the result vector 2 has the same number of data bits as the data column 2.

Here, the first flag bit of the data of the result vector described above is set to 0, and the second flag bit is set to 1. Of course, other setting methods can also be used, which will not be repeated here. If the data in the data vector 1 is judged to meet the aggregation function query condition by step 402, the data in the corresponding bit in the result vector 1 will be rewritten from 0 to 1; similarly, if the data in the data vector 2 is judged to meet the aggregation function When the query condition is specified, the data in the corresponding bit in the result vector 2 is rewritten from 0 to 1.

Step 404: Combine the result vector 1 and the result vector 2 according to the query condition of the aggregation function to obtain the calculation result.

By merging the result vector 1 and result vector 2 processed in step 403, the calculation result of all the data of the computing node can be obtained, so that the calculation result can be returned to complete the next steps.

The data processing system for the aggregation function in the group query provided by the embodiment of the present invention is introduced below. The data processing system of the aggregation function in the group query described below and the data processing method of the aggregation function in the group query described above can be referred to in correspondence.

Please refer to FIG. 5 . FIG. 5 is a structural block diagram of a data processing system for aggregation functions in group query provided by the present application; it includes a main CPU 501 , a computing node CPU 502 , and an FPGA 503 .

The main CPU 501 is connected to a plurality of computing node CPUs 502 through a communication network, and can control data transmission and program scheduling between each computing node through a host computer program. The computing node is specifically composed of a computing node CPU502 and an FPGA503, and the computing node CPU502 and the FPGA503 are connected through a PCIe high-speed interface. The Opencl language parallel optimization code of the calculation algorithm of the aggregation function in the group query is transplanted and deployed on each FPGA503. Each computing node has the ability to store and process data independently, and multiple computing nodes form a heterogeneous parallel accelerated computing system under the unified control and management of the main CPU 501 .

The main CPU 501 is used to divide the data file to be queried into data blocks, and distribute the data blocks to multiple computing nodes. Specifically, before dividing the data file to be queried into data blocks, the main CPU 501 is also used to parse the query statement in the program, and load the data file to be queried from the designated storage area according to the query statement. After the FPGA 503 returns the calculation results, the main CPU 501 merges the different calculation results of the FPGA 503 from different computing nodes, and obtains the complete final query result of the data file to be queried.

After the calculation node CPU502 receives the data block from the main CPU501 from the communication network, it transmits the data block to the FPGA503, so that the FPGA503 performs the calculation operation of the SQL grouping query aggregation function on the data block to obtain the calculation result, thereby transmitting the FPGA503 return to the main CPU501 calculation results.

FPGA503 is used as the hardware acceleration device in the data processing system of the aggregation function in the group query provided by this application. After obtaining the data block from the computing node CPU502, it is used to perform SQL grouping on the data block according to the calculation algorithm of the aggregation function in the group query Query the calculation of the aggregation function to obtain the calculation result of the data block, and return the calculation result to the main CPU 501 through the calculation node CPU 502 . As mentioned above, the calculation algorithm of the group query aggregation function is realized by parallel optimization of the Opencl high-level synthesis language, and is efficiently transplanted and deployed to each FPGA, so that the SQL group query aggregation function can be calculated on the data block by the FPGA. And, the calculation algorithm of the aggregate function in the group query is preferably implemented by writing code in an unroll loop expansion mode, so that when the FPGA performs the calculation of the SQL group query aggregate function, the workload of the FPGA is further increased in each clock cycle, In this way, the calculation speed of the aggregation function of the group query is improved.

Wherein, when the FPGA503 obtains the data block from the computing node CPU502, it is specifically used to apply for space in the local memory, and loads the data block transmitted by the computing node 502 from the DDR into the applied space to obtain a data column. And when FPGA503 carried out the calculation of SQL grouping query aggregation function, it first loaded the data in the data column in the data vector; Then judge whether the data in the data vector satisfies the query condition of the aggregation function in the SQL grouping query calculation; if, Execute the calculation operation of the SQL grouping query aggregation function, and rewrite the corresponding bit data of the result vector from the first flag bit to the second flag bit; finally combine the result vectors according to the query conditions of the aggregation function to obtain the calculation result.

It can be seen that the data processing system of the aggregation function in the group query provided by the present application, by setting FPGA503 in each computing node, and transplanting and deploying the Opencl language parallel optimization code of the aggregation function calculation algorithm in the group query on the FPGA503, constitutes A heterogeneous parallel accelerated computing system is established, so the hardware parallel feature of FPGA503 can be used to accelerate the processing process of a single computing node, and then realize the acceleration of the data processing process of the group query aggregation function for the entire file to be queried.

Each embodiment in the present application is described in a progressive manner, each embodiment focuses on the differences from other embodiments, and the same and similar parts of the various embodiments can be referred to each other. As for the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and for the related information, please refer to the description of the method part.

Professionals can further realize that the method steps described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, in In the above description, the components and steps of each example have been generally described in terms of functions. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present application.

The steps of the methods or algorithms described in connection with the embodiments disclosed herein may be directly implemented by hardware, software modules executed by a processor, or a combination of both. Software modules can be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other Any other known storage medium.

The data processing method and system of the aggregation function in the group query provided by this application have been introduced in detail above. In this paper, specific examples are used to illustrate the principles and implementation methods of the present application, and the descriptions of the above embodiments are only used to help understand the methods and core ideas of the present application. It should be pointed out that those skilled in the art can make several improvements and modifications to the application without departing from the principles of the application, and these improvements and modifications also fall within the protection scope of the claims of the application.

Claims (10)

1. the data processing method of aggregate function in a kind of Querying by group, it is characterised in that comprise the following steps：

Data file to be checked is divided into data block by host CPU, and the data block is assigned into multiple calculate nodes；

Each calculate node CPU is respectively by the transmission of data blocks to FPGA, and the FPGA and calculate node CPU passes through PCIe Interface correspondence is connected；

The FPGA obtains the data block, and the data block is carried out according to the computational algorithm of aggregate function in Querying by group The calculating of SQL Querying by group aggregate functions, and result of calculation is returned to the host CPU by the calculate node CPU；Described point The code of the computational algorithm of aggregate function is realized using Opencl language parallel optimization in group polling, and is transplanted and be deployed in and be described On FPGA；

The host CPU merges the result of calculation, obtains final Query Result.

2. the data processing method of aggregate function in Querying by group according to claim 1, it is characterised in that in the master Data file to be checked is divided into before data block by CPU, in addition to：

The host CPU parses query statement；

The data file to be checked is loaded according to the query statement.

3. the data processing method of aggregate function in Querying by group according to claim 1, it is characterised in that the FPGA Obtaining the data block includes：

The FPGA is in local memory application space；

The data block is loaded into the space from DDR and obtains data row.

4. the data processing method of aggregate function in Querying by group according to claim 3, it is characterised in that the FPGA The calculating bag of SQL Querying by group aggregate functions is carried out to the data block according to the computational algorithm of aggregate function in Querying by group Include：

The data in the data row are loaded in data vector；

Judge whether the data in the data vector meet the querying condition of aggregate function in SQL Querying by group；If so, performing The calculating operation of SQL Querying by group aggregate functions, and the data of corresponding positions in result vector are rewritten as the by the first flag bit Two flag bits；

The result vector is merged according to the querying condition of the aggregate function and obtains result of calculation.

5. the data processing method of aggregate function in Querying by group according to claim 4, it is characterised in that the FPGA The calculating bag of SQL Querying by group aggregate functions is carried out to the data block according to the computational algorithm of aggregate function in Querying by group Include：

In the way of unroll loop unrollings according to Querying by group in aggregate function computational algorithm to the data block carry out SQL The calculating of Querying by group aggregate function.

6. the data handling system of aggregate function in a kind of Querying by group, it is characterised in that including：

Host CPU：For data file to be checked to be divided into data block, and the data block is assigned to multiple calculate nodes； Receive and merge each FPGA and carry out the result of calculation that SQL Querying by group aggregate function is calculated, obtain final Query Result, its In, the FPGA is connected with the calculate node CPU by the way that PCIe interface is corresponding；

Calculate node CPU：Institute is transmitted for the obtained transmission of data blocks will to be distributed to the FPGA, and to the host CPU State the result of calculation of FPGA returns；

The FPGA：For obtaining the data block, according to the computational algorithm of aggregate function in Querying by group to the data block The calculating of SQL Querying by group aggregate functions is carried out, and described calculate is returned to the host CPU by the calculate node CPU tying Really；In the Querying by group code of the computational algorithm of aggregate function using Opencl language parallel optimization realize, and transplant and It is deployed on the FPGA.

7. according to claim 6 in Querying by group aggregate function data handling system, it is characterised in that the host CPU It is additionally operable to：

Parse query statement；The data file to be checked is loaded according to the query statement.

8. according to claim 6 in Querying by group aggregate function data handling system, it is characterised in that FPGA tool Body is used for：

In local memory application space；And from DDR by the data block be loaded into the space obtain data row.

9. according to claim 8 in Querying by group aggregate function data handling system, it is characterised in that FPGA tool Body is used for：

The data in the data row are loaded in data vector；Judge whether the data in the data vector meet SQL points The querying condition of aggregate function in group polling；If so, perform SQL Querying by group aggregate functions calculating operation, and by result to The data of amount corresponding positions are rewritten as the second flag bit by the first flag bit；According to merging the querying condition of the aggregate function Result vector obtains result of calculation.

10. according to claim 9 in Querying by group aggregate function data handling system, it is characterised in that the FPGA Specifically for：

In the way of unroll loop unrollings according to Querying by group in aggregate function computational algorithm to the data block carry out SQL The calculating of Querying by group aggregate function.