CN104361113B - A kind of OLAP query optimization method under internal memory flash memory mixing memory module - Google Patents

Abstract

The present invention relates to an OLAP query optimization method under a memory-flash memory hybrid storage mode, which comprises: OLAP storage adopts a flash-aware storage model, and accesses parts according to data between a relatively small DRAM and a relatively large flash storage The property is divided, and the storage is optimized on the heterogeneous two-level memory; the memory OLAP adopts array storage, and each attribute column is stored in a continuous array unit, and the traditional connection operation is simplified as an array subscript access, and the AIR algorithm is implemented. OLAP query processing; where AIR is array subscript access; decompose OLAP query processing based on array storage and AIR algorithm into three sequential data access processes; access flash storage metric columns to specify metric values by selecting vectors; use keyword-based On the basis of the bitmap connection index, K keyword connection bitmaps are optimally stored in the DRAM-flash two-level storage to form a two-level connection bitmap index structure. The invention can improve the cost performance of memory storage, memory and CPU usage efficiency and data storage efficiency, and can be widely used in general OLAP application scenarios.

Description

An OLAP query optimization method in memory-flash hybrid storage mode

technical field

The invention relates to a storage optimization and OLAP (Online Analytical Processing) query optimization method in the field of databases, in particular to a two-level DRAM (Dynamic Random Access Memory) and Flash (flash memory) suitable for the memory database integrated machine platform OLAP query optimization method under storage-based memory-flash hybrid storage mode.

Background technique

In-memory analysis processing (in-memory OLAP) is an important technology for real-time analysis and processing of big data. With the support of large memory and parallel processing capabilities of multi-core processors, in-memory OLAP has excellent real-time analysis and processing capabilities, but compared to other storage devices, such as flash , disk, etc., memory is still a very expensive storage medium, and its storage energy consumption is an order of magnitude higher than that of flash (DRAM: ~100mW/GB, NAND flash: 1-10mW/GB), memory OLAP analysis and processing need to use big data As a basis, the hardware cost of in-memory analytical processing is high. PCIe Flash Card, as a large-capacity (hundreds of GB to TB-level storage capacity) high-speed storage technology, has been widely used in the field of high-performance databases. Provide Smart Flash Cache to cache hot data, optimize the cache algorithm through database logic, and specify cache optimization strategies based on tables. On the one hand, the application of high-speed flash memory cards provides cheap secondary storage expansion capabilities for expensive memory storage, but on the other hand, high-speed flash memory cards are mainly used as extended caches for databases on flash, expanding the capacity of memory caches (buffers), However, it has not been combined with in-memory OLAP storage optimization and query processing optimization technology, and has not implemented flash-aware OLAP optimization technology at the OLAP algorithm level.

The current analytical in-memory database mainly uses DRAM as the main storage device, and flash is used as a backup storage or disk cache to replace the disk. Flash has not yet been incorporated into the OLAP algorithm design of the memory. How to apply the secondary storage mode formed by memory and large-capacity flash to the field of memory analysis and processing, making it a high-performance, cost-effective mainstream platform, and the memory database must not only support the analysis and processing of the full memory model, but also need to support DRAM - The memory analysis and processing under the flash two-level storage has become an urgent technical problem to be solved.

Contents of the invention

In view of the above problems, the object of the present invention is to provide an OLAP query optimization method under a memory-flash memory hybrid storage mode. The method is based on DRAM-flash two-level storage, and the cost performance of memory storage is improved through large-capacity and cheap flash. At the same time, the method can effectively improve data storage efficiency. Further, the method provided by the present invention can effectively improve memory and CPU usage efficiency.

In order to achieve the above object, the present invention adopts the following technical solutions: a method for optimizing OLAP queries under a memory-flash memory hybrid storage mode, which includes the following steps: 1) OLAP storage adopts a flash-aware storage model, that is, according to OLAP star or In the snowflake model, the dimension table is small and there are many predicate operations, and the fact table is composed of foreign keys and measurement attributes. It is divided according to the locality of data access between relatively small DRAM and relatively large flash storage. , storage optimization is performed on heterogeneous two-level memory; 2) In-memory OLAP uses array storage, and each attribute column is stored in a continuous array unit. The table is composed of attribute arrays of the same length, and the dimension table uses array subscripts as The primary key, the foreign key of the fact table is the data subscript of the corresponding record in the dimension table, and the fact table record can directly locate the corresponding array unit in the dimension table according to the foreign key value, simplifying the traditional connection operation to array subscript access, and performing the AIR algorithm OLAP query processing; where AIR is array subscript access; 3) OLAP query processing based on array storage and AIR algorithm is decomposed into three sequential data access processes: dimension table access, fact table foreign key access and fact table measurement attributes Access, the intermediate data structures generated in three stages include: dimension table filtering grouping vector, selection vector and grouping vector, grouping multidimensional array; dimension table filtering grouping vector, selection vector and grouping vector are common data structures for each query, different queries It only needs to update the content of the vector, and the grouped multidimensional array is dynamically generated according to different queries; 4) Based on the keyword-based bitmap connection index, K keyword connection bitmaps are optimally stored in the DRAM-flash two-level storage , select n high-frequency access relation words and store the corresponding bitmaps in DRAM, and store the remaining bitmaps in flash to form a two-level connection bitmap index structure.

Described step 1) in, adopt memory storage engine, dimension table resides in DRAM; The foreign key in the fact table is the row that access frequency is higher when star connection in the multidimensional analysis, resides in DRAM equally; Fact table's The metric attributes are stored in flash, and the access API supports random access of metric columns by position on the metric columns.

In the step 3), the dimension table filtering grouping vector, selection vector and grouping vector, grouping multidimensional array are stored in DRAM; the fact table measurement attributes are stored in flash.

In the described step 2), the OLAP query processing of the AIR algorithm comprises the following steps: 1. the OLAP query is decomposed into grouping and filtering operations on the dimension table: the selection and grouping operations in the query are divided by the dimension table, and the query is decomposed into It is a subquery on each dimension table; ② Generate a dimension table filter grouping vector: each dimension table filters records according to their respective where clauses and projects the grouping attributes that meet the selection conditions, and the grouping attributes that meet the selection conditions are converted into a dictionary Table compression, the dictionary table is stored in the array, the dictionary compression code is the subscript of the dictionary array, in the filter group attribute, the position where the selection condition is false is set to -1, otherwise it is set to the group compression code, and recorded in the same length as the dimension table ③ Create selection and grouping vectors by multiple scans of foreign keys in the fact table: scan the foreign key columns corresponding to the fact table in order of selection rate from low to high according to the selection rate of dimension table filtering grouping vectors, and scan each column by foreign key The key value is mapped to the position specified by the filter grouping vector of the corresponding dimension table. When the vector value is non-negative, insert the record position of the current fact table into the selection vector; when scanning the next foreign key column, the random access method is adopted according to the record position in the current selection vector, and Update the selection vector by filtering the grouping vector of the corresponding dimension table, and delete the record positions of the fact table that do not meet the subsequent foreign key mapping conditions; When the total selection rate is high, update the grouping vector while updating the selection vector, and use the dimension table to filter the subscript of the current grouping in the grouping vector to incrementally calculate the grouping multidimensional array subscript; when the total selection rate is very low, first generate the selection vector, and finally randomly access each foreign key column according to the position of the selected vector, and generate a grouping vector at one time; ④Access the flash storage metric column to specify the metric value by selecting the vector: access the metric attribute value on the flash storage according to the position in the selected vector; The random access of the selection vector on the flash adopts multi-core parallel execution; ⑤Aggregate the measurement values in the multi-dimensional array through the grouping vector: according to the subscript of the grouping recorded by the grouping vector, the value of the measurement attribute returned from the flash storage is "pushed" Perform aggregation calculation in the grouping multidimensional array unit indicated by the subscript value in the grouping vector, and complete all OLAP query processing; ⑥ Reversely map each dimension of the grouping multidimensional array to the grouping dictionary table to obtain the grouping attribute value and output the query result.

In the step 2., the filter grouping vector is an additional column of a dynamically generated dimension table, which replaces the dimension table and the fact table to perform a connection operation and provides the encoding of the grouping attribute of the connection record on the current dimension; when there is no grouping attribute on the dimension table When , the filter grouping vector is simplified to a bitmap, which is used for the connection filtering of the foreign key of the fact table.

In described step 4), adopt DRAM-flash two-level bitmap connection index method on the basis of keyword bitmap connection index, select the most frequently used in K keyword connection bitmaps accurately according to memory storage space quota N are connected indexes as DRAM resident bitmaps, and the remaining K-n keyword bitmaps are stored in flash as secondary indexes.

Because the present invention adopts the above technical scheme, it has the following advantages: 1. The present invention adopts DRAM-flash two-level storage as the storage platform of memory OLAP. Compared with the full memory storage model, DRAM-flash two-level storage greatly reduces the demand for expensive memory and reduces the overall cost of hardware. Through the optimization of the flash-aware storage model and the AIROLAP algorithm, a large number of measurement attributes stored on the flash High-efficiency random access is used to narrow the performance gap of flash storage. 2. The AIR OLAP query processing algorithm adopted in the present invention divides a complete OLAP query processing process into three independent processing stages, and only involves a small proportion of data in the dimension table and fact table foreign key processing stage, each dimension table Generate a fixed-length filtering grouping vector; no matter how many columns are involved in the query on the fact table, only one selection vector and one grouping vector are needed. The initial length of the selection and grouping vectors is determined by the lowest selection rate on the dimension, and outside the fact table The length of the star connection of the key is continuously shortened, and the memory storage overhead is limited; the metric attribute with the largest storage space in the database is stored in the large-capacity flash, and the traditional row access (including row storage and row access on the column storage) OLAP query processing completes pipelined hash join and hash group aggregation operations during the full table scan of the fact table. The AIR OLAP query algorithm decomposes the join, group operation, and It is to complete the connection grouping operation on a limited number of foreign keys of the fact table, and then access the specified position of the specified measurement column according to the position according to the selection vector with a very low selectivity rate, and push the data access operation on the flash storage to the final processing stage, which is extremely large. The data access load on the flash is reduced, and the good random access performance of the flash can also be brought into play. 3. The bitmap connection index adopted by the present invention is a kind of incremental index, and the index update operation generated because of the fact table record increase is the sequential growth of the bitmap, instead of updating the node content and structure like the B+-tree index, It can eliminate the data rewriting cost of data update on flash, so it is more suitable for flash storage. The DRAM-flash two-level memory bitmap connection index method adopts a two-level storage mode for the keyword bitmap by querying the keyword access frequency and memory storage space based on the keyword bitmap connection index. In the figure, n frequently used keyword bitmaps are stored in DRAM, and the remaining K-n keyword bitmaps are stored in flash, which reduces the storage overhead of the memory index. 4. The present invention establishes a heterogeneous storage model through the characteristics of OLAP patterns and loads and the locality characteristics of data in the OLAP query processing algorithm (that is, the degree of frequent access to data), and uses different data locality strengths in tables, indexes, and tables. The column group and the intermediate data structure that the query depends on are used as objects. According to the memory capacity and data access performance constraints, they are optimized and distributed in high-performance but relatively small-capacity DRAM and large-capacity but relatively low-performance flash storage to improve data quality. storage efficiency. 5. According to the characteristics of DRAM and flash storage during query processing, the present invention delays the execution of data access on the flash, divides the complete query processing process of OLAP into two stages of memory processing and flash calculation, and supports different OLAP queries. Pipeline parallelism in the storage access stage improves memory and CPU usage efficiency. The present invention can be widely used in general OLAP application scenarios.

Description of drawings

Fig. 1 is the storage schematic diagram of memory OLAP of the present invention on DRAM-flash two-level storage;

Fig. 2 is a schematic diagram of query processing of memory OLAP on DRAM-flash two-level storage in an embodiment of the present invention;

FIG. 3 is a schematic diagram of key bitmap connection index processing on the DRAM-flash two-level storage of the present invention.

detailed description

Existing in-memory OLAP technologies usually adopt a full-memory computing mode, or use large-capacity flash as a cache. The former increases the cost of in-memory computing, and the latter makes it difficult to extend OLAP optimization from memory to flash. For this reason, the present invention proposes an OLAP query optimization method based on DRAM-flash two-level storage memory-flash memory hybrid storage mode, and optimizes the storage of OLAP data on memory and flash according to the characteristics of OLAP mode, load and OLAP algorithm , to optimize the OLAP query algorithm for the characteristics of two-level storage. The present invention is applicable to general OLAP application scenarios. The present invention will be described in detail below in conjunction with the accompanying drawings and embodiments.

As shown in Figure 1, the present invention provides an OLAP query optimization method under a memory-flash hybrid storage mode, which is based on DRAM-flash two-level storage, and is oriented to DRAM-flash two-level storage composed of DRAM and large-capacity PCIe Flash Card A memory OLAP query optimization method on storage, which includes the following steps:

1) OLAP storage adopts the flash-aware storage model, that is, according to the characteristics of small dimension tables and many predicate operations in OLAP star or snowflake models and the characteristics of fact tables composed of foreign keys and measurement attributes, relatively small The DRAM and relatively large flash storage are divided according to the locality of data access, and storage optimization is performed on the heterogeneous two-level memory to improve the efficiency of high-frequency use of data in the memory.

Since dimension tables are small and modern OLAP supports updates on dimension tables, an in-memory storage engine is used, and dimension tables reside in DRAM. The foreign key in the fact table is a column with high access frequency in the star connection in multidimensional analysis, and it also resides in DRAM. Fact tables have many metric attributes, and OLAP queries are usually only for a few metric attributes and metric values with a very low selection rate. Therefore, they are stored in cheap and large-capacity flash, and an API is provided to support metric column access by location on the metric column. Location random access.

2) In-memory OLAP adopts array storage, and each attribute column is stored in a continuous array unit. The table is composed of attribute arrays of the same length. Preferably, the dimension table uses the array subscript as the primary key, and the foreign key of the fact table is dimension The data subscript of the corresponding record in the table, and the fact table record can directly locate the corresponding array unit in the dimension table according to the foreign key value, simplifying the traditional connection operation to the array subscript access ( _Array Index Reference, AIR), and performing the AIR algorithm OLAP query processing.

3) Decompose the OLAP query processing based on array storage and AIR algorithm into three sequential data access processes: dimension table access, fact table foreign key access, and fact table measurement attribute access. The intermediate data structures generated in the three stages include: dimension table access Table filtering grouping vectors, selecting vectors and grouping vectors, grouping multidimensional arrays. Dimension table filtering grouping vector, selection vector and grouping vector are data structures shared by all queries. Different queries only need to update the content of the vector; grouping multidimensional arrays are dynamically generated according to different queries. These three types of intermediate data structures are reused in the query processing process and belong to strong locality data sets, which need to be stored in DRAM; the fact table measurement attributes occupy a large storage space, but usually only a small number of measurement attributes are accessed in the query , and only a very low proportion of records in the metric attribute column are randomly accessed by selecting the vector, so the metric attribute column can be stored in flash to reduce the demand for DRAM in memory OLAP and reduce the hardware cost of the system.

The AIR OLAP algorithm adopted in the present invention needs to select data such as vectors, grouping vectors, dimension table filtering grouping vectors, and grouping multidimensional arrays for the OLAP query processing process. These data structures can be reused between OLAP queries, and the occupied memory space is fixed. Therefore, it is resident in DRAM.

4) Data warehouses usually use bitmap join indexes to optimize or eliminate join costs between dimension tables and fact tables. The present invention adopts on the basis of keyword-based bitmap connection index (that is, set up bitmap connection index by keyword instead of whole attribute members) in DRAM-flash two-level storage to optimize and store K keyword connection bitmaps ( That is, each keyword corresponds to a bitmap with the same length as the fact table, record the fact table record position corresponding to the keyword), select n high-frequency access relation words and store the corresponding bitmap in DRAM, and the rest The bitmap is stored in flash to form a secondary link bitmap index structure;

Among them, the present invention adopts the DRAM-flash two-level bitmap connection index method on the basis of the keyword bitmap connection index, and accurately selects the most frequently used n among the K keyword connection bitmaps according to the memory storage space quota as The DRAM resides in the bitmap connection index, and the remaining K-n keyword bitmaps are used as secondary indexes. Specifically: the index item is the bitmap of the fact table location corresponding to the attribute value of a certain dimension table. The size of the bitmap index is the product of the fixed-length bitmap size and the number of keywords. The bitmap index can further reduce the memory through data compression. storage. By querying execution logs and query keyword analysis, the K most frequently used dimension attribute keywords can be determined and a bitmap index can be established for them. According to the memory space quota, the n most frequently used keyword bitmaps can reside in memory, and the remaining K-n keyword bitmaps are stored in flash to form a secondary connection bitmap index structure.

In the above step 2), the OLAP query processing of the AIR algorithm comprises the following steps:

① Decompose the OLAP query into grouping and filtering operations on dimension tables: divide the selection and grouping operations in the query into dimension tables, and decompose the query into subqueries on each dimension table.

②Generate dimension table filter grouping vector: each dimension table filters records according to their respective where clauses and projects the grouping attributes that meet the selection conditions. The grouping attributes that meet the selection conditions are compressed in the dictionary table, and the dictionary table is stored in the array. The dictionary compression code is the subscript of the dictionary array. In the filter group attribute, the position where the selection condition is false is set to -1, otherwise it is set to the group compression code and recorded in the filter group vector with the same length as the dimension table.

The filter grouping vector is a dynamically generated additional column of the dimension table, which replaces the dimension table and the fact table for the connection operation and provides the encoding of the grouping attribute of the connection record on the current dimension. When there is no grouping attribute on the dimension table, the filter grouping vector is simplified to a bitmap, which is used for connection filtering of the foreign key of the fact table.

③Multiple scans of foreign keys in fact table to create selection and grouping vectors: filter grouping vectors according to dimension tables, scan the foreign key columns corresponding to fact tables in order of selection rate from low to high, and map each column to the corresponding dimension according to the foreign key value when scanning The position specified by the table filtering grouping vector, when the vector value is non-negative, insert the current fact table record position into the selection vector; when scanning the next foreign key column, use random access method according to the record position in the current selection vector, and filter the grouping through the corresponding dimension table The vector updates the selection vector, and deletes the record positions of the fact table that do not meet the subsequent foreign key mapping conditions; after the scanning of each foreign key column is completed, the selection vector records the record positions in the fact table that meet all the connection conditions. When the total selection rate is high, update the grouping vector while updating the selection vector, and use the dimension table to filter the subscript of the current grouping in the grouping vector to incrementally calculate the grouping multidimensional array subscript; when the total selection rate is low, first generate Select the vector, and finally randomly access each foreign key column according to the position of the selected vector, and generate a grouping vector at one time.

④ Specify the metric value by selecting the vector to access the metric column of the flash storage: access the metric attribute value on the flash storage according to the position in the selection vector, and only need to return a very small number of metric values for grouping and aggregation calculation; flash has good parallel random access performance, The random access on the flash according to the selection vector adopts multi-core parallel execution to reduce the flash data access delay.

⑤Aggregate the metric values in the grouped multidimensional array through the grouping vector: according to the grouping subscript recorded in the grouping vector, "push" the metric attribute value returned from the flash storage to the grouping multidimensional array unit indicated by the subscript value in the grouping vector Aggregation calculations are performed in the database to complete all OLAP query processing.

⑥ Reversely map each dimension of the grouped multidimensional array to the grouping dictionary table to obtain the grouping attribute value and output the query result.

Example:

As shown in Figure 2, flash-aware is reflected in the optimization of the OLAP query algorithm to push the access to the metric attributes stored on the flash to the end, and convert the sequential scan of the fact table to perform low selection on the metric attributes of the fact table according to the selection vector High-rate random access, reducing the latency of metric attribute access on flash.

Step 2) Take the following query command as an example:

SELECT c_nation, s_nation, sum(l_revenue), sum(l_price)

FROM customer, line order, supplier

WHERE lo_custkey=c_custkey

and lo_suppkey=s_suppkey

and c_region='AMERICA'

and s_region='ASIA'

group by c_nation, s_nation;

The query command needs to connect the fact table lineorder with the dimension tables customer and supplier, and then calculate the cumulative sum of the fact table measurement columns l_revenue and l_price according to the c_region and s_region attributes of the dimension table.

As shown in Figure 3, the keyword bitmap connection index processing embodiment on the DRAM-flash two-level storage in step 4) of the present invention:

In a traditional database, an index is a flat structure that is assumed to be used on the same storage level. The index technology of disk database such as B+-tree index realizes the data exchange between memory missing nodes on disk and memory through the buffer mechanism, but it is still an opaque mechanism, and its index access efficiency depends on the efficiency of the buffer replacement algorithm, which cannot be Database customization optimization. In OLAP applications, index items exist in dimension table attributes, but the index objects are the corresponding connection records in the fact table. Ordinary B+-tree indexes can only retrieve data on a single table. For OLAP applications, it is relatively small. The improvement of the index access performance on the dimension table has a limited effect on the acceleration of the overall performance of OLAP queries, and the establishment of an index on the foreign key of the fact table can speed up the connection operation between the fact table and the dimension table, but the index storage on the foreign key of the fact table The overhead is extremely large, and the fact table in modern OLAP applications needs to be updated in real time, and the update cost of the index is huge. The present invention adopts a bitmap connection index method, that is, establishes a fact table connection bitmap for specified attribute members on the dimension table through connection operations, and establishes a bitmap for each member to indicate the connection position of the member on the fact table record. The bitmap join index is a kind of incremental index. When a large amount of data is inserted into the fact table, it only needs to expand the length of the bitmap incrementally, and there is no need to reconstruct the established join bitmap. The bitmap join index used in the traditional database is to establish a join bitmap for all members at the granularity of attributes. Faced with low potential set attribute bitmap join index, the space cost is small but the selection rate is too high, while the high potential set attribute has a low selectivity rate but The number of connection bitmaps is large, and the storage overhead is too large. The present invention selects the bitmap connection index keywords according to the TOP K access frequency of the dimension attribute keywords in the query load, and establishes a global bitmap connection index for K global high-frequency access keywords to form a key/value index structure , key is the global name of the keyword, including table and keyword information, and value is the connection bitmap. Keywords of different dimension tables and different attributes have the same connection bitmap length, which can be stored in the global bitmap connection index. Wherein preferably, the present invention proposes a DRAM-flash two-level bitmap connection index method on the basis of the keyword bitmap connection index, and accurately selects the most frequently used keywords in the K keyword connection bitmaps according to the memory storage space quota The n of Kn are used as DRAM resident bitmap connection indexes, and the remaining Kn keyword bitmaps are used as secondary indexes. As shown in Figure 3, it shows the application scenario where the query keyword bitmap exists in DRAM and flash respectively. First, the bitmap in DRAM completes the selection operation to generate a filter bitmap. When the selection rate of the filter bitmap is at a certain When between the threshold ranges (S _low , S _high ), perform random access to the specified position of the corresponding bitmap in the flash storage according to the position of "1" in the generated filter bitmap, and determine the final logical result of the position. Flash has better random access ability, and the present invention adopts to filtering bitmap sequential access, to the strategy of flash bitmap parallel access, improves the parallel access performance of data on the flash, reduces the delay of bitmap index calculation, as shown in Figure 3 Parallel flash access threads shown. Assuming that the selectivity rate of bitmap filtering in DRAM is S ₁ , and the selectivity rate of bitmap in flash is S ₂ (the selectivity rate of bitmap can be precisely given by the number of "1"), T _F (S ₁ ) is in Bitmap access delay on flash, T _P (S ₂ ,S ₁ ) is the query processing time difference between index selectivity S ₂ and S ₁ , when T _P (S ₂ ,S ₁ )-T _F (S ₁ )>0, bitmap access on flash has query performance benefits. S _low and S _high are the difference between the lowest and highest selectivity ratios (such as S ₂ -S ₁ ) that satisfy the query benefit.

In summary, compared with the prior art, the present invention adopts a memory OLAP query optimization method based on DRAM-flash two-level storage, supports memory OLAP query processing on DRAM-flash two-level storage, and reduces the impact of memory OLAP on expensive memory. demand and improve the cost performance of in-memory OLAP. Through the storage optimization method based on DRAM-flash two-level storage, OLAP query optimization method and index optimization method, the data access and query processing performance in the OLAP query processing process are transparently optimized. The invention simultaneously improves the access data management capability and query processing performance of the memory OLAP big data.

The above-mentioned embodiments are only used to illustrate the present invention, wherein the structure, connection mode and manufacturing process of each component can be changed to some extent, and any equivalent transformation and improvement carried out on the basis of the technical solution of the present invention should not excluded from the protection scope of the present invention.

Claims (8)

1. a kind of internal memory-flash memory mixes the OLAP query optimization method under memory module, and it is comprised the following steps：

1) OLAP storage using flash-aware storage model, i.e., according to dimension table in OLAP stars or snowflake shape model it is smaller, The characteristics of the characteristics of predicate operation is more and true table are made up of external key and metric attribute, in relatively small DRAM and relatively Divided by the locality of data access between big flash storages, storage optimization is carried out on the two-stage internal memory of isomery；

2) memory OLAP uses storage of array, and each attribute column is stored in continuous array location, and table is by each of equal length Attribute array is constituted, and dimension table uses array index as major key, and true off-balancesheet key is the data subscript of respective record in dimension table, thing The record of real token can directly position corresponding array location in dimension table according to foreign key value, and traditional attended operation is reduced into array Subscript is accessed, and carries out the OLAP query treatment of AIR algorithms；Wherein AIR algorithms are array index access；

3) the OLAP query treatment based on storage of array and AIR algorithms is decomposed into three data access processes of order：Dimension table Access, true off-balancesheet key is accessed and true metric table attribute access, the intermediate data structure that three phases are produced includes：Dimension table mistake Filter packet vector, selection vector sum packet vector, packet Multidimensional numerical；Dimension table filter packets vector, selection vector sum be grouped to It is the shared data structure of each inquiry to measure, and different inquiries only needs to the content of renewal vector, is grouped Multidimensional numerical according to looking into Inquiry different and dynamic is generated；

4) using optimization storage K in DRAM-flash two-level memories on the basis of the bit join index based on keyword Keyword connects bitmap, therefrom selects n high frequency access relation word and corresponding bitmap is stored in into DRAM, and remaining bitmap is deposited Flash is stored in, two grades of connection bitmap index structures are formed.

2. a kind of internal memory-flash memory as claimed in claim 1 mixes the OLAP query optimization method under memory module, and its feature exists In：The step 1) in, using memory storage engine, dimension table is resident DRAM；External key in true table is star in multidimensional analysis Visiting frequency row higher during connection, in equally residing in DRAM；The metric attribute of true table is stored in flash, and is provided Opsition dependent accesses the random access that API supports measure column on measure column.

3. a kind of internal memory-flash memory as claimed in claim 1 mixes the OLAP query optimization method under memory module, and its feature exists In：The step 3) in, dimension table filter packets vector, selection vector sum packet vector, packet Multidimensional numerical are stored in DRAM； True metric table attribute is stored in flash.

4. a kind of internal memory-flash memory as claimed in claim 2 mixes the OLAP query optimization method under memory module, and its feature exists In：The step 3) in, dimension table filter packets vector, selection vector sum packet vector, packet Multidimensional numerical are stored in DRAM； True metric table attribute is stored in flash.

5. a kind of internal memory as claimed in claim 1 or 2 or 3 or 4-flash memory mixes the OLAP query optimization side under memory module Method, it is characterised in that：The step 2) in, the OLAP query treatment of the AIR algorithms is comprised the following steps：

1. OLAP query is decomposed into the packet filtering operation on dimension table：Selection in inquiry and division operation are carried out by dimension table Divide, be the subquery on each dimension table by query decomposition；

2. generation dimension table filter packets are vectorial：Each dimension table is filtered and is projected out according to respective where clause to record Meet the packet attributes of alternative condition, the packet attributes for meeting alternative condition carry out dictionary table compression, and dictionary table is stored in array In, dictionary compression code is dictionary array index, and alternative condition is set to -1 for the position of false in filter packets attribute, otherwise Packed compressed coding is set to, and is recorded in the filter packets vector isometric with dimension table；

3. true many times scanning of off-balancesheet key creates selection and packet vector：By dimension table filter packets vector selection rate from low to high Order scans the corresponding foreign key column of true table successively, is mapped to corresponding dimension table filter packets vector by foreign key value during per column scan and refers to Fixed position, will the current fact table record position insertion selection vector during vector value non-negative；During next external key column scan according to Record position in current selection vector uses random access manner, and by corresponding dimension table filter packets vector update selection to Amount, deletes the fact that be unsatisfactory for follow-up external key mapping condition table record position；After each foreign key column end of scan, selection vector note The record position of the whole condition of contacts of satisfaction in true table is recorded；When total selection rate is higher, while selection vector is updated Packet vector is updated, packet Multidimensional numerical subscript is incrementally calculated with the subscript of current group in dimension table filter packets vector；When When always selection rate is very low, selection vector is firstly generated, it is disposably raw finally by each foreign key column of position random access of selection vector Into packet vector；

4. measure column specified metric value is stored by selecting vector to access flash：Position in selection vector accesses flash Metric attribute value in storage；Performed using multi-core parallel concurrent according to random access of the selection vector on flash；

5. metric is assembled in Multidimensional numerical is grouped by being grouped vector：According to the packet subscript that packet vector is recorded, To be stored from flash return metric attribute value " pushing away " to packet vector in subscript value indicate packet Multidimensional numerical unit in Aggregation computation is carried out, whole OLAP query treatment are completed；

6. each dimension back mapping of Multidimensional numerical will be grouped to acquisition packet attributes value in packet dictionary table and Query Result will be exported.

6. a kind of internal memory-flash memory as claimed in claim 5 mixes the OLAP query optimization method under memory module, and its feature exists In：The step 2. in, the filter packets vector be one dynamic generation dimension table additional column, instead of dimension table with the fact table enter Row attended operation simultaneously provides the coding of linkage record packet attributes on leading dimension；When there is no packet attributes on dimension table, filtering Packet vector is reduced to a bitmap, for the connection filtering of true off-balancesheet key.

7. the OLAP query optimization under a kind of internal memory as described in claim 1 or 2 or 3 or 4 or 6-flash memory mixing memory module Method, it is characterised in that：The step 4) in, the levels of DRAM-flash two are used on the basis of keyword bit join index Figure join index method, accurately selects most frequently used according to memory storage space quota in K keyword connection bitmap N be resident bit join index as DRAM, remaining K-n keyword bitmap is as secondary index storage in flash.

8. a kind of internal memory-flash memory as claimed in claim 5 mixes the OLAP query optimization method under memory module, and its feature exists In：The step 4) in, DRAM-flash two-stages bit join index side is used on the basis of keyword bit join index Method, accurately most frequently used n is selected as DRAM according to memory storage space quota in K keyword connection bitmap Resident bit join index, remaining K-n keyword bitmap is stored in flash as secondary index.