CN112241354B - Application-oriented transaction load generation system and transaction load generation method - Google Patents

Abstract

The invention proposes an application-oriented synthetic load generation system, which can capture the load characteristics of real online transaction processing applications, and then generate a synthetic load that is highly similar to the actual application load performance index, while ensuring information concealment, tool scalability and load. Scalability. The invention proposes a new method for describing the load characteristics of online transaction processing applications: depicting the OLTP load characteristics from two dimensions of transaction logic and data access distribution; a method for extracting transaction logic and data access distribution from real application load traces, while ensuring The concealment of application information; from the perspective of controlling transaction conflicts and the proportion of distributed transactions, it proposes a method to ensure implicit transaction logic by controlling the dependencies of operating parameters; design and implement three angles to describe the distribution of data access: inclination, Dynamic and continuous; the first application-oriented OLTP load generator is implemented to ensure the authenticity and scalability of the generated load in performance evaluation.

Description

An application-oriented transaction load generation system and transaction load generation method

technical field

The invention relates to the technical field of transaction load generation, in particular to an application-oriented transaction load generation system and a transaction load generation method.

Background technique

There are many benchmarks for database performance evaluation in different application areas. In the OLAP application domain, commonly used benchmarks are TPC-H, TPC-DS and SSB [14], which have predefined standard database schemas and test queries. There are also TPC-C [1], TPC-E and SmallBank [10] benchmarks for evaluating the transaction processing capabilities of database systems. CH-benCHmark [15] and HTAPBench [16] can be transactions. Analytical Hybrid Processing (HTAP) systems provide unified assessments. Furthermore, YCSB [11] is often used to measure the throughput of cloud service systems with simple loads but high scalability. However, the evaluation workloads of these standard benchmarks are abstractions of a class of applications, so they are too general to evaluate database performance for a specific application.

In order to obtain a similar load to the target application, load trajectory replay is an optional method. Microsoft SQL Server is equipped with two tools, SQL Server Analyzer [17] and SQL Server Distributed Replay [2], for reproducing production workloads based on SQL traces. Oracle Database Replay [3], [18] enables users to record workload traces on a production system with minimal performance impact, and then replay a full workload with the same concurrency and load characteristics as the actual workload. Due to data privacy concerns, load replay is difficult to apply in practical database performance evaluation, as it requires a realistic database state and original load trajectory [3]. In addition, load expansion (such as expanding concurrency) is also a problem that current replay technology is difficult to solve.

Therefore, load simulation is very necessary and urgent. Load-aware data and query generators [4], [5], [19], [20] are used to evaluate database performance for OLAP applications. Inputs to these jobs typically include database schemas, basic data characteristics, and size specifications for intermediate results in query trees. The output is a synthetic database instance and instantiated test queries that match the specified data and load characteristics. For OLAP applications, there is database scaling work [6], [7], which scales up/down a given database instance, supporting application-specific database benchmarks. Load profilers [8], [9] are designed to study and better understand application load, but neither can generate synthetic load. There are also load generators [12], [13] prepared for database performance benchmarking. [12] proposed a load generator that can simulate real hardware resource consumption states. NoWog [13] introduced a workload description language for generating synthetic workloads for testing NoSQL databases. None of these works can be used to simulate various loads of real OLTP applications for application-oriented database performance evaluation.

When evaluating the query performance of a database management system (DBMS), it is common to perform a synthetic workload on the DBMS, and then observe the throughput and response time of the system, which makes the synthetic workload critical in the process of DBMS performance evaluation. If you want to perform DBMS performance evaluation for a specific application, the similarity between the synthetic load and the real load directly determines whether the evaluation result is credible. However, it is difficult for the current workload used for performance evaluation to have similar load characteristics to the target application, which leads to inaccurate evaluation results. In order to solve this problem, the present invention designs a synthetic load generation method oriented to transaction applications, which is used to capture the load characteristics of real online transaction processing (OLTP) applications, and then generates a synthetic load that is highly similar to the actual application load performance index.

SUMMARY OF THE INVENTION

In order to solve the defects in the background technology, the present invention proposes an application-oriented transaction load generation system, including: a database generation module and a load generation module; wherein,

The database generation module is used to generate a test database by acquiring database schema and data features;

The load generation module generates a synthetic load similar to the real load by analyzing the load trajectory of the real application; the load generation module includes:

A transaction logic analyzer, which extracts transaction logic information by analyzing the complete load trajectory in a short period of time;

A data access distribution analyzer, which extracts data access distribution information and throughput information by analyzing partial load trajectories over a long period of time;

A load generator, which utilizes the transaction logic information, the data access distribution information and the throughput information extracted previously to instantiate the parameter values in the transaction template to generate a synthetic load.

In the application-oriented transaction load generation system proposed by the present invention, the data feature is automatically obtained by using SQL query through the data feature extractor.

In the application-oriented transaction load generation system proposed by the present invention, the load generator can configure the number of test nodes and the number of test threads on each node to simulate concurrency; for each test thread, a separate database connection is established.

In the application-oriented transaction load generation system proposed by the present invention, when executing the transaction, the load generator uses the structure information of the transaction logic to determine whether the operation in the branch structure needs to be executed, and the number of times to execute the operation in the loop structure; The execution of an SQL operation first instantiates all parameters one by one, and then sends the operation with specific parameter values to the test database; after the operation is executed, the result set and parameters are saved as an intermediate state for subsequent execution within the same transaction instance. Additional parameters are generated in the operation.

In the application-oriented transaction load generation system proposed by the present invention, if the load generator only depends on $1 for a parameter, it can directly calculate the value of the parameter according to the increment Δ and the associated smaller parameter; $2, first try to instantiate the parameter by randomly selecting a dependency according to its probability, when none of the dependencies are selected, use the data access distribution to instantiate this parameter; if there are both dependencies $2 and $3, then the corresponding The operation must be in the loop structure. When the loop is executed for the first time, the instantiation parameters are based on the dependency $2 and the data access distribution. For non-first loop executions, first try to use the dependency $3 instantiation parameter based on the probability. If none is selected Dependencies, the parameters are instantiated with dependency$2 and the data access distribution.

Based on the above system, the present invention also proposes an application-oriented transaction load generation method, which includes the following steps:

Step A: Generate a test database by acquiring the database schema and data features;

Step B: Generate a synthetic load similar to the real load by analyzing the load trajectory of the real application, including:

Step B1: Extract transaction logic information by analyzing the complete load trajectory in a short period of time;

Step B2: Extract data access distribution information and throughput information by analyzing partial load trajectories in a long period of time;

Step B3: Using the previously extracted transaction logic information, the data access distribution information and the throughput information, instantiate the parameter values in the transaction template to generate a synthetic load.

In the application-oriented transaction load generation method proposed by the present invention, in step A, the test database is a plurality of tables that satisfy the data characteristics of primary key, foreign key constraint and non-key-value attribute data; specifically, the following steps are included:

Step A1: Generate primary keys in sequence;

Step A2: When generating a foreign key, randomly generate it within the value range of the primary key it refers to;

Step A3: Generate the value of the non-key-valued attribute through the random attribute generator including the random index generator and the index value converter, while satisfying the expected data characteristics.

d_fk是复合主键中的外键属性之一的值域；如果涉及到级联引用，第二步和第三步执行多次。In the application-oriented transaction load generation method proposed by the present invention, before generating the key value, first determine the value range: in the first step, if the primary key only includes a single attribute, then its value range is [1, s], s is the size of the table; in the second step, the value domain of the foreign key attribute can be determined by the primary key it refers to; in the third step, when processing the value domain of the non-foreign key attribute in the composite primary key, only a non-foreign key attribute, then the value range of this attribute is

d _fk is the value field of one of the foreign key attributes in the composite primary key; if cascading references are involved, the second and third steps are performed multiple times.

In the application-oriented transaction load generation method proposed by the present invention, the output of the random index generator is an integer from 1 to n, where n is the cardinality of the attribute; given an index, the index value converter deterministically maps it to the attribute value The value in the field; according to the data type of the attribute, different index value converters are used: for the numerical type, a linear function is used, which evenly maps the index to the attribute value field; for the string type, the randomly generated satisfying length is used The desired seed string; first select a seed string based on the input index, then concatenate the index and the selected seed string as the output value.

In the application-oriented transaction load generation method proposed by the present invention, in step B1, the transaction logic extraction algorithm includes:

Step B11: Calculate the execution times of each operation in the transaction template by traversing the load trajectory, thereby calculating the execution possibility of each branch and the average execution times of the loop operation;

Step B12: Identify all parameter pairs <pi,l,pi,j> that satisfy BR in the transaction template, and then traverse the load trajectory to obtain the average increment Δ; construct a dependency $1 for pi,j;

Step B13: For the parameters pi,j in each transaction template, traverse each parameter pm,n before it, and calculate the number of transactions that satisfy the ER parameter pair; similarly, traverse the previous return set rx,y, Calculate the number of transactions that satisfy ER and IR respectively;

Step B14: randomly select N transaction instance groups from the K transaction instances, two in each group; then calculate the LR coefficients (a, b) of the parameter pairs in each group of transactions;

Step B15: Use the statistical information data obtained in steps B13-B14 to construct a dependency $2 for each parameter pi,j;

Step B16: The same operation in the loop structure will be run multiple times, and the change of the parameters in it is described by the dependency $3; by traversing the load trajectory, the change of the parameter value in the continuously executed loop operation is calculated; the calculation of the coefficients (a, b) is the same as Step B14 is the same, and then construct the dependency $3 according to the statistical information.

The application-oriented transaction load generation method proposed by the present invention, in step B, specifically includes:

Step B21: generate all high-frequency terms that meet the expected repetition rate;

Step B22: traverse all parameters in the previous time window, and select repetition parameters for each interval until the parameter repetition rate on the interval is satisfied;

Step B23: derive the index of the parameter according to the value of the parameter, thereby identifying the interval to which it belongs in the current time window; if the index is not in the index domain of the current time window, ignore the parameter;

Step B24: Generate random parameters added to each interval to meet the cardinality requirement; based on the candidate parameters, use the parameter generation mechanism to instantiate the parameters; in a certain interval, just randomly select a candidate parameter as the output.

The invention discloses an application-oriented synthetic load generation system and method, which can capture the load characteristics of a real online transaction processing (OLTP) application, and then generate a synthetic load that is highly similar to the actual application load performance index, while ensuring information concealment and tools. Scalability and load scalability. The innovations mainly include:

1. A new method to describe the load characteristics of online transaction processing (OLTP) applications is proposed: OLTP load characteristics are characterized from two dimensions of transaction logic and data access distribution.

2. A method is proposed to extract transaction logic and data access distribution from real application load trajectories, while ensuring the concealment of application information.

3. From the perspective of controlling transaction conflicts and the proportion of distributed transactions, a method is proposed to ensure implicit transaction logic by controlling the dependencies of operation parameters.

4. Three angles to describe the data access distribution are designed and implemented: inclination, dynamics and continuity.

5. Implemented the first application-oriented OLTP load generator to ensure the authenticity and scalability of the generated load in performance evaluation.

references

[1] TPC-C benchmark, http://www.tpc.org/tpcc/.

[2] SQL Server Distributed Replay, https://docs.microsoft.com/enus/sql/tools/distributed-replay/sql-server-distributed-replay?id=2 view=sqlserver-2017.

[3] L. Galanis, S. Buranawatanachoke, R. Colle, B. Dageville, K. Dias, J. Klein, S. Papadomanolakis, L. L. Tan, V. Venkataramani, Y. Wang,

et al., “Oracle database replay,” in SIGMOD, 2008, pp.1159–1170.

[4] E.Lo, N.Cheng, W.W.K.Lin, W.Hon, and B.Choi, “Mybenchmark: generating databases for query workloads,” in VLDBJ, 2014, pp.895–913.

[5] Y. Li, R. Zhang, X. Yang, Z. Zhang, and A. Zhou, “Touchstone: Generatingenormous query-aware test databases,” in USENIX ATC, 2018, pp.575–586.

[6] Y.Tay, B.T.Dai, D.T.Wang, E.Y.Sun, Y.Lin, and Y.Lin, “Upsizer: Synthetically scaling an empirical relational database,” in Information Systems, 2013, pp.1168–1183.

[7] J. Zhang and Y. Tay, "Dscaler: Synthetically scaling a givenrelational database," in PVLDB, 2016, pp.1671–1682.

[8] P.S.Yu, M.-S.Chen, H.-U.Heiss, and S.Lee, “On workload characterization of relational database environments,” in IEEE Transactions on Software Engineering, 1992, pp.347–355.

[9] Q.T.Tran, K.Morfonios, and N.Polyzotis, "Oracle workload intelligence," in SIGMOD, 2015, pp.1669–1681.

[10] M. Alomari, M. Cahill, A. Fekete, and U. Rohm, “The cost of serializability on platforms that use snapshot isolation,” in ICDE, 2008, pp. 576–585.

[11] B.F. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan, and R. Sears, “Benchmarking cloud serving systems with ycsb,” in SoCC, 2010, pp.143–154.

[12] H.J.Jeong and S.H.Lee, "A workload generator for databasesystem benchmarks," in iiWAS, 2005, pp.813–822.

[13] P. Ameri, N. Schlitter, J. Meyer, and A. Streit, "Nowog: a workload generator for database performance benchmarking," in DASC/PiCom/DataCom/CyberSciTech, 2016, pp.666–673.

[14] P.E.O’Neil, E.J.O’Neil, X.Chen, and S.Revilak, “The star schemabenchmark and augmented fact table indexing,” in TPCTC, 2009, pp.237–252.

[15] R. Cole, F. Funke, L. Giakoumakis, W. Guy, A. Kemper, S. Krompass, H. Kuno, R. Nambiar, T. Neumann, M. Poess, et al., “The mixed workload ch-benchmark, "inDBTest, 2011, pp.8.

[16] F. Coelho, J. Paulo, R. Vilac, a, J. Pereira, and R. Oliveira, “Htapbench: Hybrid transactional and analytical processing benchmark,” in ICPE, 2017, pp. 293–304.

[17] SQL Server Profiler, https://docs.microsoft.com/en-us/sql/tools/sqlserver-profiler/sql-server-profiler?id=en view=sql-server-2017.

[18] Y. Wang, S. Buranawatanachoke, R. Colle, K. Dias, L. Galanis, S. Papadomanolakis, and U. Shaft, “Real application testing with databasereplay,” in DBTest, 2009, pp.8.

[19] C. Binnig, D. Kossmann, E. Lo, and M.T. Ozsu, "Qagen: generating¨query-aware test databases," in SIGMOD, 2007, pp.341–352.

[20] A. Arasu, R. Kaushik, and J. Li, “Data generation using declarative constraints,” in SIGMOD, 2011, pp. 685–696.

Description of drawings

Fig. 1 is the basic structure diagram of the present invention.

FIG. 2 is a schematic diagram of a value range for determining a key-value attribute according to the present invention.

Figure 3 is an example of the S-Dist of the present invention.

FIG. 4 is an example of parameter generation of the present invention.

Figure 5 is an example of the C-Dist of the present invention.

Figures 6a-6d are the deviations of the TPC-C load performance index simulated on the PostgreSQL database by the present invention.

Figure 7 is a discussion of S-Dist and D-Dist in tilted and dynamic loads in the present invention.

FIG. 8 is a discussion of C-Dist in continuous load in the present invention.

FIG. 9 is the performance of the transaction logic extraction algorithm of the present invention (K=N).

Figure 10 shows the performance of the C-Dist extraction algorithm of the present invention.

Detailed ways

The invention will be further described in detail with reference to the following specific embodiments and accompanying drawings. Except for the content specifically mentioned below, the process, conditions, experimental methods, etc. for implementing the present invention are all common knowledge and common knowledge in the field, and the present invention is not particularly limited.

Application-oriented database performance evaluation has the following requirements:

Fidelity. The load used for evaluation should closely resemble the real application load. The performance indicators (such as throughput, latency, physical resource utilization) obtained by the evaluation should be consistent with the results of actual application operation. The similarity between the evaluation load and the real application load is measured by the deviation of the performance index. The smaller the deviation, the higher the similarity.

concealment. Data privacy is a basic requirement of commercial applications, so the workload of real applications cannot be directly used for database performance evaluation.

Tool Extensibility. The target application may have large data size and high concurrency/throughput. This requires load generation tools that can scale across multiple nodes and support concurrent database and load generation.

Load scalability. Sometimes, it is necessary to scale the current application load to measure the performance of the DBMS under the scale of the expected synthetic load. Since the main focus is on generating transactional workloads, query concurrency and query throughput are key concerns.

Based on these requirements, the present invention formalizes the application-oriented transaction load generation problem:

Application-oriented synthetic workload generation: Generate synthetic workloads that are highly similar to the target application while maintaining fidelity, stealth, tool scalability, and workload scalability.

Based on the above definition of the data generation problem, the basic architecture designed by the present invention is shown in FIG. 1 . In order to solve the data privacy problem, the method of the present invention will isolate the production environment and the evaluation environment, so that the data owner can protect the data privacy. From the perspective of functions, the implementation of the present invention can be divided into a database generation module (Database Generation) and a load generation module (Workload Generation).

Database generation module (Database Generation): The input of the database generator mainly includes two parts: database schema and data characteristics; the output is a test database (Test DB). Since data characteristics are monotonous and verbose and need to be obtained from a real database, the present invention provides a data characteristic extractor (Data Characteristics Extractor) to help the present invention automatically obtain these information using simple SQL queries. In the design, the present invention only focuses on some basic data characteristics of the test database (such as the value range of attributes), because the load characteristics of the synthetic load are the key factors affecting the performance of the DBMS.

Load Generation Module (Workload Generation): The load generation module consists of three parts: Transaction Logic Analyzer, Data Access Distribution Analyzer and Load Generator. The transaction logic analyzer extracts transaction logic information by analyzing the full load trace (including all parameters and returned result sets for each SQL operation) over a short period of time. The data access distribution analyzer extracts data access distribution information and throughput information by analyzing partial load trajectories over a long period of time (including only some key parameters of SQL operations). The load generator uses this information to instantiate the parameter values in the transaction template to generate a synthetic load.

Database generation in the present invention:

The data characteristics required for all database generation are the same as Touchstone [5], such as table size, attribute value range, attribute cardinality, that is, the number of distinct values. Generating a test database is actually generating multiple tables that satisfy primary/foreign key constraints and non-key attribute data characteristics.

d_fk是复合主键中的外键属性之一的值域。其它情况的处理类似。如果涉及到级联引用，第二步和第三步可能要执行多次。Without loss of generality, the present invention assumes that both primary and foreign keys are integers. Primary keys are identifiers for records and usually have no physical meaning, so their data characteristics are not considered. When generating a primary key, the present invention simply generates sequentially; when generating a foreign key, the present invention randomly generates within the value range of the primary key it refers to. This guarantees the uniqueness of primary keys and the referential integrity of foreign keys. Before generating these keys, three steps are required to determine their range. In the first step, if the primary key only includes a single attribute, its value range is [1, s], and s is the size of the table (① in Figure 3); in the second step, the value range of the foreign key attribute can be It is determined by the primary key it refers to (2 in Figure 3); in the third step, the present invention deals with the value range problem of non-foreign key attributes in the composite primary key (3 in Figure 3). The most common and reasonable case is that there is only one non-foreign key attribute in the composite primary key, then the value range of this attribute is

d _fk is the value field of one of the foreign key attributes in the composite primary key. Other cases are handled similarly. Steps 2 and 3 may be performed multiple times if cascading references are involved.

A random attribute generator [5], which includes a random index generator and an index value converter, is used to generate values for non-key-valued attributes while satisfying desired data characteristics, especially cardinality characteristics. The output of the random index generator is an integer from 1 to n, where n is the cardinality of the attribute. Given an index, the index-to-value converter deterministically maps it to a value in the property's range. According to the data type of the attribute, the present invention uses different index value converters. For numeric types, such as integers, the present invention simply uses a linear function, which maps indices uniformly to attribute value ranges; for string types, such as varchar, there will be some randomly generated seed strings that meet length requirements. The present invention first selects a seed string (such as the (i%k)th seed string according to the input index, where i is the index and k is the number of all seed strings. Then the index and the selected seed string are connected together as the output value.

In summary, each table is generated independently of each other. And for each table, by assigning primary key generation to each thread by range, the present invention can realize parallel data generation on multiple nodes.

Transaction logic in the present invention:

The transaction logic mentioned in this article represents the underlying business logic in an OLTP application. In the process of database testing, transaction logic will have a significant impact on the possibility of deadlock and the proportion of distributed transactions, thereby affecting the performance of the test database. In this part, the present invention first introduces the definition of transaction logic, and then presents the algorithm for extracting transaction logic.

The relationship between SQL parameters and SQL parameters in a transaction template and the relationship between SQL parameters and returned items determine the hidden semantics between SQL operations. After investigating existing OLTP benchmark loads and real-world application loads, the present invention focuses on four types of relationships. First, the equality relation (ER) is the most common relation. For example, with a certain probability, two SQL parameters are equal. Second, inclusion relations (IR) are also common. Because the SQL result return set may be a set of tuples, the value of the SQL parameter may be a value in the previous returned result set. Third, the linear relationship (LR) is a supplement and extension to the equality relationship, and has stronger expressive power. Fourth, the between relation (BR) is proposed for predicates of the form "col between p1 and p2" and "col≥p1 and col≤p2", where there is a between relation between p2 and p1.

The transaction logic is formally defined below. O _i represents the i-th operation in the transaction template; pi _,j represents the j-th parameter of O _i ; ri _,j represents the j-th return item of O _i ; both i and j are counted from 1.

Definition 2 Transaction logic: For a transaction template, the transaction logic consists of transaction structure information and parameter dependency information, as detailed below:

· Transaction structure information:

#1 Possibility of executing each branch in a branch structure.

The average number of executions of #2 operations in a loop structure.

Parameter dependency information (for each parameter p _i,j ):

$1[pi _,l ,pi _,j ,BR,Δ]

$2 a list of dep-item, dep-item∈{[p _m,n ,pi _,j ,ER,ξ],[p _m,n ,pi _,j ,LR,ξ,(a,b)] ,[r _x,y ,pi _,j ,ER,ξ],[r _x,y ,pi _,j ,LR,ξ,(a,b)],[r _x,y ,pi _,j , IR,ξ]}; m≤i; x<i; if m=i, then n<j.

$3 A list of [pi _,j ,LR,ξ,(a,b)], O _i must be an operation in a loop structure.

If a parameter pi _,j has a dependency on $1, then neither $2 nor $3 need exist, because pi _,j can be expressed as (pi _,l + Δ), where Δ is the average increment calculated from the load trajectory . In each dependency, ξ denotes the probability that the corresponding dependency is satisfied; (a, b) are two coefficients describing the linear relationship. Dependency $3 represents a linear relationship between the values of the same parameter in successively executed loop operations.

The transaction logic is the embodiment of the application layer business logic and does not change frequently, so there is no need to analyze the load trajectory over a long period of time. Since the transaction logic analysis process of each transaction template is the same and independent of each other, the subsequent extraction algorithm of the present invention only analyzes one transaction template. The transaction logic extraction algorithm consists of six steps, and the input is a transaction template and K transactions on the corresponding load trajectory. The algorithm is as follows:

Step 1: Extract transaction structure information. By traversing the load trajectory, the number of executions of each operation in the transaction template is calculated, thereby calculating the probability of each branch execution and the average execution times of the loop operation.

Step 2: Determine BR. First, the present invention identifies all parameter pairs <pi _,l ,pi _,j > in the transaction template that satisfy BR, and then traverses the load trajectory to obtain the average increment Δ. Then, the present invention builds the dependency $1 for pi _,j , after which the processing of steps 3-6 can skip pi _,j .

Step 3: Collect ER and IR information. For the parameters p _i,j in each transaction template, the present invention traverses each parameter p _m,n before it, and calculates the number of transactions that have the ER parameter pair (ie p _i,j =p _m,n ) ; Similarly, traverse the previous return set r _x,y to calculate the number of transactions that satisfy ER and IR respectively.

Step 4: Collect LR information. LR only involves parameters and return sets of numeric types, and the return sets must be obtained by filtering operations by primary keys. Since the calculation of the LR coefficients (a, b) requires two transaction instances, the present invention randomly selects N transaction instance groups (two in each group) from the K transaction instances. The LR coefficients (a, b) of the parameter pairs in each set of transactions are then calculated. LR with coefficient (1,0) needs to be ignored here because it is represented by ER.

Step 5: Determine ER, IR and LR by weighing. Using the statistical data obtained in steps 3-4, the present invention can easily construct a dependency $2 for each parameter p _i,j . However, for each parameter, there may be many dependencies, and some parameters have a small probability (ie, ξ), so the present invention requires a trade-off between these dependencies to eliminate noise and reduce subsequent computations. The present invention selects the most important dependencies, such as dependencies with higher probability, and ensures that the sum of the probabilities of the selected dependencies is less than 1. The present invention prefers to preserve ER, as ER was found to be much more important than IR and LR in the experiments.

Step 6: Construct the LR of the cyclic structure. In the loop structure, the same operation will be run multiple times, and the present invention uses the dependency $3 to describe the change of the parameters. By traversing the load trajectory, the changes in parameter values in the continuously executed loop operation are calculated. The calculation of coefficients (a, b) is similar to step 4. Then construct the dependency $3 based on the statistics.

If steps 3-4 encounter an operation in the loop, the present invention only uses the load trajectory for the first execution of the loop. Data access distribution in the present invention:

Data access distribution has a non-negligible impact on the conflict intensity of load transactions and the cache hit rate of database systems, so it has always been regarded as an important feature of application load. In this part, the present invention firstly describes the basic oblique data access distribution, then the dynamics and continuity of the data access distribution, and finally gives the generation algorithm of the candidate parameters.

The distribution of data access for synthetic workloads depends on the parameter values instantiated in the transaction template. Without loss of generality, it is assumed that the form of predicates that determine the distribution of data access in OLTP applications can be expressed as "col op para". The present invention represents the skewed data access distribution (S-Dist) for each parameter using high frequency itemsets (HFI) and histogram statistics (HS) extracted from the load trajectory. HFI records the H hot data items with the highest frequency. The domain value of the attribute is evenly divided into I intervals, and then the frequency and cardinality of the parameters on each interval (except those that have appeared in HFI) are recorded as HS. Figure 3 is an example of S-Dist, where the value of H and I are both 5, the corresponding attribute is an integer, and the domain value is [0, 2000]. In re-HFI, the hottest item is 57, and the frequency of occurrence is 0.17; in the first interval of re-HS, there are 20 unique parameter values, and the total access frequency is 0.08.

The data items in the S-Dist all come from load trajectories running on real data, but the same data does not necessarily exist in the synthetic database generated by the present invention, so the present invention first needs to do data conversion for HFI. Suppose the attribute generator is "index=ranInt[1,400], value=index*5", where 400 is the attribute cardinality. The first step is to use the attribute generator to regenerate the data items in the HFI, as shown in Figure 4, where 57 is replaced by 195. The second step is to calculate the cumulative probability array based on the frequency of the high frequency items and all intervals, that is, Figure 4 "cumu prob" in . Finally, a random number between 0 and 1 is generated and mapped to an item in the cumulative probability array, thereby picking out suitable parameters to complement the predicate. Figure 4 shows two examples of selecting parameter values.

其中cdn_i是目标区间的基数，cdn_avg是每个区间的平均基数，minIdx_i是目标区间的最小索引。在图4中，区间2的随机索引生成器为

其中cdn₂＝50，

minIdx₂＝80*2+1＝161.In addition, in order to control the cardinality of the generated parameters in each interval, the present invention redefines the random index generator as

where cdn _i is the cardinality of the target interval, cdn _avg is the average cardinality of each interval, and minIdx _i is the minimum index of the target interval. In Figure 4, the random index generator for interval 2 is

where cdn ₂ =50,

minIdx ₂ =80*2+1=161.

Although the parameters in the above examples are of integer type, the method of the present invention is general. For all numeric parameters of non-key-value attributes, S-Dist and parameter generation are identical. For parameters of key-value properties, there are small differences. Because the primary keys in the synthetic database are generated sequentially, the field values of the key-valued properties in the synthetic database may differ from the primary keys in the real database. Therefore, when collecting S-Dist statistics, the present invention uses the domain value of the key-value attribute in the real database to divide the interval and construct the HS. However, in the synthetic load generation process, the present invention uses the domain value of the key-value attribute in the synthetic database to support parameter generation. For string type parameters, the biggest difference is how to divide the interval. When constructing HS, the interval to which the string type parameter belongs is calculated by h%I, where h is the hash value of the parameter. The index value converter is identical to that used for synthetic database generation.

dynamic. If the data access distribution changes dynamically, S-Dist is inaccurate, or even completely wrong. Suppose there is a table with 100 records and a load of 100 seconds. In the ith second, all database requests of the load access only the ith record in the table. It is assumed that the throughput of the database is stable during this period. At this point, if you only use S-Dist to represent the entire data access process, you will find that there is no hotspot data, and the data access distribution is very uniform. Obviously, this is very different from the truth. Using the synthetic workload generated by this S-Dist, the transaction conflict intensity on the database will be significantly lower than that of the actual workload. Therefore, D-Dist is proposed in the present invention, and on the basis of S-Dist, dynamic characterization is added. First, the load trajectory of the same parameter is divided into multiple time windows of equal length according to the timestamp of the log. Then, for the parametric trajectory in any time window, the present invention will generate a separate S-Dist, and define the D-Dist of the entire parametric trajectory as a list of S-Dist. Finally, the present invention instantiates the symbol parameters using the S-Dist corresponding to the generation time. Also, for numeric parameters, the data range used in the time window can be much smaller than the domain value. In order to improve the accuracy of HS, the interval can be divided according to the data range of the current window. Of course, when generating parameters, the corresponding index range for each interval must also be used.

continuity. In some applications, the popularity of data is closely related to time, which means that the data can be accessed continuously for a period of time. The present invention refers to the continuity of data access distribution. Previously, D-Dist captured the deviation of data access in time windows, ignoring the continuity of data access between consecutive time windows. When using it to generate synthetic loads, the data accessed between consecutive time windows can be quite different, resulting in lower cache hit rates. Therefore, the present invention proposes C-Dist, and on the basis of D-Dist, the characterization of continuity is added. When collecting statistical data, the present invention calculates the repetition rate of the high frequency item in the current time window and the high frequency item in the previous time window, as well as the parameter repetition rate of all intervals. FIG. 5 increases the repetition frequency of HFI and HS based on the example in FIG. 3 . In this example, the present invention can see that the repetition rate of the HFI is 0.6, that is, three high frequency terms are kept from the previous time window. The repetition frequencies of the five intervals are 0, 0.33, 0.5, 0.46 and 0.56, respectively. Assuming cdn ₁ =15, then in interval 1 there are 15*0.33≈5 parameters appearing in the previous time window.

In order to ensure the parameter repetition rate in the C-Dist, the present invention needs to pre-generate candidate parameters for each time window. The detailed generation process of candidate parameters is given in Algorithm 1. In lines 1-2, generate all high frequency terms that satisfy the expected repetition rate. In lines 3-10, iterate over all parameters in the previous time window and choose repetition parameters for each interval until the parameter repetition rate on the interval is satisfied. In line 6, the index of the parameter is derived according to the value of the parameter, thereby identifying the interval to which it belongs in the current time window. This parameter is ignored if the index is not in the index field of the current time window. For parameters of string type, such as "296#dgtckuy", the part before the "#" character is the index required by the present invention. Finally, in lines 11-13, the present invention generates random parameters added to each interval to meet cardinality requirements. Based on these candidate parameters, the present invention instantiates the parameters using the parameter generation mechanism of FIG. 4 . Within a certain interval, the present invention only needs to randomly select a candidate parameter as an output. In addition, if candidate parameters are generated online during the synthetic load generation process, the load generator may become a performance bottleneck, thus affecting the correctness of the evaluation results. Therefore, the present invention can generate candidate parameters for all time windows offline, store them on disk, and then read them as needed when generating synthetic loads.

The generated load in this example:

Given the transaction logic of each transaction template and the data access distribution of each parameter, the workload generator (Workload Generator) in Figure 1 is responsible for generating a synthetic workload that satisfies the specified configuration. At the same time, efficiently generating a high-concurrency and high-throughput synthetic load in a distributed environment is also a basic requirement for the load generator of the present invention. The details of load generation are described below from three levels: threading model, transaction execution, and parameter instantiation.

threading model. When deploying the load generator, the user can configure the number of test nodes and the number of test threads on each node to simulate concurrency. For each test thread, the present invention establishes a separate database connection. Two different execution models are implemented to support test thread invocation transactions: no wait loop and fixed throughput. In the no-wait-loop setup, all test threads kept issuing transactions without any think time between requests. In the fixed throughput setting, the user can specify a fixed request throughput or throughput scaling factor. If a throughput scale factor is specified, the product of the throughput in each time window and the throughput scale factor is the target throughput for that window. The test thread achieves the desired throughput by controlling the think time between transaction requests. When the required throughput exceeds the maximum throughput that can be achieved by the current test thread, the execution model degenerates into a no-wait loop. Different execution models enable the present invention to build scalable synthetic payloads.

Transaction execution. The test thread invokes different types of transactions according to the proportion of transactions extracted from the load trace. The transaction ratio is adjusted periodically over time windows. When executing a transaction, the structure information of the transaction logic is used to determine whether an operation in a branch structure needs to be performed, and how many times to perform an operation in a loop structure. For the execution of an SQL operation, the present invention first instantiates all parameters one by one, and then sends the operation with specific parameter values to the test database. After an operation is performed, the result set and parameters are saved as an intermediate state so that additional parameters can be generated in subsequent operations within the same transaction instance.

parameter instantiation. When instantiating parameters, first ensure the consistency of the transaction logic, and then ensure the data access distribution of the synthetic load. For a parameter, there are the following situations. Case 1: If only $1 is dependent, the value of this parameter can be calculated directly from the increment Δ and the associated smaller parameter. Case 2: If there are only dependencies $2, the present invention first tries to instantiate the parameters by randomly selecting dependencies according to their probabilities. When none of the dependencies are selected, the data access distribution is used to instantiate this parameter. Case 3: If there are dependencies $2 and $3 at the same time, the corresponding operation must be in a loop structure. When the loop is executed for the first time, as in case 2, the present invention still instantiates parameters based on dependency $2 and data access distribution; for non-first loop executions, the present invention first tries to use dependency $3 instantiation parameters based on probability, if no Checking any of the dependencies uses dependency $2 and the data access distribution to instantiate the parameters.

In summary, transaction execution and parameter instantiation are independent of each other for all test threads, so the load generator of the present invention can be deployed on multiple nodes to efficiently generate high concurrency, throughput synthetic load while satisfying the required load characteristics and configuration.

Example

lab environment

Experimental hardware configuration: 4 physical nodes, each with 2 CPUs, the model is Intel Xeon Silver4110@2.1GHz; the memory is 120GB; the storage is 4TB, RAID-5, 4GB RAID cache. 10 Gigabit Ethernet communication is used between physical nodes.

Experiment 1: Use the TPC-C load as the simulation object, change the expansion factor, and detect the synthetic load by comparing the throughput, latency, CPU utilization and disk utilization of the real load (ie TPC-C load) and the synthetic load on the same database Load fidelity. The experimental database is PostgreSQL. The experimental results are shown in Figure 6a-6d. The number of concurrent database requests is the same as the scaling factor. In Figure 6a, we present the transaction execution throughput for real and synthetic workloads. The results show that the throughput of the two workloads is very similar, with a maximum deviation of 6.29%. The metrics for average delay and 95% delay are shown in Figure 6b. From these two metrics, the synthetic load is very close to the real load, with a maximum deviation of only 8.99%. Figure 6c and Figure 6d show the CPU and disk usage for the two workloads, respectively. The results show that the resource consumption of executing the synthetic load and the real load in the PostgreSQL database is consistent, which further verifies the high fidelity of the synthetic load generated by the present invention.

Experiment 2: This experiment demonstrates the ability of data access distributions (i.e., S-Dist, D-Dist, and C-Dist) to describe the inclination, dynamics, and continuity of data access. Since the data access distribution of existing benchmark workloads is usually neither dynamic nor continuous, we build the evaluation workload based on YCSB. The experiments were all performed on a MySQL database with a test table from YCSB. The size of the test table is 10 ⁶ , and the number of concurrent database requests is 20.

The evaluation load in Figure 7 has only one transaction type. The transaction consists of five pairs of read and write operations, each of which reads a record and then updates it. The extended YCSB load runs for 90 seconds and is divided into three phases, with data requests in each phase randomly selected within ¹⁰³ records. In the first stage, the data access distribution is a Zipf distribution with parameter s=1; the second stage is still a Zipf distribution, but with a parameter s=1.2; the third stage is a uniform distribution. Figure 7 shows the dynamic changes in transaction throughput, latency, and deadlock amount for YCSB and loads generated by the present invention. It can be seen from the results that when using D-Dist, the synthetic load generated by the present invention is dynamically consistent with the real load generated by YCSB in throughput, delay and deadlock amount, which shows that D-Dist can well Describe the dynamic nature of the workload. Meanwhile, in each time window, D-Dist is represented by S-Dist, which also shows that S-Dist can well characterize the inclination of the load. But the global S-Dist is not very good, it is defined in the whole load time, regardless of the dynamic changes of the load.

The evaluation workload in Figure 8 is a single-row update transaction for YCSB running for 100 seconds with a time window of 1 second. Data requests in each time window are based on ¹⁰³ random records, while the selected records for each time window are 50% identical to those of the previous window. MySQL's Innodb_buffer_pool_size is set to 16MB. Figure 8 presents the throughput and Innodb_buffer_pool_reads delta for loads generated by YCSB and the present invention, respectively. Innodb_buffer_pool_reads is the number that InnoDB must read directly from disk. The results show that the disk access rate of D-Dist is significantly higher than that of YCSB, and the throughput is lower. This is because D-Dist cannot capture the continuity of data access distribution, resulting in almost completely different data requests in each time window and a low cache hit rate. The load performance of using C-Dist is consistent with that of YCSB, which indicates that C-Dist can well characterize the continuity of data access.

Experiment 3: When the number of transaction instances (K) and the number of transaction instance groups (N) are equal, change K and N at the same time, and observe the performance of the transaction logic extraction algorithm through execution time and memory consumption. The experimental results are shown in Figure 9. As can be seen from the results, when both K and N are ¹⁰⁴ , the fetch time of the transaction logic is only 2.1 seconds, and the memory consumption is 1.1GB. As K and N increase, the execution time and memory consumption increase almost linearly. Overall, the transaction logic extraction proposed in this paper is efficient and can be completed in seconds.

Experiment 4: Change the length of the load trajectory, and observe the performance of the C-Dist extraction algorithm through execution time and memory consumption. The experimental results are shown in Figure 10. The results show that the extraction time of C-Dist is linear with the load trajectory, while the memory consumption is constant. This is because C-Dist is a window-based data access distribution, and the load trace for each time window can be removed from memory after processing is complete. In Figure 10, when the TPC-C load trajectory time is 10 ⁴ seconds (the transaction throughput is 3610.3, and the log volume is 33.8GB), the extraction time of C-Dist is 678.7s, and the memory consumption is 1.2GB. Since the maximum load period in the actual evaluation is generally one day, the present invention can effectively support the performance evaluation of high-throughput loads.

The protection content of the present invention is not limited to the above embodiments. Variations and advantages that can occur to those skilled in the art without departing from the spirit and scope of the inventive concept are included in the present invention, and the appended claims are the scope of protection.

Claims (10)

1. an application-oriented transaction load generation system, characterized in that, comprising: a database generation module and a load generation module; wherein, The database generation module is used to generate a test database by acquiring database schema and data features; The load generation module generates a synthetic load similar to the real load by analyzing the load trajectory of the real application; the load generation module includes: A transaction logic analyzer, which extracts transaction logic information by analyzing the full load trajectory in a short period of time; Data access distribution analyzer, which extracts data access distribution information and throughput information by analyzing partial load trajectories over a long period of time; A load generator, which utilizes the transaction logic information, the data access distribution information and the throughput information extracted before to instantiate the parameter values in the transaction template to generate a synthetic load; the load generator is executing During a transaction, the structure information of the transaction logic is used to determine whether the operation in the branch structure needs to be executed, and the number of times to execute the operation in the loop structure; for the execution of an SQL operation, all parameters are first instantiated one by one, and then the parameters with specific parameter values are instantiated one by one. The operation is sent to the test database; after the operation is executed, the result set and parameters are saved as an intermediate state for generating additional parameters in subsequent operations within the same transaction instance. 2 . The application-oriented transaction load generation system of claim 1 , wherein the data features are automatically obtained using SQL queries through a data feature extractor. 3 . 3. The application-oriented transaction load generation system of claim 1, wherein the load generator can configure the number of test nodes and the number of test threads on each node to simulate concurrency; for each test thread , to establish a separate database connection. 4. The application-oriented transaction load generation system according to claim 1, wherein, if the load generator only depends on $1 for one parameter, it can directly calculate according to the increment Δ and the associated smaller parameter The value of this parameter; if there are only dependencies $2, first try to instantiate the parameter by randomly selecting the dependencies according to their probabilities, when none of the dependencies are selected, use the data access distribution to instantiate this parameter; if both Depending on $2 and $3, the corresponding operation must be in the loop structure. When the loop is executed for the first time, the parameters are instantiated based on the dependency $2 and the data access distribution. For non-first loop execution, first try to use the dependency $3 based on the probability. Instantiate the parameter, if no dependencies are checked, use the dependency $2 and the data access distribution to instantiate the parameter. 5. An application-oriented transaction load generation method, comprising the following steps: Step A: Generate a test database by acquiring the database schema and data characteristics; Step B: Generate a synthetic load similar to the real load by analyzing the load trajectory of the real application, including the following sub-steps: Step B1: Extract transaction logic information by analyzing the complete load trajectory in a short period of time; Step B2: Extract data access distribution information and throughput information by analyzing partial load trajectories in a long period of time; Step B3: Using the previously extracted transaction logic information, the data access distribution information, and the throughput information, instantiate the parameter values in the transaction template to generate a synthetic load; when the load generator executes the transaction, use the The structure information of the transaction logic to determine whether the operation in the branch structure needs to be executed, and the number of times to execute the operation in the loop structure; for the execution of an SQL operation, all parameters are first instantiated one by one, and then the operation with specific parameter values is sent to Test the database; after an operation is performed, the result set and parameters are saved as an intermediate state to generate additional parameters in subsequent operations within the same transaction instance. 6. The application-oriented transaction load generation method according to claim 5, wherein in step A, the test database is a plurality of tables satisfying primary key, foreign key constraints and non-key attribute data characteristics; specifically including The following steps: Step A1: Generate primary keys in sequence; Step A2: When generating a foreign key, randomly generate it within the value range of the primary key it refers to; Step A3: Generate the value of the non-key-valued attribute through the random attribute generator including the random index generator and the index value converter, while satisfying the expected data characteristics. 7. The application-oriented transaction load generation method according to claim 6, characterized in that, before generating the key value, first determine the value domain: in the first step, if the primary key only includes a single attribute, then its The value range is [1, s], s is the size of the table; the second step, the value range of the foreign key attribute can be determined by the primary key it refers to; the third step, processing the value of the non-foreign key attribute in the composite primary key domain, there is only one non-foreign key attribute in the composite primary key, then the value domain of this attribute is

d _fk is the value field of one of the foreign key attributes in the composite primary key; if cascading references are involved, the second and third steps are performed multiple times. 8. The application-oriented transaction load generation method of claim 6, wherein the output of the random index generator is an integer from 1 to n, where n is the cardinality of the attribute; given an index, the index value converter Deterministically map it to a value in the property's range; use different index-to-value converters depending on the data type of the property: for numeric types, use a linear function that maps indices uniformly to the property's range; for characters String type, randomly generated seed string that meets the length requirement; first select a seed string according to the input index, and then concatenate the index and the selected seed string as the output value. 9. The application-oriented transaction load generation method according to claim 5, wherein in the step B1, the transaction logic extraction algorithm comprises: Step B11: Calculate the execution times of each operation in the transaction template by traversing the load trajectory, thereby calculating the execution possibility of each branch and the average execution times of the loop operation; Step B12: Identify all parameter pairs <pi _,l ,pi _,j > in the transaction template that satisfy the BR, and then traverse the load trajectory to obtain the average increment Δ; build a dependency $1 for pi _,j ; Step B13: For the parameters p _i,j in each transaction template, traverse each parameter p _m,n before it, and calculate the number of transactions that satisfy the ER parameter pair; similarly, traverse the previous return set r _{x ,y} , respectively calculate the number of transactions that satisfy ER and IR; Step B14: randomly select N transaction instance groups from the K transaction instances, two in each group; then calculate the LR coefficients (a, b) of the parameter pairs in each group of transactions; Step B15: Use the statistical data obtained in steps B13-B14 to construct a dependency $2 for each parameter p _i,j ; Step B16: In the loop structure, the same operation will be run multiple times, and use dependencies to describe the changes in the parameters; by traversing the load trajectory, calculate the changes of parameter values in the continuously executed loop operations; the calculation and steps of the coefficients (a, b) Same as B14, then construct dependency $3 based on statistics. 10. The application-oriented transaction load generation method according to claim 5, wherein in step B, the method specifically comprises: Step B21: Generate all high-frequency terms that meet the expected repetition rate; Step B22: traverse all parameters in the previous time window, and select repetition parameters for each interval until the parameter repetition rate on the interval is satisfied; Step B23: derive the index of the parameter according to the value of the parameter, thereby identifying the interval to which it belongs in the current time window; if the index is not in the index domain of the current time window, ignore the parameter; Step B24: Generate random parameters added to each interval to meet the cardinality requirement; based on the candidate parameters, use the parameter generation mechanism to instantiate the parameters; in a certain interval, just randomly select a candidate parameter as the output.

Images