CN114445220B - Transaction data sorting method and device, database server and storage medium - Google Patents
Transaction data sorting method and device, database server and storage mediumInfo
- Publication number
- CN114445220B CN114445220B CN202011214451.7A CN202011214451A CN114445220B CN 114445220 B CN114445220 B CN 114445220B CN 202011214451 A CN202011214451 A CN 202011214451A CN 114445220 B CN114445220 B CN 114445220B
- Authority
- CN
- China
- Prior art keywords
- clearing
- transaction
- transaction data
- sorting
- cleared
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/02—Banking, e.g. interest calculation or account maintenance
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/04—Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
Landscapes
- Business, Economics & Management (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- Engineering & Computer Science (AREA)
- Development Economics (AREA)
- Economics (AREA)
- Marketing (AREA)
- Strategic Management (AREA)
- Technology Law (AREA)
- Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a transaction data sorting method, a transaction data sorting device, a database server and a storage medium. The method comprises the steps of sorting transaction data in a transaction batch by utilizing a plurality of sorting stages, determining transaction data to be sorted corresponding to the transaction batch in a transaction database in each sorting stage, and sorting the transaction data to be sorted by utilizing a plurality of sorting execution stages in the sorting stage, wherein the amount of the transaction data to be sorted in each sorting execution stage is not greater than a first threshold value. According to the technical scheme, the transaction data to be cleared in one transaction batch is processed by utilizing a plurality of clearing stages instead of a single clearing stage, and the upper limit of the transaction data amount which can be cleared in each clearing execution stage is limited, so that the excessive high transaction data amount cleared in the clearing execution stage is avoided, a clearing database is not required to be independently erected outside a transaction database, the hardware cost is saved, and meanwhile, the delay of the database main-standby synchronization is reduced.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a transaction data sorting method, a transaction data sorting device, a database server, and a storage medium.
Background
The clearing (Clearing) is a data preparation stage of clearing, and is mainly to collect, sort and classify all transaction data of the current day according to the cost, the cost of the generation, the credit, the debit, the number of strokes, the amount of money, the net amount of the rolling stock and the like among the member rows.
Currently, the technical problem that the instantaneous TPS (Transaction Per Second for processing transaction data volume per second) of a database is too high exists in the sorting process, and the technical problem needs to be solved.
Disclosure of Invention
The embodiment of the application provides a clearing method and device for transaction data, a database server and a storage medium, so as to avoid the problem that the instantaneous TPS of a transaction database is too high.
The embodiment of the application adopts the following technical scheme:
In a first aspect, an embodiment of the present application provides a method for sorting transaction data, which is executed by a database server, and includes sorting transaction data in a transaction batch by using a plurality of sorting periods, wherein in each sorting period, transaction data to be sorted in a transaction database corresponding to the transaction batch is determined, and sorting the transaction data to be sorted by using a plurality of sorting execution periods in the sorting period, wherein the amount of the transaction data to be sorted in each sorting execution period is not greater than a first threshold.
Optionally, determining the transaction data to be cleared corresponding to the transaction batch in the transaction database comprises inquiring the transaction data which is matched with the transaction time condition, is in a final transaction state and is not cleared according to the transaction time condition corresponding to the transaction batch from the transaction database as the transaction data to be cleared corresponding to the transaction batch.
Optionally, the transaction data which is matched with the transaction time condition, is in the final state of the transaction and is not cleared is inquired from the transaction database and used as the transaction data to be cleared, wherein the transaction data which is in the final state of the transaction is determined according to the transaction state identification of the transaction data, and the transaction data which is not cleared is determined according to the clearing identification of the transaction data.
Optionally, the sorting the transaction data to be sorted respectively by using the sorting execution periods in the sorting period includes sorting the transaction data to be sorted piece by piece in the sorting execution period, accumulating the amount of the sorted transaction data in the sorting execution period, and terminating sorting in the sorting execution period when the accumulated amount of the transaction data reaches a first threshold or all the transaction data to be sorted are sorted.
Optionally, the method further comprises determining, in batches, a clearing identifier of the transaction data for which clearing has been completed during a clearing execution period.
Optionally, determining the clearing identifier of the transaction data with the clearing completed in batches comprises accumulating the transaction data amount with the clearing completed and the clearing identifier not determined, determining the clearing identifier for the transaction data batch with the clearing completed and the clearing identifier not determined under the condition that the accumulated transaction data amount reaches a second threshold value, and clearing the accumulated transaction data amount after determining the clearing identifier.
Optionally, the maximum executable time length of each clearing execution period is the same, and the clearing the transaction data to be cleared respectively by using a plurality of clearing execution periods in the clearing execution periods comprises clearing the transaction data to be cleared by using a clearing line process in the clearing execution periods, and counting the total clearing time length, wherein if the total clearing time length is smaller than the maximum executable time length, the dormancy time length is determined according to the total clearing time length and the maximum execution time length, so that the clearing line process is dormant according to the dormancy time length.
Optionally, a total length of the plurality of clearing execution periods within a single clearing period is not greater than the third threshold.
Optionally, the method further comprises the steps of receiving a first clearing task, wherein the first clearing task corresponds to the transaction batch, determining the first execution time of the first clearing task according to the initial transaction time and the first delay time of the transaction batch, starting the first clearing period of the transaction batch according to the first execution time of the first clearing task, and executing the first clearing task in each clearing period respectively to realize clearing steps by utilizing a plurality of clearing periods.
Optionally, the method further comprises the steps of receiving a second clearing task, wherein the second clearing task corresponds to the transaction batch, the execution time of the second clearing task is determined according to the initial transaction time and the second delay time of the transaction batch, and executing the second clearing task according to the execution time of the second clearing task to clear all the transaction data of the transaction batch, which are not completed.
In a second aspect, an embodiment of the present application further provides a transaction data sorting device, configured to implement the transaction data sorting method as described in any one of the above.
In a third aspect, embodiments of the present application also provide a database server comprising a processor, and a memory arranged to store computer executable instructions which, when executed, cause the processor to perform a method of sorting transaction data as described in any of the above.
In a fourth aspect, embodiments of the present application also provide a computer-readable storage medium storing one or more programs that, when executed by a database server comprising a plurality of application programs, cause the database server to perform a method of sorting transaction data as described in any of the above.
The at least one technical scheme adopted by the embodiment of the application has the advantages that the transaction data to be cleared in one transaction batch is processed by utilizing a plurality of clearing stages instead of a single clearing stage, and the upper limit of the transaction data amount which can be cleared in each clearing execution stage is limited, so that the excessive high transaction data amount cleared in the clearing execution stage is avoided, the clearing database is not required to be independently erected outside the transaction database, the hardware cost is saved, and the delay of the main and standby synchronization of the database is reduced.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:
FIG. 1 shows a flow diagram of a transaction clearing method according to one embodiment of the application;
FIG. 2 is a schematic diagram of a prior art split TPS utilizing a single split stage to concentrate the split;
FIG. 3 illustrates a split TPS schematic diagram for splitting with multiple split stages according to one embodiment of the application;
FIG. 4 shows a schematic diagram of a transaction clearing device according to one embodiment of the application;
fig. 5 shows a schematic structural diagram of a database server according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be clearly and completely described below with reference to specific embodiments of the present application and corresponding drawings. It will be apparent that the described embodiments are only some, but not all, embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
To solve the problems presented in the background section, one embodiment is to add hardware resources. Specifically, in addition to storing the transaction data in the transaction database, a clear database is additionally arranged for storage, so that even if the TPS of the clear database is too high, the normal use of the transaction database is not affected, and the method is acceptable.
However, this clearly increases the hardware cost greatly, since to ensure the security of data, the database is usually a primary-backup architecture, i.e. one database has a plurality of backup databases for synchronous backup, and these backup databases may also be disposed in different cities to implement disaster recovery in different places.
The analysis finds that the database TPS is too high due to the clearing of the transaction data, mainly because the clearing is usually completed in a short time to avoid the overutilization of the database, for example, the clearing is usually performed in a short time (for example, 3 minutes) after the transaction time of 9 to 10 points 20, on the one hand, the clearing of the transaction data is performed in the transaction batch unit by taking the 9 to 10 points as a transaction batch, on the other hand, because of the characteristics of online transactions, if the transactions are unsuccessful, the clearing is required to be performed, and the clearing cannot be performed immediately after the transaction data is generated.
In addition, due to the reasons, the data volume of database synchronization is too large, so that the delay of database synchronization is higher in the sorting process, and the stability is affected.
In this regard, the technical concept of the present application is to avoid processing the clearing of the transaction data in one transaction batch with a centralized time period, but to implement with a plurality of time slices, thereby integrating the transaction data into zero and implementing peak clipping and valley filling for database utilization. Meanwhile, the data volume required to be cleared in each time slice is not large, so that an upper limit of the cleared transaction data volume can be set, and excessive occupation of a transaction database is avoided.
The following describes in detail the technical solutions provided by the embodiments of the present application with reference to the accompanying drawings.
Fig. 1 shows a flow diagram of a transaction scoring method according to one embodiment of the application, which may be performed by a database server, which may be a database server of a transaction database. On the other hand. Because the data volume of the transaction data is very large, a single transaction database may be difficult to cope with high concurrency, and a plurality of IDCs (INTERNET DATA CENTER, internet data centers) are often utilized in practice, a plurality of transaction databases are deployed in each IDC, and a database server performing the transaction sorting method of the present application may be a database server of a transaction database in one IDC.
The method for sorting transaction data in the embodiment of the application can sort the transaction data in one transaction batch by utilizing a plurality of sorting stages, and sorting of the transaction data in each sorting stage can be realized by referring to the flow shown in fig. 1. Each clearing period is understood to be a time slice, i.e. not to clear all transaction data within a transaction batch collectively over a short period of time.
The transaction batches may be divided according to transaction time, for example, 09 batches of transaction data generated between 8 points and 9 points. In other embodiments, the division of transaction batches may also be performed according to transaction serial numbers, etc.
As shown in fig. 1, the sorting method of transaction data executed in the sorting period includes:
step S110, determining transaction data to be cleared corresponding to the transaction batch in the transaction database.
It can be seen that the technical scheme of the application directly determines the transaction data to be cleared in the transaction database, and does not need to separately set a clearing database, thereby realizing the multiplexing of the transaction database and further saving the hardware cost and the operation and maintenance cost.
And step S120, sorting the transaction data to be sorted by utilizing a plurality of sorting execution periods in the sorting period, wherein the amount of the sorted transaction data in each sorting execution period is not more than a first threshold value.
In the clearing period, a plurality of clearing execution periods are used, transaction data which can be cleared in each time slice is limited through a preset first threshold, so that excessive utilization of a transaction database is avoided, and the transaction data can be cleared timely. Also because of this, the amount of data that needs to be synchronized at a time is not large, thereby improving the problem of synchronization delay.
It should be noted that, the clearing policy may be implemented according to the service requirement, for example, summarizing, classifying, etc., as long as the transaction content recorded by the transaction data can support the clearing policy.
The clearing execution period and the duration of the clearing period, and the first threshold may be set according to the requirement, for example, the clearing execution period is set to 1 second, the clearing period is set to 5 minutes, and the first threshold is set to 5200 pieces.
Therefore, in the method shown in fig. 1, the transaction data to be cleared in one transaction batch is processed by using a plurality of clearing stages instead of a single clearing stage, and the upper limit of the transaction data amount which can be cleared in each clearing execution stage is limited, so that the excessive high transaction data amount cleared in the clearing execution stage is avoided, a clearing database is not required to be independently erected outside a transaction database, the hardware cost is saved, and the delay of the database master-slave synchronization is reduced.
In one embodiment of the present application, the determining transaction data to be cleared corresponding to the transaction batch in the transaction database includes inquiring transaction data which is matched with the transaction time condition, in a final transaction state and not completed to be cleared as the transaction data to be cleared corresponding to the transaction batch from the transaction database according to the transaction time condition corresponding to the transaction batch.
After the transaction is formally completed, the transaction data is in a transaction ending state, and the transaction content is not changed, so that the transaction data can be cleared. However, a certain time is required from the generation of the transaction data to the final end of the transaction, and in order to ensure the accuracy of the clearing, the transaction time condition corresponding to the transaction batch may be used to limit, for example, the current time is 8 points 25 points, and only 09 batches of transaction data before 8 points 05 points are allowed to be cleared.
In one embodiment of the application, the method comprises the steps of inquiring transaction data which is matched with the transaction time condition, is in the final state of the transaction and is not cleared in the transaction time condition from a transaction database, wherein the transaction data to be cleared comprises the steps of determining the transaction data in the final state of the transaction according to the transaction state identification of the transaction data, and determining the transaction data which is not cleared in the transaction according to the clearing identification of the transaction data.
Based on the transaction time condition, the clearing identification, and the transaction status identification, a corresponding database operation statement may be generated to set the data route, from which it may be determined which are transaction data that require clearing.
In one embodiment of the present application, in the method, the sorting of the transaction data to be sorted respectively using the plurality of sorting execution periods in the sorting period includes sorting the transaction data to be sorted piece by piece in the sorting execution period, accumulating the amount of the transaction data sorted in the sorting execution period, and terminating sorting in the sorting execution period when the accumulated amount of the transaction data reaches a first threshold or all the transaction data to be sorted are sorted.
Specifically, the multiple clearing execution periods can share a variable for counting the transaction data volume cleared in the clearing execution period, and after clearing in the clearing execution period is ended, the value of the variable is reset to zero, so that the statistics of the next clearing execution period is facilitated.
In one embodiment of the application, the method further comprises determining, in batches, the clearing identification of the transaction data for which clearing has been completed during the clearing execution period.
After the transaction data is cleared, in order to avoid repeated clearing, the cleared transaction data needs to be marked, i.e., marked as cleared. In the prior art, the marking is often performed while sorting, which is also one reason for the TPS being too high. The application further proposes to separate the marking from the sorting, and mark the sorted transaction data in stages or batches. For example, after sorting 100 pieces of transaction data, the 100 pieces of transaction data are marked, and the sorting and marking of the 100 pieces of transaction data are integrally regarded as one database transaction. Through verification, the processing mode can ensure the clearing efficiency and simultaneously reduce the TPS of the database.
In some embodiments, there are a plurality of transaction databases, and the clearing results of the transaction databases need to be summarized, so that the corresponding clearing results can be reported during marking, and thus the clearing results of the table level are sequentially summarized into a library level, an IDC level, a transaction batch level, and the like.
In particular, in one embodiment of the present application, the method for determining the clearing identifier of the transaction data with the clearing completed in batches includes accumulating the transaction data amount with the clearing completed but not with the clearing identifier determined, determining the clearing identifier for the transaction data with the clearing completed but not with the clearing identifier in batches if the accumulated transaction data amount reaches a second threshold, and clearing the accumulated transaction data amount after determining the clearing identifier.
In one embodiment of the application, in the method, the maximum executable time length of each clearing execution period is the same, and clearing the transaction data to be cleared respectively by utilizing a plurality of clearing execution periods in the clearing execution period comprises clearing the transaction data to be cleared by utilizing a clearing thread in the clearing execution period, counting the total clearing time length, and if the total clearing time length is smaller than the maximum executable time length, determining the dormancy time length according to the total time length and the maximum execution time length, so that the clearing thread dormancy according to the dormancy time length.
The clearing of the present application may be performed by using a clearing thread, specifically, a transaction database may use a clearing thread, where the clearing thread clears transaction data in a plurality of clearing execution periods in a clearing period, so that, in order to achieve both robustness and utilization of computing resources, a setting may be made that a time required for clearing the first threshold number of transaction data is less than a maximum executable time of the clearing execution period, for example, the maximum executable time of the clearing execution period is 1 second, and a time for actually clearing 5200 transaction data generally does not exceed 500 milliseconds, so as to ensure that stable clearing can be achieved even if the database fluctuates, which may cause a waste of computing resources, and thus the clearing thread may be dormant. The sleep time period may be determined according to the method described above, and will not be described herein.
In one embodiment of the present application, in the method, a total duration of the plurality of sorting execution periods within a single sorting period is not greater than a third threshold.
For example, the clearing period is 5 minutes, and the total duration of a plurality of clearing execution periods in the clearing period is limited to 4 minutes, so that the situation that the clearing period is started at the point of 9, 25 and the point of 9, 30 and the next clearing period is started at the point of 9, 30 and the clearing period is started at the point of 9, 25 and 30 seconds due to certain faults can be avoided, the clearing period can still achieve the original target before the next clearing period is started, and the method has strong robustness. The setting of the third threshold may be determined according to actual requirements.
In one embodiment of the application, the method further comprises the steps of receiving a first clearing task, wherein the first clearing task corresponds to a transaction batch, determining the first execution time of the first clearing task according to the initial transaction time and the first delay time of the transaction batch, starting the first clearing period of the transaction batch according to the first execution time of the first clearing task, and executing the first clearing task in each clearing period respectively to realize clearing by utilizing a plurality of clearing periods.
In particular, the first scoring task may be generated and issued by a scoring server, which is responsible for execution. Taking the case of 8-9 as one transaction batch as an example, if the first delay time is 25 minutes and each clearing period is 5 minutes, the first clearing period is started separately at 8-25 and the second clearing period is started at 8-30.
In the scenario that a plurality of IDC deployment transaction databases are used, each IDC deployment transaction database is used, and the like, the first clearing task can be obtained by splitting the total clearing task one or more times. For example, the total clearing task can be configured or triggered by an operation and maintenance interface of the clearing server, the clearing server determines a transaction database used for clearing according to the total clearing task, further splits the total clearing task into sub clearing tasks at the IDC level according to the transaction database, and issues the sub clearing tasks to the corresponding IDCs, and further splits the sub clearing tasks into first clearing tasks at the database level at the IDCs. When in execution, the method of the previous embodiment can be referred to query and acquire the transaction data in the data table for subsequent clearing.
In some embodiments, the first sorting task is performed within a sorting period of 5 minutes by querying the transaction data to be sorted, each transaction database sorting the queried transaction data piece by piece using one sorting thread, and sorting every 100 transaction data pieces is considered as a database transaction.
The amount of the transaction data for clearing is accumulated for 1 second (the maximum executable time of the clearing execution period), and if the clearing TPS (first threshold) is reached or there is no transaction data to be cleared, clearing is stopped. If the transaction data which is not marked reaches the second threshold value, reporting and marking of the clearing result are started, and the method is analogically performed, under the condition that the accumulated clearing transaction data amount reaches the first threshold value, clearing is stopped, marking can be performed after clearing is completed but not marked, then the execution time is counted, the dormancy time is calculated, and the clearing thread is put into dormancy.
If the sorting period is cumulatively performed for 4 minutes, sorting is stopped, and the sorted but unlabeled label can be continued, and then the operation in the sorting period is ended.
In one embodiment of the application, the method further comprises the steps of receiving a second clearing task, wherein the second clearing task corresponds to the transaction batch, the execution time of the second clearing task is determined according to the initial transaction time and the second delay time of the transaction batch, and executing the second clearing task according to the execution time of the second clearing task to clear all the transaction data which are not cleared in the transaction batch.
The purpose of the second sorting task is mainly to avoid that the first sorting task performs abnormally or the amount of transaction data is excessive, and generally, the transaction data required to be sorted by the second sorting task is very small, and ideally, no data required to be sorted may be generated.
Fig. 2 is a schematic diagram of a clearing TPS using a single clearing period for centralized clearing in the prior art, and fig. 3 is a schematic diagram of a clearing TPS using multiple clearing periods according to an embodiment of the present application, with dark gray blocks representing the transaction data amount of clearing. It can be seen that the first clearing task is performed at points 8, 25 and 9, 20 and the second clearing task is performed at points 9, 20 and 23, with very little transaction data volume cleared by the second clearing task.
Because the second clearing task is mainly a spam process, a threshold may not be set to perform TPS control.
The application also provides a clearing device for the transaction data, which is used for realizing the clearing method for the transaction data.
Specifically, fig. 4 is a schematic structural diagram of a sorting device for transaction data according to an embodiment of the present application, and the sorting device 400 for transaction data may sort transaction data in one transaction batch using a plurality of sorting stages. As shown in fig. 4, the clearing device 400 for transaction data includes:
A determining unit 410, configured to determine transaction data to be cleared corresponding to the transaction batch in the transaction database in each clearing period.
And the clearing execution unit 420 is configured to, in each clearing period, respectively clear the transaction data to be cleared by using a plurality of clearing execution periods in the clearing period, where the amount of the transaction data cleared in each clearing execution period is not greater than a first threshold.
In an embodiment of the present application, in the above apparatus, the clearing execution unit 420 is configured to query, from the transaction database, transaction data that matches the transaction time and the transaction time, is in a final state of transaction, and has not completed clearing as the transaction data to be cleared, according to the preset transaction time condition.
In an embodiment of the present application, in the above apparatus, the clearing performing unit 420 is configured to determine transaction data in a final state of a transaction according to a transaction state identifier of the transaction data, and determine transaction data of incomplete clearing according to a clearing identifier of the transaction data.
In one embodiment of the present application, in the method, the sorting of the transaction data to be sorted respectively using the plurality of sorting execution periods includes sorting the transaction data to be sorted piece by piece in the sorting execution period, accumulating the amount of the transaction data sorted in the sorting execution period, and terminating sorting of the sorting execution period when the accumulated amount of the transaction data reaches a first threshold or all the transaction data to be sorted are sorted.
In an embodiment of the present application, in the foregoing apparatus, the clearing execution unit 420 is further configured to determine, in a batch, a clearing identifier of the transaction data that has completed clearing during a clearing execution period.
In an embodiment of the present application, in the foregoing apparatus, the clearing performing unit 420 is configured to accumulate the transaction data amount that has been cleared but has not determined the clearing identifier, determine the clearing identifier for the batch of the transaction data that has been cleared but has not determined the clearing identifier if the accumulated transaction data amount reaches the second threshold, and clear the accumulated transaction data amount after determining the clearing identifier.
In an embodiment of the present application, in the above device, the maximum executable duration of each clearing execution period is the same, and the clearing execution unit 420 is configured to clear the transaction data to be cleared by using the clearing thread in the clearing execution period, and count the total duration of clearing, and if the total duration is less than the maximum executable duration, determine the sleep duration according to the total duration and the maximum execution duration, so that the clearing thread sleeps according to the sleep duration.
In one embodiment of the present application, in the apparatus, a total duration of the plurality of sorting execution periods within a single sorting period is not greater than a third threshold.
In one embodiment of the present application, in the above apparatus, the determining unit 410 is configured to receive a first sorting task, where the first sorting task corresponds to a transaction batch, determine a first execution time of the first sorting task according to a start transaction time and a first delay time of the transaction batch, open a first sorting period of the transaction batch according to the first execution time of the first sorting task, and execute the first sorting task in each sorting period, so as to implement sorting by using a plurality of sorting periods.
In one embodiment of the present application, the determining unit 410 is further configured to receive a second sorting task, where the second sorting task corresponds to a transaction batch, and the execution time of the second sorting task is determined according to the initial transaction time and the second delay time of the transaction batch, and the sorting executing unit 420 is further configured to execute the second sorting task according to the execution time of the second sorting task, and sort all the transaction data of the transaction batch that are not completed with sorting.
It can be understood that the above-mentioned transaction data sorting device can implement the steps of the transaction data sorting method performed by the database server provided in the foregoing embodiment, and the relevant explanation about the transaction data sorting method is applicable to the transaction data sorting device, which is not repeated herein.
Fig. 5 is a schematic diagram of a database server according to an embodiment of the present application. Referring to fig. 5, at the hardware level, the database server includes a processor, and optionally an internal bus, a network interface, and a memory. The Memory may include a Memory, such as a Random-Access Memory (RAM), and may further include a non-volatile Memory (non-volatile Memory), such as at least 1 disk Memory. Of course, the database server may also include hardware required for other services.
The processor, network interface, and memory may be interconnected by an internal bus, which may be an ISA (Industry Standard Architecture ) bus, a PCI (PERIPHERAL COMPONENT INTERCONNECT, peripheral component interconnect standard) bus, or EISA (Extended Industry Standard Architecture ) bus, among others. The buses may be divided into address buses, data buses, control buses, etc. For ease of illustration, only one bi-directional arrow is shown in FIG. 5, but not only one bus or type of bus.
And the memory is used for storing programs. In particular, the program may include program code including computer-operating instructions. The memory may include memory and non-volatile storage and provide instructions and data to the processor.
The processor reads the corresponding computer program from the nonvolatile memory into the memory and then runs the computer program to form a clearing device of the transaction data on a logic level. The processor is used for executing the programs stored in the memory and is specifically used for executing the following operations:
Sorting by using a plurality of sorting stages;
and in each clearing period, determining the transaction data to be cleared in the transaction database, and respectively clearing the transaction data to be cleared by utilizing a plurality of clearing execution periods, wherein the transaction data amount cleared in each clearing execution period is not more than a first threshold value.
The method performed by the transaction data sorting device disclosed in the embodiment of fig. 1 of the present application may be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware in a processor or by instructions in the form of software. The Processor may be a general-purpose Processor including a central processing unit (Central Processing Unit, CPU), a network Processor (Network Processor, NP), etc., or may be a digital signal Processor (DIGITAL SIGNAL Processor, DSP), application SPECIFIC INTEGRATED Circuit (ASIC), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware components. The disclosed methods, steps, and logic blocks in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be embodied directly in the execution of a hardware decoding processor, or in the execution of a combination of hardware and software modules in a decoding processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory, and the processor reads the information in the memory and, in combination with its hardware, performs the steps of the above method.
The database server may also execute the method executed by the transaction data sorting device in fig. 1, and implement the functions of the transaction data sorting device in the embodiment shown in fig. 1, which are not described herein.
The embodiment of the present application also proposes a computer-readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by a database server including a plurality of application programs, enable the database server to perform a method for performing the sorting apparatus of transaction data in the embodiment shown in fig. 1, and specifically for performing:
Sorting by using a plurality of sorting stages;
and in each clearing period, determining the transaction data to be cleared in the transaction database, and respectively clearing the transaction data to be cleared by utilizing a plurality of clearing execution periods, wherein the transaction data amount cleared in each clearing execution period is not more than a first threshold value.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises an element.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and variations of the present application will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. which come within the spirit and principles of the application are to be included in the scope of the claims of the present application.
Claims (10)
1. A method for sorting transaction data, which is executed by a database server, wherein the method comprises sorting the transaction data in one transaction batch by using a plurality of sorting stages;
wherein, in each clearing period,
Determining transaction data to be cleared corresponding to the transaction batch in a transaction database;
respectively sorting the transaction data to be sorted by utilizing a plurality of sorting execution periods in the sorting period, wherein the amount of the transaction data sorted in each sorting execution period is not more than a first threshold value;
the utilizing the clearing execution periods to clear the transaction data to be cleared respectively includes:
Clearing transaction data to be cleared one by one in the clearing execution period, and accumulating the cleared transaction data amount in the clearing execution period; terminating the clearing of the clearing execution period when the accumulated transaction data quantity reaches a first threshold value or all the transaction data to be cleared is cleared;
the method further comprises the steps of:
determining the clearing identification of the transaction data with the cleared transaction data in batches in the clearing execution period;
The batch determination of the clearing identifier of the transaction data for which clearing is completed comprises:
Accumulating the transaction data quantity which is subjected to clearing and is not determined with clearing identification, determining clearing identification for the transaction data batch which is subjected to clearing and is not determined with clearing identification under the condition that the accumulated transaction data quantity reaches a second threshold value, and resetting the accumulated transaction data quantity after determining the clearing identification;
the determination of the clearing mark and clearing of the transaction data to be cleared are mutually independent processes.
2. The method of claim 1, wherein determining transaction data in a transaction database corresponding to the transaction batch to be cleared comprises:
And according to the transaction time conditions corresponding to the transaction batch, inquiring transaction data which is matched with the transaction time conditions, is in a final transaction state and is not cleared completely from the transaction database, and taking the transaction data as transaction data to be cleared corresponding to the transaction batch.
3. The method of claim 2, wherein the querying transaction data from the transaction database for transaction data having a transaction time that matches the transaction time condition, is in a final state of the transaction, and has not completed the clearing as transaction data to be cleared comprises:
and determining transaction data in a final transaction state according to the transaction state identification of the transaction data, and determining the transaction data with incomplete clearing according to the clearing identification of the transaction data.
4. The method of claim 1, wherein the maximum executable duration of each of the clearing execution periods is the same, and wherein the clearing the transaction data to be cleared using the plurality of clearing execution periods in the clearing period includes:
In the clearing execution period, clearing transaction data to be cleared by utilizing a clearing thread, and counting the total clearing time;
and if the total time length is smaller than the maximum executable time length, determining a sleep time length according to the total time length and the maximum executable time length, so that the clear thread sleeps according to the sleep time length.
5. The method of claim 1, wherein a total duration of a plurality of clearing execution periods within a single clearing period is not greater than a third threshold.
6. The method of any one of claims 1-5, wherein the method further comprises:
Receiving a first clearing task, wherein the first clearing task corresponds to the transaction batch, and the first execution time of the first clearing task is determined according to the initial transaction time and the first delay time of the transaction batch;
opening a first sorting period of the transaction batch according to the first execution time of the first sorting task, and respectively executing the first sorting task in each sorting period to realize sorting by utilizing a plurality of sorting periods.
7. The method of any one of claims 1-5, wherein the method further comprises:
receiving a second clearing task, wherein the second clearing task corresponds to the transaction batch, and the execution time of the second clearing task is determined according to the initial transaction time and the second delay time of the transaction batch;
And executing the second clearing task according to the execution time of the second clearing task, and clearing all the transaction data of the transaction batch, which are not cleared completely.
8. A transaction data sorting device applied to a database server, wherein the device is used for implementing the method of any one of claims 1-7.
9. A database server, comprising:
Processor, and
A memory arranged to store computer executable instructions which, when executed, cause the processor to perform the method of any of claims 1 to 7.
10. A computer readable storage medium storing one or more programs, which when executed by a database server comprising a plurality of application programs, cause the database server to perform the method of any of claims 1-7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011214451.7A CN114445220B (en) | 2020-11-04 | 2020-11-04 | Transaction data sorting method and device, database server and storage medium |
| PCT/CN2021/111006 WO2022095525A1 (en) | 2020-11-04 | 2021-08-05 | Method and apparatus for clearing transaction data, database server, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011214451.7A CN114445220B (en) | 2020-11-04 | 2020-11-04 | Transaction data sorting method and device, database server and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114445220A CN114445220A (en) | 2022-05-06 |
| CN114445220B true CN114445220B (en) | 2025-07-18 |
Family
ID=81361486
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011214451.7A Active CN114445220B (en) | 2020-11-04 | 2020-11-04 | Transaction data sorting method and device, database server and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN114445220B (en) |
| WO (1) | WO2022095525A1 (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115048369A (en) * | 2022-06-29 | 2022-09-13 | 中信百信银行股份有限公司 | Memory computing-based streaming data processing method and system |
| CN115617403A (en) * | 2022-12-19 | 2023-01-17 | 深圳华锐分布式技术股份有限公司 | Clearing task execution method, device, equipment and medium based on task segmentation |
| CN119539948B (en) * | 2024-11-13 | 2025-09-30 | 天翼支付科技有限公司 | Batch file processing method and device, storage medium and electronic device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107025236A (en) * | 2016-02-01 | 2017-08-08 | 阿里巴巴集团控股有限公司 | Data processing method and data system for settling account between system for settling account |
| CN109726183A (en) * | 2018-09-07 | 2019-05-07 | 网联清算有限公司 | Net connection platform liquidation method and device |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9898717B2 (en) * | 2013-03-25 | 2018-02-20 | Paypal, Inc. | Online remittance system with methodology for predicting disbursement times of online electronic funds transfers |
| US10311516B2 (en) * | 2015-06-10 | 2019-06-04 | Chicago Mercantile Exchange Inc. | Transaction processing system performance evaluation |
| CN107203875B (en) * | 2017-04-01 | 2021-07-20 | 网联清算有限公司 | Capital clearing method, device and server |
| CN111105314A (en) * | 2018-10-29 | 2020-05-05 | 中国人民财产保险股份有限公司 | Insurance data clearing system |
-
2020
- 2020-11-04 CN CN202011214451.7A patent/CN114445220B/en active Active
-
2021
- 2021-08-05 WO PCT/CN2021/111006 patent/WO2022095525A1/en not_active Ceased
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107025236A (en) * | 2016-02-01 | 2017-08-08 | 阿里巴巴集团控股有限公司 | Data processing method and data system for settling account between system for settling account |
| CN109726183A (en) * | 2018-09-07 | 2019-05-07 | 网联清算有限公司 | Net connection platform liquidation method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2022095525A1 (en) | 2022-05-12 |
| CN114445220A (en) | 2022-05-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114445220B (en) | Transaction data sorting method and device, database server and storage medium | |
| CN107450979B (en) | A blockchain consensus method and device | |
| CN110674213A (en) | Data synchronization method and device | |
| CN110221914B (en) | File processing method and device | |
| CN109800204B (en) | Data distribution method and related product | |
| CN108492150B (en) | Method and system for determining entity heat degree | |
| CN113204464A (en) | Real-time service monitoring method, system, terminal and medium based on service scene | |
| CN111651595A (en) | Abnormal log processing method and device | |
| CN111611207B (en) | State data processing method and device and computer equipment | |
| CN110222016B (en) | File processing method and device | |
| CN111522876B (en) | Block chain consensus method, device, computer equipment and block chain link point | |
| CN109947713B (en) | Log monitoring method and device | |
| CN111694816B (en) | Processing method and device for optimizing database table | |
| CN113923268A (en) | Analysis method, equipment and storage medium for multi-version communication protocol | |
| CN109246216B (en) | Access request processing method and device | |
| CN116703467A (en) | User data monitoring method, system, electronic device and readable storage medium | |
| CN108519909B (en) | Stream data processing method and device | |
| CN113225218A (en) | Method and device for checking call ticket quality | |
| CN110334909B (en) | Risk management and control method, device and equipment | |
| CN111885159B (en) | Data acquisition method and device, electronic equipment and storage medium | |
| CN115564576A (en) | Method and device for identifying risks of service request, storage medium and processor | |
| CN111314440B (en) | Method and system for loop detection on graph-structured data | |
| CN113010495A (en) | Database optimization method and device | |
| CN114625595B (en) | Method, device and system for rechecking dynamic configuration information of service system | |
| CN111061712A (en) | Data connection operation processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |