CN118296044A - Data processing method, device, equipment and medium based on high concurrency point - Google Patents

Abstract

The present invention relates to a method, device, equipment and medium for processing data based on high concurrency hotspots, and belongs to the technical field of data processing. The method includes: obtaining a business request, wherein the business request includes a business type, user information, a request account identifier and amount information corresponding to the business type; verifying the business request, and after the business request verification passes, storing the request flow of the business request in a system database; after the request flow table is successfully stored, based on the business type, the request account identifier, the amount information corresponding to the business type and a preset cache data processing rule, performing cache update processing on the account information in the cache database; after the cache update processing is successful, storing the cache operation voucher corresponding to the business request in the system database. The present application has the effect of improving the response speed and concurrent processing capability of high concurrency hotspot data.

Description

Method, device, equipment and medium for processing data based on high concurrency hotspot

Technical Field

The present invention relates to the field of data processing technology, and in particular to a method, device, equipment and medium for processing data based on high-concurrency hot spots.

Background technique

With the popularization of electronic payment and the increase in transaction volume, especially in high-frequency transaction scenarios such as public accounts or merchant payment accounts, a large number of concurrent requests are often concentrated on a hotspot account. In this scenario, there will be frequent entry and exit, and the pressure on data operations will also increase, and the problem of hotspot data will also be faced.

At present, relying on the lock mechanism of the database has a certain effect in solving concurrency problems. Through the lock rules, MySQL ensures the integrity and consistency of the data, guarantees the security of the data, and improves the stability of the system.

However, when faced with extremely high concurrent requests, lock contention and lock waiting can lead to increased system overhead, degraded performance, and long user waiting times, making it impossible to meet business needs by relying solely on database lock mechanisms.

Summary of the invention

The technical problem to be solved by the present invention is to improve the response speed and concurrent processing capability of high-concurrency hot spot data in payment scenarios, high-frequency transaction scenarios, etc.

In a first aspect, the present application provides a method for processing high-concurrency hotspot data, which adopts the following technical solutions:

A method for processing high-concurrency hotspot data, comprising:

Obtaining a business request for the order to be traded, the business request including a request serial number, a business type, user information, a request account identifier, and amount information corresponding to the business type, the business type being any one of deposit, withdrawal, freeze, unfreeze, batch deposit, batch withdrawal, batch freeze, and batch unfreeze;

Performing parameter legitimacy check on the user information of the service request and performing serial number anti-duplicate check on the request serial number. After the service request passes the verification, the request serial number of the service request is stored in the system database. The request serial number is used to record the request time, service type, user information and request account identifier of the service request;

After the request flow is successfully stored, based on the business type, the request account identifier, the amount information corresponding to the business type and the preset cache data processing rules, the account information in the cache database is cached and updated;

After the cache update process is successful, the cache operation credentials corresponding to the business request are stored in the system database, and the operation success result is returned to the user corresponding to the business request.

The beneficial effects of the present invention are as follows: by introducing a cache database for cache processing, the direct read and write operations on the system database can be significantly reduced, thereby reducing the burden on the database. When processing business requests, the integrity and consistency of the data are ensured through steps such as parameter legitimacy verification, flow anti-duplicate verification, request flow storage, and cache operation credential storage. Since cache processing can increase the data processing speed, business requests initiated by users can get faster responses, which improves the response speed and concurrent processing capability of high-concurrency hotspot data.

Based on the above technical solution, the present invention can also be improved as follows.

Further, the cache update processing of the account information in the cache database based on the business type, the requesting account identifier, the amount information corresponding to the business type and the preset cache data processing rules includes:

Performing an account query in a cache database based on the request account identifier;

If the account information of the requested account identifier is found in the cache database, then based on the business type and the amount information corresponding to the business type, the found account information is verified for funds;

After the queried account information passes the fund verification, performing signature verification on the cache database;

After the signature verification is passed in the cache database, the account balance is calculated based on the business type, the amount information corresponding to the business type and the account information, and the cache data of the account information is modified in the cache database according to the account balance to obtain new account information;

An operation flow is generated based on the new account information, operation time, business type, and amount information corresponding to the business type, and the operation flow of the business request is stored in a cache database.

The beneficial effects of adopting the above further scheme are: by directly performing account queries and fund verification in the cache database, frequent access to the system database is avoided. During the cache processing, the integrity and authenticity of the data are ensured through the signature verification mechanism to prevent the data from being tampered with or forged; the account balance is calculated in real time according to the business type and amount information, and the cache data of the account information is updated in the cache database, ensuring the real-time and accuracy of the account information; by generating and storing operation flow to the cache database, it can not only facilitate subsequent audits and queries, but also further optimize the use of the cache.

Further, the cache update processing of the account information in the cache database based on the business type, the requesting account identifier, the amount information corresponding to the business type and the preset cache data processing rules also includes:

If the account information of the requesting account identifier is not found in the cache database, an asynchronous thread is started, where the asynchronous thread includes a first thread and a second thread;

The first thread is used to perform a cache reconstruction step, wherein the cache reconstruction includes: sending a cache reconstruction request to a cache reconstruction server, wherein the cache reconstruction request includes a request account identifier, so that the cache reconstruction server performs cache reconstruction based on account information corresponding to the request account identifier;

When the first thread executes the cache rebuilding step, the second thread starts to wait for a preset time, and after waiting for the preset time, performs an account query in the cache database based on the request account identifier;

If the query fails, the step of waiting for a preset time is repeated. If the query fails after the preset number of times is reached, the operation failure result is returned to the user corresponding to the business request.

The beneficial effect of adopting the above further scheme is: when the account information is not queried in the cache database, by starting an asynchronous thread, the first thread sends a cache reconstruction request, and the second thread continues to process other business requests, avoiding the blocking of the main thread, thereby improving the overall processing efficiency of the system; by sending a cache reconstruction request to the cache reconstruction server, the cache reconstruction server uses a distributed lock mechanism to reconstruct the cache of the account information to ensure the integrity and consistency of the data. The account information after cache reconstruction is stored in the cache database, providing an accurate data basis for subsequent business processing. After the main thread waits for a preset time, the account query is performed again in the cache database based on the request account identifier. If the query fails, continue to wait and query until the preset number of times is reached. This retry mechanism can deal with temporary problems such as network delays and cache synchronization delays, and improve the reliability and fault tolerance of the system.

Further, the request flow includes a request record status, and after the cache operation credentials corresponding to the account information are saved in the system database, it also includes:

The request record status of the request pipeline is recorded as successful processing.

The beneficial effect of adopting the above further solution is that the request record status is recorded as successfully processed, which is helpful for subsequent business tracking and auditing. The system can trace back and analyze the business processing process according to the status record of the request flow to find potential problems or anomalies.

Furthermore, after the cache processing fails, the method further includes:

Based on preset timed tasks, abnormal content detection is performed on business requests that fail to be processed by the cache. The abnormal content includes network interruption, database connection failure, operation failure, and timeout. The operation failure includes at least one of insufficient balance, status stop, and account non-existence.

The beneficial effect of adopting the above further scheme is: by using the scheduled task to detect abnormal content of the business request that failed to be processed by the cache, it is possible to timely discover abnormal situations such as network interruption, database connection failure, operation failure, etc., so that the system can respond quickly and take corresponding processing measures to avoid further spread and impact of abnormal situations. Scheduled tasks can automatically perform abnormal content detection without manual intervention. This greatly reduces the workload of system maintenance personnel and improves maintenance efficiency.

Further, the request flow includes a request record state, and after performing abnormal content detection on the service request that fails in cache processing based on the preset timed task, it also includes:

Obtain the abnormal content type of cache processing failure corresponding to the business request;

Determine whether the abnormal content type of the failure of the business request cache processing is any one of the insufficient balance type, the status stop type, and the account does not exist type;

If yes, the request record status of the request flow corresponding to the business request is recorded as processing failure;

If not, the request record status of the request flow corresponding to the business request is recorded as being processed.

The beneficial effect of adopting the above further scheme is that by determining the type of abnormal content that failed to be processed by the cache, the system can accurately record the request flow of the business request as a processing failure or processing status. For specific types of abnormal content such as insufficient balance, status stop and account non-existence, recording the request flow as a processing failure status can avoid invalid subsequent operations and save system resources. At the same time, for other types of abnormal content, recording the request flow as a processing status can reserve a certain processing space, waiting for further processing or recovery by the system.

Further, the cache operation credential includes a processing status and a credential creation time, and the method further includes:

Based on a first preset time interval, querying the system database for a cache operation credential that meets a preset condition, wherein the preset condition is that the processing status of the cache operation credential is unprocessed, and the storage time of the cache operation credential is the cache operation credential within a preset time, wherein the preset time is calculated based on the current time and a second preset time interval;

If a cache operation credential that meets the preset conditions is found in the system database, each cache operation credential is divided according to the requesting account of the cache operation credential, and at least one cache operation credential corresponding to each requesting account is processed in parallel using multiple threads;

For each thread, obtaining account information in a system database based on the request account identifier;

For each thread, based on a preset balance calculation formula, the account information and at least one cache operation voucher corresponding to the requesting account, the amount of the requesting account identifier is aggregated to obtain the balance information of the requesting account;

For each thread, based on the account balance, modify the account information corresponding to the requesting account identifier in the system database;

For each thread, after the account information corresponding to the request account identifier is modified successfully, the status of at least one cache operation credential corresponding to the request account identifier is modified to be processed successfully, and the change flow corresponding to the request account is stored in the system database.

The beneficial effect of adopting the above further scheme is: by querying the unprocessed cache operation vouchers based on the preset time interval, and dividing the parallel processing by the request account, the system resources can be fully utilized, the waiting time can be reduced, and the data processing efficiency can be significantly improved. In a high concurrency scenario, this parallel processing method can greatly improve the throughput and response speed of the system. The amount of the cache operation vouchers for each request account is summarized, and the account information in the system database is modified based on the summary results, ensuring the accuracy and consistency of the data. At the same time, the status of the cache operation voucher is modified to successful processing, and the change flow is stored in the system database, providing a complete data record for subsequent auditing and tracking. By processing cache operation vouchers in parallel through multiple threads, the computing power of multi-core CPUs can be fully utilized to avoid waste of resources.

In a second aspect, the present application provides a high-concurrency hotspot data processing device, which adopts the following technical solution:

A high-concurrency hotspot data processing device, comprising:

an acquisition module, used to acquire a business request for a pending transaction order, wherein the business request includes a request serial number, a business type, user information, a request account identifier, and amount information corresponding to the business type, wherein the business type is any one of: deposit, withdrawal, freeze, unfreeze, batch deposit, batch withdrawal, batch freeze, and batch unfreeze;

A parameter verification module is used to perform parameter legitimacy verification on the user information of the service request and to perform serial number anti-duplicate verification on the request serial number. After the service request passes the verification, the request serial number of the service request is stored in the system database. The request serial number is used to record the request time, service type, user information and request account identifier of the service request;

A cache processing module, configured to perform cache update processing on the account information in the cache database based on the business type, the requesting account identifier, the amount information corresponding to the business type and a preset cache data processing rule after the request flow is successfully stored;

The storage module is used to store the cache operation credentials corresponding to the business request in the system database after the cache processing is successful, and return the operation success result to the user corresponding to the business request.

The beneficial effects of the present invention are as follows: by introducing a cache database for cache processing, the direct read and write operations on the system database can be significantly reduced, thereby reducing the burden on the database, and when processing business requests, the integrity and consistency of the data are ensured through steps such as parameter verification, request flow storage, and cache operation credential storage. Since cache processing can increase the data processing speed, business requests initiated by users can receive faster responses, thereby improving the response speed and concurrent processing capability of high-concurrency hotspot data.

In a third aspect, the present application provides an electronic device, which adopts the following technical solution:

An electronic device comprises a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executes the high-concurrency hot spot data processing method according to any one of the first aspects.

In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution:

A computer-readable storage medium stores a computer program that can be loaded by a processor and execute the high-concurrency hot spot data processing method according to any one of the first aspects.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic flow chart of a method for processing high-concurrency hotspot data according to the present invention;

FIG. 2 is a schematic diagram of a process flow of an electronic device of the present invention initiating a cache reconstruction process to a cache reconstruction server.

FIG3 is a schematic diagram of a process of cache entry by an electronic device of the present invention;

FIG4 is a structural diagram of a high-concurrency hotspot data processing device according to the present invention;

FIG5 is a structural block diagram of an electronic device of the present invention.

Detailed ways

The present application is further described in detail below in conjunction with the accompanying drawings.

In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

An embodiment of the present application provides a high-concurrency hot spot data processing method, which can be executed by an electronic device, and the electronic device can be a server or a mobile terminal device, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services; the mobile terminal device can be a laptop computer, a desktop computer, etc., but is not limited to this.

The following is a further detailed description of the embodiments of the present application in conjunction with the accompanying drawings. As shown in FIG1 , a method for processing high-concurrency hotspot data includes steps S1 to S4:

Step S1, obtaining a business request for a transaction scenario, wherein the business request includes a request serial number, a business type, user information, a request account identifier, and amount information corresponding to the business type, wherein the business type is any one of deposit, withdrawal, freeze, unfreeze, batch deposit, batch withdrawal, batch freeze, and batch unfreeze;

In the implementation mode of the present application, the electronic device obtains at least one business request, which is accessed by the user through a client or other system, and the electronic device stores the obtained at least one business request in the request flow table of the system database according to the time of the business request, and then the electronic device obtains the business request in sequence according to the order of multiple business requests in the request flow table to process the request. Among them, the user information includes user ID, user name, etc., the request account identifier is the account information involved in the operation, such as account ID, the amount information corresponding to the business type includes transfer amount, frozen amount, etc., and the request serial number is a number that uniquely identifies a specific business request.

Step S2, performing parameter legitimacy check on the user information of the service request and performing serial number anti-duplicate check on the request. After the service request passes the check, the request serial number of the service request is stored in the system database. The request serial number is used to record the request time, service type, user information and request account identifier of the service request.

In an implementation manner of the present application, the electronic device preprocesses the received business request, and the preprocessing includes parameter legitimacy verification (such as checking the validity of user information, the correctness of the business type, the rationality of the amount information, etc.), request flow anti-duplicate verification, and formatting or standardizing the request information for subsequent processing.

If the business request does not pass the parameter verification, the request will continue to be processed. If the verification fails, the user's client or other system response failure is returned and the request processing is terminated.

It should be noted that the parameter validity check includes: the system confirms whether the business type belongs to the scope supported by the system (such as deposit, withdrawal, freeze, unfreeze, etc.), and checks whether there are illegal or undefined business types; verifies the integrity and validity of user information, including whether the user name, user ID, etc. conform to the format specified by the system, and whether they exist in the system user list; checks whether the requested account identifier is legal, such as whether the account exists, whether the account status is normal, etc.; verifies whether the amount information is legal, such as whether the amount is greater than 0, whether it exceeds the preset maximum amount limit, etc. At the same time, for batch processing type business requests, it is also necessary to verify whether the format and content of the batch data are correct; and verify other specific parameters, such as timestamps, signatures, etc.

The request serial number anti-duplicate check includes: the system generates a unique request serial number for the business request, which is usually generated based on certain rules, such as timestamp, random number, etc., to ensure its global uniqueness. The generated request serial number is stored in the system database as the unique identifier of the business request. At the same time, the request information related to the serial number is recorded. When a new business request is received, the system database is first queried to check whether the request serial number already exists. If the query result is empty, it means that the request serial number does not exist, that is, the request is new and can continue to be processed.

If the query result is not empty, it means that the request serial number already exists, that is, the request is repeated. At this time, the user's client or other system responds to failure and terminates the request processing to avoid repeated processing of the same request.

When the system attempts to store the request flow into the system database, it performs an INSERT operation to insert the relevant data of the business request (such as the request flow number, business type, request account ID, amount information, etc.) into the request flow table. After the database operation is completed, the system returns an operation result. This result is usually a status code or Boolean value to indicate whether the operation is successful.

If the system storage operation is successful, step S3 is executed. If the system storage operation is unsuccessful, a business response failure result is returned.

Step S3, after the request flow is successfully stored, based on the business type, the request account identifier, the amount information corresponding to the business type and the preset cache data processing rules, the account information in the cache database is cached and updated;

In the implementation manner of the present application, the electronic device obtains cache information of the requesting account in the cache database based on the business type, the requesting account identifier, the amount information corresponding to the business type and the preset cache data processing rules. In the present application, the cache database is a redis database.

Based on the business type, the requesting account identifier, the amount information corresponding to the business type and the preset cache data processing rules, obtaining the cache information of the requesting account in the cache database includes: performing a rides operation according to the Lua script, if the Lua script is executed successfully, saving the redis operation success flow, if the Lua script fails to execute, the script returns the redis operation result, the failure reason description, and the cache operation timestamp.

The script includes the following operations: Step 1, declare the redis result return object operation; Step 2, query the request account identifier operation; Step 3, verify whether the request account identifier exists; Step 4, account data parsing operation; Step 5, fund verification operation; Step 6, signature verification; Step 7, balance calculation; Step 8, obtain redis operation time; Step 9, account data assembly; Step 10, account modification; Step 77, save redis transaction; Step 12, operation result return.

Specifically, step S3 includes the following sub-steps:

Step S31, performing an account query in a cache database based on the request account identifier;

In the implementation mode of the present application, the system uses the request account identification information as a query condition to query in the cache database. The cache database is usually a memory database or a high-speed cache system for quickly retrieving account information.

If the query is successful, the account information related to the requested account ID is obtained, including account balance, account status, account restrictions, etc.

Step S32, if the account information of the requesting account identifier is found in the cache database, then based on the business type and the amount information corresponding to the business type, the found account information is subjected to fund verification;

In the implementation mode of the present application, the system performs fund verification according to the queried account information and the amount information corresponding to the business type according to the preset definition rules. The preset definition rules include checking whether the account balance is sufficient, whether it exceeds the preset limit, etc.

If the funds verification is passed, the subsequent steps will be continued; if the verification fails (for example, the account balance is insufficient), an error message will be returned and the processing will be terminated.

Step S33, after the queried account information passes the fund verification, the cache database is signature verified;

Step S34, after the signature verification is passed in the cache database, the account balance is calculated based on the business type, the amount information corresponding to the business type and the account information, and the cache data of the account information is modified in the cache database according to the account balance to obtain new account information;

Step S35, generating an operation flow based on the new account information, operation time, business type, and amount information corresponding to the business type, and storing the operation flow of the business request in a cache database.

Step S36, if the account information of the requesting account identifier is not found in the cache database, an asynchronous thread is started, wherein the asynchronous thread includes a first thread and a second thread;

The first thread is used to perform a cache reconstruction step, wherein the cache reconstruction includes: sending a cache reconstruction request to a cache reconstruction server, wherein the cache reconstruction request includes a request account identifier, so that the cache reconstruction server performs cache reconstruction based on account information corresponding to the request account identifier, and stores the cache account information after cache reconstruction in a cache database;

In the implementation mode of the present application, if the Lua script operation returns that the account does not exist, the cache is rebuilt, and if the reconstruction is successful, the script logic is re-executed. If the reconstruction fails, the business side failure result is returned.

Step S37, when the first thread executes the cache rebuilding step, the second thread starts to wait for a preset time, and after waiting for the preset time, performs an account query in the cache database based on the request account identifier;

Step S38, if the query fails, the step of waiting for a preset time is repeated. If the query fails after the number of queries reaches the preset number, an operation failure result is returned to the user corresponding to the business request.

Step S4, after the cache processing is successful, the cache operation credentials corresponding to the business request are stored in the system database, and the operation success result is returned to the user corresponding to the business request.

In the implementation mode of the present application, the script is executed successfully and a cache operation voucher is saved. The cache operation voucher includes a voucher serial number and a voucher creation time, wherein the voucher serial number is the business request serial number; and the voucher creation time is the script execution time. After the cache operation voucher is saved successfully, the redis operation success serial is deleted from the cache database.

FIG. 2 is a schematic diagram showing a process flow of an electronic device initiating a cache reconstruction process to a cache reconstruction server.

The following description of step S36 is given in conjunction with Figures 1 and 2. As shown in Figures 1 and 2, when the system fails to find the account information of the requested account identifier in the cache database, an asynchronous thread is triggered. The asynchronous thread is responsible for constructing a cache reconstruction request and sending the cache reconstruction request to the cache reconstruction server. The cache reconstruction request includes the requested account identifier.

The first thread executes the cache reconstruction step. The cache reconstruction server receives the cache reconstruction request and obtains a distributed lock. The distributed lock is used to ensure that during the cache reconstruction process, no other operations will modify the data of the account at the same time, thereby preventing data processing conflicts.

The cache reconstruction server queries the system database and cache database for abnormal data by comparing the data consistency between the database and the cache, checking the data status or log, etc. If abnormal data is found, the cache reconstruction server handles it according to the preset data error processing process, such as cleaning up abnormal data and recording error logs.

The cache service area queries the delayed balance information of the account and the unprocessed voucher information in the cache from the system database, adds the queried delayed balance information and the cached unprocessed voucher information to obtain the actual amount of the account. After that, new cache account information is generated based on the actual amount, the account information of the requesting account identifier, the operation time, etc., and saved in the cache database.

It should be noted that the actual amount in the account = cached amount = unprocessed voucher amount + database delayed amount.

The second thread adopts a blind strategy to wait for cache reconstruction. The system requests the second thread to enter a waiting state. The waiting time or query number is recorded according to a timer or counter. During the waiting time, the second thread will periodically try to query the cached account information. If the query result is empty, the second thread will continue to wait and repeat the query operation. When the query number reaches the preset maximum waiting number count, the second thread will stop waiting and respond to the failure of the business request. If the query successfully obtains the account information of the requested account identifier, the second thread exits the waiting state and continues the subsequent cache processing operation of the cached account information.

By sending a cache reconstruction request to the cache reconstruction server, the cache reconstruction server uses the distributed lock mechanism to rebuild the cache of the account information to ensure the integrity and consistency of the data. The account information after cache reconstruction is stored in the cache database, providing an accurate data basis for subsequent business processing. After the main thread waits for a preset time, it performs an account query in the cache database again based on the requested account identifier. If the query fails, it continues to wait and query until the preset number of times is reached. This retry mechanism can cope with temporary problems such as network delays and cache synchronization delays, improving the reliability and fault tolerance of the system.

As another optional implementation of the implementation of the present application, after the cache processing fails, the method further includes:

Based on preset timed tasks, abnormal content detection is performed on business requests that fail to be processed by the cache. The abnormal content includes network interruption, database connection failure, operation failure, and timeout. The operation failure includes at least one of insufficient balance, status stop, and account non-existence.

In one implementation, if the Lua script operation is abnormal, the abnormal content type of the cache processing failure corresponding to the business request is obtained; it is determined whether the abnormal content type of the cache processing failure of the business request is any one of the insufficient balance type, the status stop type, and the account does not exist type;

If yes, the request record status of the request flow corresponding to the business request is recorded as processing failure; if no, the request record status of the request flow corresponding to the business request is recorded as processing. And respond to the business party with an intermediate status return code such as "processing". The business party can query the final processing result through the request query interface.

Since redis and MySQL cannot guarantee atomicity directly, data cannot be rolled back in case of abnormal interruption. We rely on the timing scheme to handle abnormal data intermittently. Error handling is based on the redis operation result (whether the redis operation success flow exists). If the redis operation is successful, the final status of this request is success; if the redis execution fails, the final status of this request is considered failure.

FIG. 3 is a schematic diagram showing a process flow of cache entry by an electronic device.

As shown in FIG3 , as another optional implementation of the implementation of the present application, the method further includes:

Step S41, the electronic device searches the system database for a cache operation credential that meets a preset condition based on a first preset time interval, the cache operation credential including a processing status and a credential creation time, the preset condition being that the processing status of the cache operation credential is unprocessed, and the storage time of the cache operation credential is the cache operation credential within a preset time, and the preset time is calculated based on the current time and a second preset time interval;

Step S42, if a cache operation credential meeting the preset condition is found in the system database, each cache operation credential is divided according to the requesting account of the cache operation credential, and at least one cache operation credential corresponding to each requesting account is processed in parallel using multiple threads;

Step S43, for each thread, obtaining account information from a system database based on the request account identifier;

Step S44, for each thread, based on a preset balance calculation formula, the account information and at least one cache operation credential corresponding to the requesting account, the amount of the requesting account identifier is aggregated to obtain the balance information of the requesting account;

Step S45, for each thread, based on the account balance, modifying the account information corresponding to the request account identifier in the system database;

Step S46, for each thread, after the account information corresponding to the request account identifier is successfully modified, the status of at least one cache operation credential corresponding to the request account identifier is modified to successful processing, and the change flow corresponding to the request account is stored in the system database, and the change flow is a record of changes in the account balance.

In the implementation mode of the present application, the system triggers a summary batch task every five minutes. Each time the batch task is triggered, the current time is obtained, and the time point two minutes before is calculated as the upper limit of the time range for querying the voucher. The record of the operation voucher in the cache is queried, and the condition is that the processing status is "unprocessed" and the operation time is within two minutes before the current time.

If no voucher record is found, the batch task execution ends and waits for the next scheduled trigger. If cached operation vouchers that meet the preset conditions are found, the queried voucher records are grouped according to the account number, and a separate thread is created for each account (or a thread pool is used) to process cached operation vouchers of different accounts in parallel. Each thread is responsible for processing the voucher records of the corresponding account, including database query, amount aggregation, and account balance modification.

Afterwards, the thread queries the database to obtain the account information of the corresponding account, summarizes the amount according to the voucher record, calculates the account balance amount that needs to be adjusted, and updates the entity account balance information in the system to ensure that multiple redis operation vouchers only update the account balance once to reduce the pressure on database operations.

After the system database account balance is modified successfully, a distributed lock is used to protect the next operation to prevent concurrent conflicts. The processing status of the corresponding voucher in the cache is changed to "successful", indicating that the voucher has been processed.

When modifying the account balance or after modifying the account balance, save the change record of the account funds. The change record contains information such as the account balance before the change, the change amount, and the account balance after the change. Use distributed locks for protection to ensure data consistency.

If an error or exception occurs in the cache entry process, such as database operation failure, cache update failure, etc., the transaction rollback mechanism is triggered and the error log is recorded. Transaction rollback ensures that all modified data is restored to the state before the operation to maintain data consistency. Error logs are used for subsequent troubleshooting and repair.

This method can significantly reduce the direct read and write operations on the system database by introducing a cache database for cache processing, thereby reducing the burden on the database. When processing business requests, the integrity and consistency of the data are ensured through steps such as parameter verification, request flow storage, and cache operation credential storage. Since cache processing can increase the data processing speed, business requests initiated by users can get faster responses, which improves the response speed and concurrent processing capability of high-concurrency hotspot data.

FIG. 4 shows a structural block diagram of a high-concurrency hotspot data processing device 200 .

As shown in FIG4 , a high-concurrency hotspot data processing device 200 mainly includes:

The acquisition module 201 is used to acquire a business request for the pending transaction order, wherein the business request includes a request serial number, a business type, user information, a request account identifier, and amount information corresponding to the business type, wherein the business type is any one of: deposit, withdrawal, freeze, unfreeze, batch deposit, batch withdrawal, batch freeze, and batch unfreeze;

The parameter verification module 202 is used to perform parameter validity verification on the user information of the service request and to perform serial number anti-duplicate verification on the request serial number. After the service request passes the verification, the request serial number of the service request is stored in the system database. The request serial number is used to record the request time, service type, user information and request account identifier of the service request;

The cache processing module 203 is used to perform cache update processing on the account information in the cache database based on the business type, the request account identifier, the amount information corresponding to the business type and the preset cache data processing rules after the request flow is successfully stored;

The storage module 204 is used to store the cache operation credentials corresponding to the business request in the system database after the cache update process is successful, and return the operation success result to the user corresponding to the business request.

Optionally, the cache processing module 203 includes:

A first query submodule, configured to perform an account query in a cache database based on the request account identifier;

A fund verification submodule, configured to perform fund verification on the queried account information based on the business type and the amount information corresponding to the business type if the account information of the request account identifier is queried in the cache database;

A signature verification submodule, used to perform signature verification on the cache database after the queried account information passes the fund verification;

a balance calculation submodule, configured to calculate the account balance based on the business type, the amount information corresponding to the business type and the account information after the signature verification is passed in the cache database, and to modify the cache data of the account information in the cache database according to the account balance to obtain new account information;

The entry voucher storage submodule is used to generate an operation flow based on the new account information, operation time, business type, and amount information corresponding to the business type, and store the operation flow of the business request in a cache database.

Optionally, the cache processing module 203 further includes:

a cache reconstruction submodule, configured to start an asynchronous thread if the account information of the requesting account identifier is not found in the cache database, the asynchronous thread comprising a first thread and a second thread;

The first thread is used to perform a cache reconstruction step, wherein the cache reconstruction includes: sending a cache reconstruction request to a cache reconstruction server, wherein the cache reconstruction request includes a request account identifier, so that the cache reconstruction server performs cache reconstruction based on account information corresponding to the request account identifier;

A second query submodule, configured to, when the first thread executes the cache rebuilding step, cause the second thread to start waiting for a preset time, and after waiting for the preset time, perform an account query in the cache database based on the request account identifier;

If the query fails, the step of waiting for a preset time is repeated. If the query fails after the preset number of times is reached, the operation failure result is returned to the user corresponding to the business request.

Optionally, the request flow includes a request record status, and the high-concurrency hotspot data processing device 200 further includes a recording module:

The first recording module is used for, after saving the cache operation credentials corresponding to the account information in the system database, further comprising: recording the request record status of the request flow as successful processing.

Optionally, the high-concurrency hotspot data processing device 200 further includes an anomaly detection module;

The anomaly detection module is used to perform abnormal content detection on the business request that failed the cache processing based on the preset scheduled task after the cache processing fails. The abnormal content includes network interruption, database connection failure, operation failure, and timeout. The operation failure includes at least one of insufficient balance, status stop, and account non-existence.

Optionally, the high-concurrency hotspot data processing device 200 further includes a second recording module;

A second recording module is used to obtain the abnormal content type of the cache processing failure corresponding to the business request after performing abnormal content detection on the business request that failed to be cached based on the preset timed task;

Determine whether the abnormal content type of the failure of the business request cache processing is any one of the insufficient balance type, the status stop type, and the account does not exist type;

If yes, the request record status of the request flow corresponding to the business request is recorded as processing failure;

If not, the request record status of the request flow corresponding to the business request is recorded as being processed.

Optionally, the cache operation voucher includes a processing status and a voucher creation time, and the high-concurrency hotspot data processing device 200 further includes a buffer entry module;

The slow charging module is specifically used for:

Based on a first preset time interval, querying the system database for a cache operation credential that meets a preset condition, wherein the preset condition is that the processing status of the cache operation credential is unprocessed, and the storage time of the cache operation credential is the cache operation credential within a preset time, wherein the preset time is calculated based on the current time and a second preset time interval;

If a cache operation credential that meets the preset conditions is found in the system database, each cache operation credential is divided according to the requesting account of the cache operation credential, and at least one cache operation credential corresponding to each requesting account is processed in parallel using multiple threads;

For each thread, obtaining account information in a system database based on the request account identifier;

For each thread, based on a preset balance calculation formula, the account information and at least one cache operation voucher corresponding to the requesting account, the amount of the requesting account identifier is aggregated to obtain the balance information of the requesting account;

For each thread, based on the account balance, modify the account information corresponding to the requesting account identifier in the system database;

For each thread, after the account information corresponding to the request account identifier is modified successfully, the status of at least one cache operation credential corresponding to the request account identifier is modified to be processed successfully, and the change flow corresponding to the request account is stored in the system database.

In one example, the module in any of the above devices can be one or more integrated circuits configured to implement the above methods, such as: one or more application specific integrated circuits (ASICs), or, one or more digital signal processors (DSPs), or, one or more field programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms.

For another example, when the modules in the device can be implemented in the form of a processing element scheduling program, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call a program. For another example, these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC).

Various objects such as various messages/information/equipment/network elements/systems/devices/actions/operations/processes/concepts that may appear in this application are named. It can be understood that these specific names do not constitute a limitation on the relevant objects. The names assigned may change with factors such as scenarios, contexts or usage habits. The understanding of the technical meaning of the technical terms in this application should be mainly determined from the functions and technical effects embodied/executed in the technical scheme.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

FIG5 is a structural block diagram of an electronic device 300 according to an embodiment of the present application.

As shown in FIG. 5 , the electronic device 300 includes a processor 301 and a memory 302 , and may further include one or more of an information input/information output (I/O) interface 303 and a communication component 304 .

The processor 301 is used to control the overall operation of the electronic device 300 to complete all or part of the steps in the above-mentioned high-concurrency hot spot data processing method; the memory 302 is used to store various types of data to support the operation of the electronic device 300, and these data may include, for example, instructions for any application or method used to operate on the electronic device 300, and application-related data. The memory 302 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, EPROM), programmable read-only memory (Programmable Read-Only Memory, PROM), read-only memory (Read-Only Memory, ROM), magnetic memory, flash memory, magnetic disk or optical disk.

The I/O interface 303 provides an interface between the processor 301 and other interface modules, and the above-mentioned other interface modules can be keyboards, mice, buttons, etc. These buttons can be virtual buttons or physical buttons. The communication component 304 is used to test the wired or wireless communication between the electronic device 300 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more of them, so the corresponding communication component 304 can include: Wi-Fi components, Bluetooth components, NFC components.

The communication bus 305 may include a path to transmit information between the above components. The communication bus 305 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus 305 may be divided into an address bus, a data bus, a control bus, etc.

The electronic device 300 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the high-concurrency hot spot data processing method given in the above embodiment.

The electronic device 300 may include but is not limited to mobile terminals such as digital broadcast receivers, PDAs (personal digital assistants), PMPs (portable multimedia players), etc., and fixed terminals such as digital TVs, desktop computers, etc., and may also be servers, etc.

The computer-readable storage medium provided in the embodiment of the present application is introduced below. The computer-readable storage medium described below and the high-concurrency hot spot data processing method described above can correspond to each other.

The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned high-concurrency hot spot data processing method are implemented.

The computer-readable storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

The terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not expressly listed, or also includes elements inherent to such process, method, article, or apparatus.

The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of application involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the aforementioned application concept. For example, the above features are replaced with (but not limited to) technical features with similar functions applied in the present application.

Claims (10)

1. A method for processing high-concurrency hotspot data, characterized in that the method comprises: Obtaining a business request for the order to be traded, the business request including a request serial number, a business type, user information, a request account identifier, and amount information corresponding to the business type, the business type being any one of deposit, withdrawal, freeze, unfreeze, batch deposit, batch withdrawal, batch freeze, and batch unfreeze; Performing parameter validity check on the user information of the service request and performing serial number anti-duplicate check on the request serial number. After the service request passes the verification, the request serial number of the service request is stored in the system database. The request serial number is used to record information during the service request processing; After the request flow is successfully stored, based on the business type, the request account identifier, the amount information corresponding to the business type and the preset cache data processing rules, the account information in the cache database is cached and updated; After the cache update process is successful, the cache operation credentials corresponding to the business request are stored in the system database, and the operation success result is returned to the user corresponding to the business request. 2. A method for processing data based on a high-concurrency hotspot according to claim 1, characterized in that the cache update processing of the account information in the cache database based on the business type, the requesting account identifier, the amount information corresponding to the business type and the preset cache data processing rules comprises: Performing an account query in a cache database based on the request account identifier; If the account information of the requested account identifier is found in the cache database, then based on the business type and the amount information corresponding to the business type, the found account information is verified for funds; After the queried account information passes the fund verification, performing signature verification on the cache database; After the signature verification is passed in the cache database, the account balance is calculated based on the business type, the amount information corresponding to the business type and the account information, and the cache data of the account information is modified in the cache database according to the account balance to obtain new account information; An operation flow is generated based on the new account information, operation time, business type, and amount information corresponding to the business type, and the operation flow of the business request is stored in a cache database. 3. A method for processing data based on high-concurrency hotspots according to claim 2, characterized in that the cache update processing of the account information in the cache database based on the business type, the requesting account identifier, the amount information corresponding to the business type and the preset cache data processing rules further comprises: If the account information of the requesting account identifier is not found in the cache database, an asynchronous thread is started, where the asynchronous thread includes a first thread and a second thread; The first thread is used to perform a cache reconstruction step, wherein the cache reconstruction includes: sending a cache reconstruction request to a cache reconstruction server, wherein the cache reconstruction request includes a request account identifier, so that the cache reconstruction server performs cache reconstruction based on account information corresponding to the request account identifier; When the first thread executes the cache rebuilding step, the second thread starts to wait for a preset time, and after waiting for the preset time, performs an account query in the cache database based on the request account identifier; If the query fails, the step of waiting for a preset time is repeated. If the query fails after the preset number of times is reached, the operation failure result is returned to the user corresponding to the business request. 4. A method for processing data based on high-concurrency hotspots according to claim 1, characterized in that the request flow includes a request record status, and after the cache operation credentials corresponding to the account information are saved in the system database, it also includes: The request record status of the request pipeline is recorded as successful processing. 5. The method for processing data based on high-concurrency hotspots according to claim 1, characterized in that after cache processing fails, the method further comprises: Based on preset timed tasks, abnormal content detection is performed on business requests that fail to be processed by the cache. The abnormal content includes network interruption, database connection failure, operation failure, and timeout. The operation failure includes at least one of insufficient balance, status stop, and account non-existence. 6. A method for processing data based on high-concurrency hotspots according to claim 5, characterized in that the request flow includes a request record status, and after performing abnormal content detection on the business request that failed to be processed by the cache based on the preset timed task, it also includes: Obtain the abnormal content type of cache processing failure corresponding to the business request; Determine whether the abnormal content type of the failure of the business request cache processing is any one of the insufficient balance type, the status stop type, and the account does not exist type; If yes, the request record status of the request flow corresponding to the business request is recorded as processing failure; If not, the request record status of the request flow corresponding to the business request is recorded as being processed. 7. A method for processing data based on high-concurrency hotspots according to claim 1, characterized in that the cache operation credential includes a processing status and a credential creation time, and the method further comprises: Based on a first preset time interval, querying the system database for a cache operation credential that meets a preset condition, wherein the preset condition is that the processing status of the cache operation credential is unprocessed, and the storage time of the cache operation credential is the cache operation credential within a preset time, wherein the preset time is calculated based on the current time and a second preset time interval; If a cache operation credential that meets the preset conditions is found in the system database, each cache operation credential is divided according to the requesting account of the cache operation credential, and at least one cache operation credential corresponding to each requesting account is processed in parallel using multiple threads; For each thread, obtaining account information in a system database based on the request account identifier; For each thread, based on a preset balance calculation formula, the account information and at least one cache operation voucher corresponding to the requesting account, the amount of the requesting account identifier is aggregated to obtain the balance information of the requesting account; For each thread, based on the account balance, modify the account information corresponding to the requesting account identifier in the system database; For each thread, after the account information corresponding to the request account identifier is modified successfully, the status of at least one cache operation credential corresponding to the request account identifier is modified to be processed successfully, and the change flow corresponding to the request account is stored in the system database. 8. The high-concurrency hotspot data processing device according to claim 1, characterized in that it comprises: an acquisition module, used to acquire a business request for a pending transaction order, wherein the business request includes a request serial number, a business type, user information, a request account identifier, and amount information corresponding to the business type, wherein the business type is any one of: deposit, withdrawal, freeze, unfreeze, batch deposit, batch withdrawal, batch freeze, and batch unfreeze; A parameter verification module is used to perform parameter legitimacy verification on the user information of the service request and to perform serial number anti-duplicate verification on the request serial number. After the service request passes the verification, the request serial number of the service request is stored in the system database. The request serial number is used to record the request time, service type, user information and request account identifier of the service request; A cache processing module, configured to perform cache update processing on the account information in the cache database based on the business type, the requesting account identifier, the amount information corresponding to the business type and a preset cache data processing rule after the request flow is successfully stored; The storage module is used to store the cache operation credentials corresponding to the business request in the system database after the cache processing is successful, and return the operation success result to the user corresponding to the business request. 9. An electronic device, comprising a processor, wherein the processor is coupled to a memory; The processor is configured to execute a computer program stored in the memory, so that the electronic device executes the method according to any one of claims 1 to 7. 10. A computer-readable storage medium, characterized in that it comprises a computer program or an instruction, and when the computer program or the instruction is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 7.