CN112988423A - Message consumption and message distribution method, device, server and storage medium - Google Patents
Message consumption and message distribution method, device, server and storage medium Download PDFInfo
- Publication number
- CN112988423A CN112988423A CN202110296047.7A CN202110296047A CN112988423A CN 112988423 A CN112988423 A CN 112988423A CN 202110296047 A CN202110296047 A CN 202110296047A CN 112988423 A CN112988423 A CN 112988423A
- Authority
- CN
- China
- Prior art keywords
- message
- consumed
- target
- identifier
- consumption
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0201—Market modelling; Market analysis; Collecting market data
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/02—Marketing; Price estimation or determination; Fundraising
- G06Q30/0201—Market modelling; Market analysis; Collecting market data
- G06Q30/0203—Market surveys; Market polls
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/548—Queue
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Development Economics (AREA)
- Strategic Management (AREA)
- Finance (AREA)
- Accounting & Taxation (AREA)
- Entrepreneurship & Innovation (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Game Theory and Decision Science (AREA)
- Economics (AREA)
- Marketing (AREA)
- General Business, Economics & Management (AREA)
- General Engineering & Computer Science (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
本发明实施例公开了一种消息消费、消息分发方法、装置、服务器及存储介质。该消息消费方法包括：确定分发服务器的各分区中与消费服务器一一对应的目标分区，并从目标分区中获取待消费消息，确定待消费消息的消息标识，其中消息标识表示待消费消息所属的消息消费任务，且分发服务器是根据消息标识将待消费消息分发到目标分区上的；根据消息标识从消费服务器的各内存队列中确定出目标内存队列，并将待消费消息发送给目标内存队列；确定消费服务器的各线程中与目标内存队列一一对应的目标线程，基于目标线程对目标内存队列中的待消费消息进行消费。本发明实施例的技术方案，可以在保证各待消费消息的顺序消费的前提下，达到了高吞吐的效果。
The embodiments of the present invention disclose a message consumption and message distribution method, device, server and storage medium. The message consumption method includes: determining target partitions in each partition of the distribution server that correspond to the consuming server one-to-one, acquiring messages to be consumed from the target partitions, and determining message identifiers of the messages to be consumed, wherein the message identifiers indicate the message to be consumed belongs to Message consumption task, and the distribution server distributes the message to be consumed to the target partition according to the message ID; according to the message ID, the target memory queue is determined from each memory queue of the consuming server, and the message to be consumed is sent to the target memory queue; Determine the target thread corresponding to the target memory queue one-to-one in each thread of the consuming server, and consume the messages to be consumed in the target memory queue based on the target thread. The technical solutions of the embodiments of the present invention can achieve the effect of high throughput on the premise of ensuring the sequential consumption of messages to be consumed.
Description
Technical Field
The embodiment of the invention relates to the technical field of computer application, in particular to a method, a device, a server and a storage medium for message consumption and message distribution.
Background
In order to increase the quantity of acquired articles, more and more e-commerce platforms have a docking relationship, that is, a user can acquire an article on another e-commerce platform having a docking relationship with a certain e-commerce platform from the certain e-commerce platform, and at this time, the state of acquiring each article in one article acquisition task between the two e-commerce platforms is synchronized by adopting asynchronous messages.
Since the production processes of the various article acquisition states in the one-time article acquisition task are sequential, the e-commerce platform needs to sequentially consume the articles according to the production sequence of the article acquisition states. At present, a message response mechanism is mainly adopted to achieve the purpose of sequential consumption, namely, after receiving feedback that a previous message is processed and sent by another e-commerce platform, a certain e-commerce platform sends a next message to the other e-commerce platform, and the process is repeated in a circulating mode until all messages in the current article acquisition task are processed.
In the process of implementing the invention, the inventor finds that the following technical problems exist in the prior art: the throughput of the prior art is low because most of the time is waiting for feedback, network fluctuations and/or messages are not processed successfully, resulting in failure to receive feedback, etc.
Disclosure of Invention
The embodiment of the invention provides a message consumption and message distribution method, a message consumption and message distribution device, a server and a storage medium, which are used for realizing the effect of high-throughput sequential consumption of messages to be consumed.
In a first aspect, an embodiment of the present invention provides a message consumption method, which is applied to a consumption server, and the method may include:
determining target partitions corresponding to the consumption servers in all partitions of the distribution server one by one, acquiring the messages to be consumed from the target partitions, and determining message identifiers of the messages to be consumed, wherein the message identifiers represent message consumption tasks to which the messages to be consumed belong, and the distribution server distributes the messages to be consumed to the target partitions according to the message identifiers;
determining a target memory queue from each memory queue of the consumption server according to the message identifier, and sending the message to be consumed to the target memory queue;
and determining target threads corresponding to the target memory queues one by one in each thread of the consumption server, and consuming the messages to be consumed in the target memory queues based on the target threads.
In a second aspect, an embodiment of the present invention further provides a message distribution method, which is applied to a distribution server, and the method may include:
receiving a message to be consumed pushed after the production of a production server, and determining a message identifier of the message to be consumed, wherein the message identifier represents a message consumption task to which the message to be consumed belongs;
and determining a target partition from each partition of the distribution server according to the message identifier, distributing the message to be consumed to the target partition, so that the consumption server which corresponds to the target partition one by one sends the message to be consumed acquired from the target partition to a target memory queue determined from each memory queue according to the message identifier, and consumes the message to be consumed based on the target thread which corresponds to the target memory queue one by one in each thread.
In a third aspect, an embodiment of the present invention further provides a message consumption apparatus configured in a consumption server, where the apparatus may include:
the first message identification determining module is used for determining target partitions corresponding to the consumption servers in all the partitions of the distribution server one by one, acquiring the messages to be consumed from the target partitions, and determining the message identifications of the messages to be consumed, wherein the message identifications represent message consumption tasks to which the messages to be consumed belong, and the distribution server distributes the messages to be consumed to the target partitions according to the message identifications;
the message to be consumed sending module is used for determining a target memory queue from each memory queue of the consumption server according to the message identifier and sending the message to be consumed to the target memory queue;
and the message to be consumed consumption module is used for determining target threads corresponding to the target memory queues one by one in all threads of the consumption server and consuming the message to be consumed in the target memory queues based on the target threads.
In a fourth aspect, an embodiment of the present invention further provides a message distribution apparatus, configured to a distribution server, where the apparatus may include:
the second message identifier determining module is used for receiving the message to be consumed pushed after the production server produces the message to be consumed and determining the message identifier of the message to be consumed, wherein the message identifier represents the message consumption task to which the message to be consumed belongs;
and the message to be consumed distribution module is used for determining a target partition from all partitions of the distribution server according to the message identifier, distributing the message to be consumed to the target partition, so that the consumption server which corresponds to the target partition one by one sends the message to be consumed acquired from the target partition to a target memory queue determined from all memory queues according to the message identifier, and consumes the message to be consumed based on target threads which correspond to the target memory queues one by one in all threads.
In a fifth aspect, an embodiment of the present invention further provides a consumption server, which may include:
one or more processors;
a memory for storing one or more programs;
when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the message consumption method provided by any embodiment of the present invention.
In a sixth aspect, an embodiment of the present invention further provides a distribution server, which may include:
one or more processors;
a memory for storing one or more programs;
when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the message distribution method provided by any embodiment of the present invention.
In a seventh aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the message consumption method or the message distribution method provided in any embodiment of the present invention.
According to the technical scheme of the embodiment of the invention, as the messages to be consumed in each partition in the distribution server are the messages distributed to the partition by the distribution server according to the respective message identifiers, and the message identifiers can represent the message consumption tasks to which the messages to be consumed belong, the consumption server can determine the target partitions corresponding to the consumption server from the partitions one by one, and acquire the messages to be consumed from the target partitions, so that the messages to be consumed belonging to the same message consumption task can be acquired by the same consumption server; for each message to be consumed, the consumption server can determine a target memory queue from each memory queue of the consumption server according to the message identifier of the consumption server and send the message to be consumed to the target memory queue, so that each message to be consumed belonging to the same message consumption task can be sent to the same memory queue; furthermore, the consumption server can determine target threads corresponding to the target memory queues one by one in each thread of the consumption server, and consume the messages to be consumed in the target memory queues based on the target threads, so that the messages to be consumed belonging to the same message consumption task can be consumed by the same thread. According to the technical scheme, the consumption server splits the messages to be consumed according to the message identifications of the messages to be consumed, so that the messages to be consumed belonging to the same message consumption task can enter the same memory queue in the same partition and the same consumption server and are consumed by the same thread, and therefore the consumption sequence of the messages to be consumed is guaranteed; meanwhile, the consumption server acquires the messages to be consumed which are split into the target partition from the target partition corresponding to the consumption server, and splits the acquired messages to be consumed into the corresponding threads respectively for consumption, so that the multi-partition and multi-thread application achieves the effect of high throughput.
Drawings
FIG. 1 is a flow chart of a message consumption method according to a first embodiment of the present invention;
FIG. 2 is a flow chart of a message consumption method according to a second embodiment of the present invention;
fig. 3 is a flowchart of a message distribution method in the third embodiment of the present invention;
fig. 4 is a schematic diagram of an alternative example of a message distribution method in the third embodiment of the present invention;
FIG. 5 is a block diagram of a message consumption apparatus according to a fourth embodiment of the present invention;
fig. 6 is a block diagram of a message distribution apparatus according to a fifth embodiment of the present invention;
FIG. 7 is a schematic diagram of a consumption server according to a sixth embodiment of the present invention;
fig. 8 is a schematic structural diagram of a distribution server in the seventh embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Before the embodiment of the present invention is described, an application scenario of the embodiment of the present invention is exemplarily described: taking the application scenario of interfacing the third party fresh platform on the public number or H5 page as an example, the order status between the two parties is synchronized by using asynchronous messages. It should be noted that, on one hand, there is a production sequence between the order states belonging to the same order, for example, the creation state, the state to be paid, the paid state, the shipped state, the in-delivery state, and the received state are order states produced in sequence, and then the consumption processes of these order states are also to be consumed according to their respective production sequences, for example, a consumer must ensure that the state to be paid is consumed before consuming the state to be paid, and the correct consumption sequence is a precondition for the order to be effectively performed; on the other hand, the production server where the public number and the H5 page are located and the consumption server where the third-party fresh platform is located are generally servers developed by different developers, so to ensure that the development processes of the two servers do not affect each other, asynchronous messages are used for message synchronization.
On this basis, aiming at the application scene that asynchronous messages are needed to be adopted to synchronize the messages with the sequence, in order to ensure the sequential consumption of the messages and realize the effect of high throughput, the embodiments of the invention provide the following message consumption method and message distribution method.
Example one
Fig. 1 is a flowchart of a message consumption method according to a first embodiment of the present invention. The embodiment can be applied to the condition of ensuring that the messages to be consumed belonging to the same message consumption task are sequentially consumed according to the production sequence, and is particularly suitable for realizing the condition of high throughput while ensuring that the messages to be consumed belonging to the same message consumption task are sequentially consumed according to the production sequence. The method can be executed by the message consumption device provided by the embodiment of the invention, the device can be realized by software and/or hardware, and the device can be integrated on a consumption server.
Referring to fig. 1, the method of the embodiment of the present invention specifically includes the following steps:
s110, determining target partitions corresponding to the consumption servers in all the partitions of the distribution server one by one, acquiring the messages to be consumed from the target partitions, and determining message identifiers of the messages to be consumed, wherein the message identifiers represent message consumption tasks to which the messages to be consumed belong, and the distribution server distributes the messages to be consumed to the target partitions according to the message identifiers.
The production server may be a server for producing the message to be consumed, that is, a server where the producer is located, and may send the message to be consumed produced by the production server to the distribution server. On this basis, the distribution server may be a server for distributing the message to be consumed sent by the production server to a corresponding partition (partition), that is, a server where the message queue is located, and a cluster formed by at least two distribution servers may be referred to as a message cluster, and in an application scenario of the embodiment of the present invention, it may also be referred to as an asynchronous message cluster. In practical applications, optionally, the partition may be a partition under a certain topic (topic), and each message to be consumed may be a message belonging to the same topic. Specifically, the distribution server may distribute the message to be consumed to the corresponding partition according to a message identifier of the message to be consumed, where the message identifier may indicate to which message consumption task the message to be consumed belongs, so that each message to be consumed belonging to the same message consumption task is distributed to the same partition, so as to perform the heap processing on the messages to be consumed. In other words, after the messages to be consumed belonging to the same message consumption task sequentially arrive at the distribution server, the distribution server can distribute the messages to be consumed to the same partition because the message identifications of the messages to be consumed are the same, so that the messages to be consumed are consumed by the same consumption server.
The consumption server may be a server for reading the message to be consumed from the partition and consuming the message to be consumed. In order to ensure the sequential consumption of the messages to be consumed belonging to the same message consumption task, each consumption server can have a partition corresponding to the consumption server one by one, so that the messages to be consumed belonging to the same message consumption task in the partition can be read by the same consumption server. In practical application, optionally, the number of the partitions in the distribution server is the same as the number of the consumption servers, and a one-to-one correspondence relationship exists between each partition and each consumption server. Therefore, for each consumption server, the consumption server can determine a target partition corresponding to the consumption server in a one-to-one manner in each partition of the distribution server, and acquire the message to be consumed from the target partition. At the same time, the consumption server can also determine the message identification of the message to be consumed.
In practical applications, optionally, in combination with an application scenario that may be involved in the embodiment of the present invention, the to-be-consumed message may include an item acquisition status in the item acquisition task, where the item acquisition status may include at least one of a creation status, a to-be-paid status, a shipped status, a in-delivery status, and a received status, and the item acquisition statuses are statuses in which a consumption order exists. In this basic mountain, the message consumption task may be understood as a task of consuming each item acquisition status in a certain item acquisition task.
S120, determining a target memory queue from each memory queue of the consumption server according to the message identifier, and sending the message to be consumed to the target memory queue.
After the messages to be consumed belonging to the same message consumption task sequentially arrive at the consumption server according to the production sequence, in order to ensure the consumption sequence of the messages to be consumed, the messages to be consumed need to enter the same memory queue and be consumed by the same thread. On this basis, considering that the messages to be consumed acquired by the consumption server may be from different message consumption tasks, the consumption server may determine a target memory queue from each memory queue of the consumption server according to the message identifier of each message to be consumed, where the target memory queue may be a memory queue specially used for processing each message to be consumed corresponding to the message identifier; furthermore, the consumption server may send the message to be consumed to the target memory queue, and at this time, each message to be consumed belonging to the same message consumption task enters the same memory queue.
S130, determining target threads corresponding to the target memory queues one by one in the threads of the consumption server, and consuming the messages to be consumed in the target memory queues based on the target threads.
As described above, in order to ensure the consumption sequence of the messages to be consumed belonging to the same message consumption task, the messages to be consumed need to be entered into the same memory queue and consumed by the same thread, at present, the messages to be consumed are already sent to the same memory queue, and then, in order to ensure that the messages can be consumed by the same thread, a one-to-one correspondence relationship may exist between each memory queue and each thread, that is, the number of the memory queues in the consumption server may be set according to the number of the threads in the consumption server, and the number of the threads may be set according to the performance and the service requirements of the consumption server.
Therefore, for the target memory queue where the message to be consumed is located, the consumption server can determine the target threads corresponding to the target memory queue one by one from the threads of the consumption server, and further consume the message to be consumed in the target memory queue based on the target threads, so that the effect that the message to be consumed in the same message consumption task is consumed by the same thread is achieved.
According to the technical scheme of the embodiment of the invention, as the messages to be consumed in each partition in the distribution server are the messages distributed to the partition by the distribution server according to the respective message identifiers, and the message identifiers can represent the message consumption tasks to which the messages to be consumed belong, the consumption server can determine the target partitions corresponding to the consumption server from the partitions one by one, and acquire the messages to be consumed from the target partitions, so that the messages to be consumed belonging to the same message consumption task can be acquired by the same consumption server; for each message to be consumed, the consumption server can determine a target memory queue from each memory queue of the consumption server according to the message identifier of the consumption server and send the message to be consumed to the target memory queue, so that each message to be consumed belonging to the same message consumption task can be sent to the same memory queue; furthermore, the consumption server can determine target threads corresponding to the target memory queues one by one in each thread of the consumption server, and consume the messages to be consumed in the target memory queues based on the target threads, so that the messages to be consumed belonging to the same message consumption task can be consumed by the same thread. According to the technical scheme, the consumption server splits the messages to be consumed according to the message identifications of the messages to be consumed, so that the messages to be consumed belonging to the same message consumption task can enter the same memory queue in the same partition and the same consumption server and are consumed by the same thread, and therefore the consumption sequence of the messages to be consumed is guaranteed; meanwhile, the consumption server acquires the messages to be consumed which are split into the target partition from the target partition corresponding to the consumption server, and splits the acquired messages to be consumed into the corresponding threads respectively for consumption, so that the multi-partition and multi-thread application achieves the effect of high throughput.
Example two
Fig. 2 is a flowchart of a message consumption method provided in the second embodiment of the present invention. The present embodiment is optimized based on the above technical solutions. In this embodiment, optionally, determining the target memory queue from the memory queues of the consumption server according to the message identifier may include: processing the message identifier based on a preset load balancing strategy to obtain a processing identifier; and acquiring the queue number of each memory queue of the consumption server, and determining a target memory queue from each memory queue according to the processing identifier and the queue number. The same or corresponding terms as those in the above embodiments are not explained in detail herein.
Referring to fig. 2, the method of the present embodiment may specifically include the following steps:
s210, determining target partitions corresponding to the consumption servers in all the partitions of the distribution server one by one, acquiring the messages to be consumed from the target partitions, and determining message identifiers of the messages to be consumed, wherein the message identifiers represent message consumption tasks to which the messages to be consumed belong, and the distribution server distributes the messages to be consumed to the target partitions according to the message identifiers.
S220, processing the message identifier based on a preset load balancing strategy to obtain a processing identifier.
The load balancing policy may be a preset policy for balancing and distributing loads (i.e., work tasks) to a plurality of operation units for execution. In combination with the application scenarios that may be related to the embodiments of the present invention, specifically, the load balancing policy may be a policy for processing each message identifier, or may be referred to as a policy for disturbing each message identifier, so that the processing identifiers respectively corresponding to each obtained message identifier exhibit normal distribution (i.e., balanced distribution) as much as possible, and when the load balancing policy is mutually matched with the subsequent steps, each message to be consumed may be distributed in each memory queue as much as possible, so as to fully utilize each memory queue, i.e., fully utilize each resource on the consumption server, and improve throughput to the maximum extent.
S230, acquiring the queue number of each memory queue of the consumption server, determining a target memory queue from each memory queue according to the processing identifier and the queue number, and sending the message to be consumed to the target memory queue.
The method includes the steps of obtaining the number of queues of each memory queue of a consumption server, determining to which memory queue a corresponding message to be consumed can be sent according to the number of queues and a disturbed processing identifier, that is, determining to which memory queue the remaining message to be consumed having the same processing identifier as the message to be consumed has been sent, so as to send the message to be consumed to the memory queue (i.e., a target memory queue), so that the messages to be consumed belonging to the same message consumption task can be sent to the same memory queue.
S240, determining target threads corresponding to the target memory queues one by one in the threads of the consumption server, and consuming the messages to be consumed in the target memory queues based on the target threads.
According to the technical scheme of the embodiment of the invention, the message identification is processed through the preset load balancing strategy to obtain the processing identification, and the distribution trend of the processing identification of each message to be consumed is relatively balanced; and then, determining a target memory queue from the memory queues according to the queue number of the memory queues of the consumption server and the scattered processing identifiers, wherein the target memory queue is a memory queue to which each message to be consumed with the same processing identifier is to be sent, so that the effects that the load of each message to be consumed is balanced and each message to be consumed belonging to the same message consumption task is sent to the same memory queue are achieved.
An optional technical solution, processing a message identifier based on a preset load balancing policy to obtain a processing identifier, may include: mapping the message identifier to obtain a digital type mapping identifier with a preset digit number; updating half of the preset digit to the preset digit, and displacing the mapping identifier based on the preset digit to obtain a displacement identifier; and carrying out XOR processing on the mapping identifier and the displacement identifier to obtain a processing identifier.
Since the message identifiers of the messages to be consumed are not necessarily continuous identifiers, for example, it is assumed that the number of queues in the memory queue is 3, some message identifiers are 1, 4, and 7, and no 2, 3, 5, or 6 exists, and at this time, if the target memory queue is determined directly according to the number of queues and the message identifiers, it is possible to distribute all the messages to be consumed corresponding to all the message identifiers to the 1 st memory queue, that is, the 1 st memory queue is used as the target memory queue corresponding to all the message identifiers, and at this time, the problem of load imbalance obviously exists. In order to solve the problem, the message identifier can be mapped to obtain a mapping identifier of a digital type with preset digits, and the specific setting of the preset digits is related to a specific mapping mode; dividing the result of the preset digit by 2 to be used as the preset digit, and displacing the mapping identifier based on the preset digit, wherein the obtained displacement identifier is an identifier disturbed in displacement; and carrying out XOR processing on the mapping identifier and the displacement identifier, namely carrying out XOR processing on the high order and the low order in the mapping identifier, thereby scattering the distribution of each message identifier on the memory queue and achieving the effect of load balancing on the preset digit.
In order to better understand the implementation process of the above technical solution, a hash code () function is taken as a specific mapping manner as an example, and is exemplarily described herein. Mapping the message identifier to a hash value h of a 32-bit numeric type by calling a hashCode () function, i.e., h ═ hashCode (message identifier); among them, the hashCode function can transform an input of arbitrary length (also called pre-mapping) into an output of fixed length through a hash algorithm, and the output is a hash value. And (2) shifting h to the right by 16 bits, wherein the 16 bits are half of 32 bits, and performing exclusive-or operation with h, namely performing exclusive-or operation on the 16 bits of the high bits and the 16 bits of the low bits in h, namely m ═ h ^ (h > > >16), so that the distribution of the message identifiers on the memory queue is scattered, and the effect of load balancing is achieved, namely normal distribution is basically presented on the numerical value of the 32 bits.
An optional technical solution, determining a target memory queue from each memory queue according to the processing identifier and the number of queues, may include: performing subtraction operation on the number of the queues to obtain the subtracted number, and performing AND operation on the subtracted number and the processing identifiers to obtain the queue numbers; or, carrying out a modulus operation on the queue number and the processing identifier to obtain a queue number; and taking the memory queue corresponding to the queue number in each memory queue as a target memory queue. Each memory queue has its own queue number, and the process of determining the target memory queue in each memory queue may also be understood as a process of determining the queue number of the target memory queue. Accordingly, the process of determining the queue number may be to subtract one from the number of the queue to obtain a subtracted number, and perform an and operation on the subtracted number and the processing identifier to obtain the queue number. It should be noted that, since the operation is equivalent to a modulo operation, the queue number may also be obtained by performing a modulo operation on the queue number and the processing identifier. Compared with the modulus operation, the operation is simpler in the computer implementation process. And then, the memory queue corresponding to the queue number in each memory queue is used as a target memory queue, so that the effect of accurately determining the target memory queue is achieved, and each message to be consumed sent to the target memory queue is a message belonging to the same message consumption task.
In order to better understand the implementation process of the above technical solution, the above example is continued as an example, and an exemplary description is provided. Assuming the number n of queues in the memory queue, performing an and operation on m and (n-1) to obtain a queue number i, i is (n-1) & m, and further taking the memory queue corresponding to i as a target memory queue, thereby ensuring that each message to be consumed in the same message consumption task enters the same memory queue.
In summary, i ═ n-1 & (hashCode (message identifier) ^ (hashCode) > >16)), so that the effects of load balancing and entering of each message to be consumed in the same message consumption task into the same memory queue are achieved. Certainly, the deterministic modification that each message to be consumed in the same message consumption task enters the same memory queue can be performed on the basis of the rest of load balancing strategies, which is not specifically limited herein.
EXAMPLE III
Fig. 3 is a flowchart of a message distribution method provided in the third embodiment of the present invention. The present embodiment is applicable to a case where it is guaranteed that messages to be consumed belonging to the same message consumption task are distributed to the same partition in the distribution server, and is particularly applicable to a case where it is guaranteed that messages to be consumed belonging to the same message consumption task are distributed to the same partition in the distribution server, and their high-throughput consumption in the consumption server in turn is guaranteed in accordance with their respective production order. The method can be executed by the message distribution device provided by the embodiment of the invention, the device can be realized by software and/or hardware, and the device can be integrated on a distribution server.
Referring to fig. 3, the method of the embodiment of the present invention specifically includes the following steps:
s310, receiving the message to be consumed sent by the production server after production, and determining a message identifier of the message to be consumed, wherein the message identifier represents a message consumption task to which the message to be consumed belongs.
The production server may be a server for producing the message to be consumed, that is, a server where the producer is located, and may send the message to be consumed produced by the production server to the distribution server. After receiving the message to be consumed, the distribution server may determine a message identifier indicating a message consumption task to which the message to be consumed belongs.
S320, determining target partitions from all partitions of the distribution server according to the message identifiers, distributing the messages to be consumed to the target partitions, enabling the consumption servers corresponding to the target partitions one by one to send the messages to be consumed acquired from the target partitions to target memory queues determined from all memory queues according to the message identifiers, and consuming the messages to be consumed based on the target threads corresponding to the target memory queues one by one in all threads.
In practical applications, the partition (partition) may be a partition under a certain topic (topic), and each message to be consumed may be a message all belonging to the same topic. The distribution server can distribute the messages to be consumed to corresponding partitions according to the message identifications of the messages to be consumed, and the corresponding partitions can be target partitions corresponding to the message identifications in all the partitions, so that all the messages to be consumed belonging to the same message consumption task can be distributed to the same partition, and the messages to be consumed are processed in a piling mode. In other words, after the messages to be consumed belonging to the same message consumption task sequentially arrive at the distribution server, the distribution server can distribute the messages to be consumed to the same partition because the message identifications of the messages to be consumed are the same, so that the messages to be consumed are consumed by the same consumption server. The message consumption process of the consumption server has been described in detail in the above embodiments, and is not described herein again.
It should be noted that the distribution server may be understood as a server where the message queue is located, and the message queue may be an MQ. A cluster formed by at least two distribution servers may be referred to as a message cluster, and in an application scenario of the embodiment of the present invention, it may also be referred to as an asynchronous message cluster. The message cluster may be a highly available message cluster, i.e. may include a master distribution server and a slave distribution server therein, the number of master distribution servers may be at least one, and the number of slave distribution servers may also be at least one. When the production server sends the messages to be consumed to the message cluster, the production server can send the messages to be consumed to at least one distribution server, and each distribution server can be provided with a catalog for recording which messages to be consumed are stored in each distribution server, so that the consumption server can determine which distribution server the messages to be consumed, which are to be acquired, are located in from the catalog of any distribution server, and the messages to be consumed are acquired from the partitions, which are in one-to-one correspondence with the consumption server, on the distribution server.
According to the technical scheme of the embodiment of the invention, the distribution server receives the message to be consumed sent after production by the production server and determines the message identifier for representing the message consumption task to which the message to be consumed belongs; furthermore, the distribution server determines a target partition from each partition according to the message identifier and distributes the message to be consumed to the target partition, namely, the message to be consumed belonging to the same message consumption task is distributed to the same partition, and the message to be consumed belonging to different message consumption tasks is distributed to different partitions. According to the technical scheme, the distribution server distributes the messages to be consumed according to the message identifications of the messages to be consumed, so that the messages to be consumed belonging to the same message consumption task can enter the same partition and further cooperate with the consumption server, so that the messages to be consumed can enter the same memory queue in the same consumption server again and are further consumed by the same thread, and therefore the consumption sequence of the messages to be consumed is guaranteed; meanwhile, the high-throughput effect is achieved by splitting each message to be consumed into the corresponding partitions and fully utilizing each consumption server.
A selectable technical solution, where determining a target partition from partitions of a distribution server according to a message identifier may include: processing the message identifier based on a preset load balancing strategy to obtain a processing identifier; and acquiring the partition number of each partition of the distribution server, and determining a target partition from each partition according to the processing identifier and the partition number.
The load balancing policy may be a preset policy for balancing and distributing loads (i.e., work tasks) to a plurality of operation units for execution. In combination with the application scenarios that may be related to the embodiments of the present invention, specifically, the load balancing policy may be a policy for processing each message identifier, or may be referred to as a policy for disturbing each message identifier, so that the processing identifiers respectively corresponding to each obtained message identifier exhibit normal distribution (i.e., balanced distribution) as much as possible, and when the load balancing policy is mutually matched with subsequent steps, each message to be consumed may be distributed in each partition as much as possible in a balanced manner, so as to fully utilize the consumption servers corresponding to each partition one to one, and improve throughput to the greatest extent.
When the throughput is improved, it is further required to ensure that the messages to be consumed belonging to the same message consumption task can be distributed to the same partition, at this time, the partition number of each partition of the distribution server may be obtained, and it is determined, according to the partition number and the disturbed processing identifier, to which partition the corresponding message to be consumed can be distributed, that is, it is determined to which partition the remaining messages to be consumed having the same processing identifier as the message to be consumed have been distributed, so as to distribute the message to be consumed to the partition (i.e., the target partition). According to the technical scheme, the effects that the loads of the messages to be consumed are balanced, and the messages to be consumed belonging to the same message consumption task are distributed to the same partition are achieved.
It should be noted that, the process identifier obtaining process may be: mapping the message identifier to obtain a digital type mapping identifier with a preset digit number; updating half of the preset digit to the preset digit, and displacing the mapping identifier based on the preset digit to obtain a displacement identifier; and carrying out XOR processing on the mapping identifier and the displacement identifier to obtain a processing identifier. And/or, the determination process of the target partition may be: carrying out subtraction operation on the number of the partitions to obtain the subtraction number, and carrying out AND operation on the subtraction number and the processing identification to obtain the partition number; or, performing a modulus operation on the partition number and the processing identifier to obtain a partition number; and taking the partition corresponding to the partition number in each partition as a target partition. Of course, other schemes may be used to determine the process identifier and/or the target partition, which are not specifically limited herein.
In order to better understand the specific implementation process of the above steps, the following continues to use the application scenario of interfacing the third party living platform on the public number or H5 page as an example, and the implementation process and the coordination process of the message consumption method and the message distribution method in each embodiment of the present invention are exemplarily described.
Exemplarily, as shown in fig. 4, the following focuses on 2 embodiments of the present invention: message consumption splitting guarantees sequential and high throughput logic and a customized load balancing algorithm (i.e., a load balancing policy) is elaborated.
Message consumption splitting guarantee order + high throughput
In order to ensure that all order states of an order can enter the same partition (partition), the same consumer, the same memory queue and the same thread to realize sequential consumption and high throughput effect,
1) when the order states sequentially reach the message queue (Kafka), the order states of the same order are guaranteed to enter the same partition according to the load balancing algorithm 1. The number of partitions can be determined according to the number of consumers (i.e. consumption servers where the consumers are located), the two are equal, and each partition corresponds to a unique consumer. In addition, Kafka is an implementation under JMQ that is the underlying encapsulated content of the latest version of JMQ.
2) The number of threads supporting high throughput can be set by the consumer according to the performance and business requirements of the consumer to perform message consumption together. Then, the message queues with the same number as the threads are set, and each thread has a unique thread queue corresponding to the thread queue. And after the order state sequence reaches the consumer, ensuring that the order states of the same order enter the same consumer and the same memory queue to be consumed by the same thread according to the load balancing algorithm 2 to ensure the consumption sequence.
Second, customized load balancing algorithm
Suppose that partition or memory queue numbers from 1, as shown in FIG. 4 (1, 2, 3)
1) Calling a hashCode () function to map the order identification of the order into a hash value h of a digital type, namely h ═ hashCode (message identification);
2) the h is shifted to the right by 16 bits and then subjected to exclusive or operation with the h, and m is h ^ (h > > >16), so that the distribution of the order mark on the partition or the memory queue is scattered, and the effect of load balancing is achieved;
3) n is the number of the partition or the memory queue, m and (n-1) are subjected to AND operation to obtain a number i, namely i is (n-1) & m, wherein the memory queue or the partition corresponding to i is the memory queue or the partition to which the order state is to be entered, and as all order states of the same order can enter the same partition or the same memory queue.
In summary, i ═ n-1 & (hashCode (order identification) ^ (hashCode) > > 16)).
Example four
Fig. 5 is a block diagram of a message consumption apparatus according to a fourth embodiment of the present invention, which is configured to execute the message consumption method according to any of the foregoing embodiments. The device and the message consumption method of the above embodiments belong to the same inventive concept, and details that are not described in detail in the embodiments of the message consumption device may refer to the embodiments of the message consumption method described above. Referring to fig. 5, the apparatus may be configured in a consumption server, and may specifically include: a first message identification determination module 410, a to-be-consumed message sending module 420, and a to-be-consumed message consuming module 430.
The first message identifier determining module 410 is configured to determine target partitions, which correspond to the consumption servers in each partition of the distribution server one to one, acquire the message to be consumed from the target partitions, and determine a message identifier of the message to be consumed, where the message identifier indicates a message consumption task to which the message to be consumed belongs, and the distribution server distributes the message to be consumed to the target partitions according to the message identifier;
a message to be consumed sending module 420, configured to determine a target memory queue from each memory queue of the consumption server according to the message identifier, and send the message to be consumed to the target memory queue;
and a message to be consumed consumption module 430, configured to determine a target thread corresponding to the target memory queue in each thread of the consumption server, and consume the message to be consumed in the target memory queue based on the target thread.
Optionally, the to-be-consumed message sending module 420 may specifically include:
the first processing identifier obtaining submodule is used for processing the message identifier based on a preset load balancing strategy to obtain a processing identifier; and the target memory queue determining submodule is used for acquiring the queue number of each memory queue of the consumption server and determining the target memory queue from each memory queue according to the processing identifier and the queue number.
On this basis, optionally, the first processing identifier obtaining sub-module may specifically include:
the mapping identifier obtaining unit is used for mapping the message identifier to obtain a mapping identifier of a digital type with preset digits; a displacement identifier obtaining unit, configured to update half of the preset digit to the preset digit, and displace the mapping identifier based on the preset digit to obtain a displacement identifier; and the processing identifier obtaining unit is used for carrying out XOR processing on the mapping identifier and the displacement identifier to obtain a processing identifier.
On this basis, optionally, the target memory queue determination submodule may specifically include:
a queue number obtaining unit, configured to subtract one from the number of queues to obtain a subtracted number, and perform an and operation on the subtracted number and the processing identifier to obtain a queue number; or, carrying out a modulus operation on the queue number and the processing identifier to obtain a queue number; and the target memory queue determining unit is used for taking the memory queue corresponding to the queue number in each memory queue as the target memory queue.
Optionally, the message to be consumed includes an item acquisition status in the item acquisition task, and the item acquisition status includes at least one of a creation status, a status to be paid, a paid status, a shipped status, a in-delivery status, and a received status.
In the message consumption apparatus provided in the fourth embodiment of the present invention, because each message to be consumed in each partition in the distribution server is a message that is distributed to the partition by the distribution server according to its respective message identifier, which may represent a message consumption task to which the message to be consumed belongs, the consumption server determines, from each partition, a target partition that corresponds to itself one to one by using the first message identifier determining module, and acquires the message to be consumed from the target partition, so that each message to be consumed belonging to the same message consumption task may be acquired by the same consumption server; aiming at each message to be consumed, the consumption server determines a target memory queue from each memory queue of the consumption server through a message sending module to be consumed according to the message identification of the consumption server, and sends the message to be consumed to the target memory queue, so that each message to be consumed belonging to the same message consumption task can be sent to the same memory queue; and then, the consumption server determines target threads corresponding to the target memory queues one by one in each thread of the consumption server through the message consumption module to be consumed, and consumes the messages to be consumed in the target memory queues based on the target threads, so that the messages to be consumed belonging to the same message consumption task can be consumed by the same thread. According to the device, the consumption server splits the messages to be consumed according to the message identifications of the messages to be consumed, so that the messages to be consumed belonging to the same message consumption task can enter the same memory queue in the same partition and the same consumption server and are further consumed by the same thread, and therefore the consumption sequence of the messages to be consumed is guaranteed; meanwhile, the consumption server acquires the messages to be consumed which are split into the target partition from the target partition corresponding to the consumption server, and splits the acquired messages to be consumed into the corresponding threads respectively for consumption, so that the multi-partition and multi-thread application achieves the effect of high throughput.
The message consumption device provided by the embodiment of the invention can execute the message consumption method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.
It should be noted that, in the embodiment of the message consumption device, the included units and modules are merely divided according to the functional logic, but are not limited to the above division as long as the corresponding functions can be implemented; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
EXAMPLE five
Fig. 6 is a block diagram of a message distribution apparatus according to a fifth embodiment of the present invention, where the apparatus is configured to execute a message distribution method according to any of the foregoing embodiments. The device and the message distribution method of the above embodiments belong to the same inventive concept, and details that are not described in detail in the embodiments of the message distribution device may refer to the embodiments of the message distribution method described above. Referring to fig. 6, the apparatus may be configured to a distribution server, and specifically includes: a second message identification determination module 510 and a message to be consumed distribution module 520.
The second message identifier determining module 510 is configured to receive a message to be consumed pushed by the production server after production, and determine a message identifier of the message to be consumed, where the message identifier indicates a message consumption task to which the message to be consumed belongs;
the message to be consumed distribution module 520 is configured to determine a target partition from each partition of the distribution server according to the message identifier, distribute the message to be consumed to the target partition, so that the consumption server corresponding to the target partition sends the message to be consumed acquired from the target partition to a target memory queue determined from each memory queue according to the message identifier, and consume the message to be consumed based on a target thread corresponding to the target memory queue in each thread.
Optionally, the to-be-consumed message distribution module 520 may specifically include:
the second processing identifier obtaining submodule is used for processing the message identifier based on a preset load balancing strategy to obtain a processing identifier;
and the target partition determining submodule is used for acquiring the partition number of each partition of the distribution server and determining the target partition from each partition according to the processing identifier and the partition number.
In the message distribution device provided by the fifth embodiment of the present invention, the distribution server receives the message to be consumed sent after production by the production server through the second message identifier determining module, and determines the message identifier used for representing the message consumption task to which the message to be consumed belongs; furthermore, the distribution server determines a target partition from each partition according to the message identifier through the message distribution module to be consumed, and distributes the message to be consumed to the target partition, namely, distributes the message to be consumed belonging to the same message consumption task to the same partition, and distributes the message to be consumed belonging to different message consumption tasks to different partitions. According to the device, the distribution server distributes the messages to be consumed according to the message identifications of the messages to be consumed, so that the messages to be consumed belonging to the same message consumption task can enter the same partition and further cooperate with the consumption server, so that the messages can enter the same memory queue in the same consumption server again and are consumed by the same thread, and therefore the consumption sequence of the messages to be consumed is guaranteed; meanwhile, the high-throughput effect is achieved by splitting each message to be consumed into the corresponding partitions and fully utilizing each consumption server.
The message distribution device provided by the embodiment of the invention can execute the message distribution method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.
It should be noted that, in the embodiment of the message distribution apparatus, each included unit and module are merely divided according to functional logic, but are not limited to the above division as long as the corresponding functions can be implemented; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
EXAMPLE six
Fig. 7 is a schematic structural diagram of a consumption server according to a sixth embodiment of the present invention, and as shown in fig. 7, the consumption server includes a memory 610, a processor 620, an input device 630, and an output device 640. The number of processors 620 in the consumption server may be one or more, and one processor 620 is taken as an example in fig. 7; the memory 610, processor 620, input device 630, and output device 640 in the consumption server may be connected by a bus or other means, such as by bus 650 in fig. 7.
The memory 610, which is a computer-readable storage medium, may be used to store software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the message consumption method in the embodiment of the present invention (e.g., the first message identification determination module 410, the message to be consumed transmission module 420, and the message to be consumed consumption module 430 in the message consumption device). The processor 620 executes various functional applications of the consumption server and data processing by executing software programs, instructions, and modules stored in the memory 610, that is, implements the message consumption method described above.
The memory 610 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the consumption server, and the like. Further, the memory 610 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, memory 610 may further include memory located remotely from processor 620, which may be connected to devices through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 630 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function controls of the device. The output device 640 may include a display device such as a display screen.
EXAMPLE seven
Fig. 8 is a schematic structural diagram of a distribution server according to a seventh embodiment of the present invention, and as shown in fig. 8, the distribution server includes a memory 710, a processor 720, an input device 730, and an output device 740. The number of the processors 720 in the distribution server may be one or more, and one processor 720 is taken as an example in fig. 8; the memory 710, processor 720, input device 730, and output device 740 in the distribution server may be connected by a bus or other means, such as by bus 750 in fig. 8.
The memory 710 is used as a computer-readable storage medium for storing software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the message distribution method in the embodiment of the present invention (for example, the second message identification determination module 510 and the message to be consumed distribution module 520 in the message distribution apparatus). The processor 720 executes various functional applications and data processing of the distribution server by executing software programs, instructions and modules stored in the memory 710, that is, implements the message distribution method described above.
The memory 710 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the distribution server, and the like. Further, the memory 710 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, memory 710 may further include memory located remotely from processor 720, which may be connected to devices through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 730 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function controls of the device. The output device 740 may include a display device such as a display screen.
Example eight
An eighth embodiment of the present invention provides a storage medium containing computer-executable instructions, which when executed by a computer processor, are configured to perform a message consumption method applied to a consumption server, where the method includes:
determining target partitions corresponding to the consumption servers in all partitions of the distribution server one by one, acquiring the messages to be consumed from the target partitions, and determining message identifiers of the messages to be consumed, wherein the message identifiers represent message consumption tasks to which the messages to be consumed belong, and the distribution server distributes the messages to be consumed to the target partitions according to the message identifiers;
determining a target memory queue from each memory queue of the consumption server according to the message identifier, and sending the message to be consumed to the target memory queue;
and determining target threads corresponding to the target memory queues one by one in each thread of the consumption server, and consuming the messages to be consumed in the target memory queues based on the target threads.
Of course, the storage medium provided by the embodiment of the present invention contains computer-executable instructions, and the computer-executable instructions are not limited to the method operations described above, and may also perform related operations in the message consumption method provided by any embodiment of the present invention.
Example nine
An embodiment of the present invention provides a storage medium containing computer-executable instructions, where the computer-executable instructions are executed by a computer processor to perform a message distribution method applied to a distribution server, and the method includes:
receiving a message to be consumed pushed after the production of a production server, and determining a message identifier of the message to be consumed, wherein the message identifier represents a message consumption task to which the message to be consumed belongs;
and determining a target partition from each partition of the distribution server according to the message identifier, distributing the message to be consumed to the target partition, so that the consumption server which corresponds to the target partition one by one sends the message to be consumed acquired from the target partition to a target memory queue determined from each memory queue according to the message identifier, and consumes the message to be consumed based on the target thread which corresponds to the target memory queue one by one in each thread.
Of course, the storage medium provided by the embodiment of the present invention contains computer-executable instructions, and the computer-executable instructions are not limited to the method operations described above, and may also perform related operations in the message distribution method provided by any embodiment of the present invention.
From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. With this understanding, the technical solutions of the present invention may be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (12)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110296047.7A CN112988423A (en) | 2021-03-19 | 2021-03-19 | Message consumption and message distribution method, device, server and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110296047.7A CN112988423A (en) | 2021-03-19 | 2021-03-19 | Message consumption and message distribution method, device, server and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112988423A true CN112988423A (en) | 2021-06-18 |
Family
ID=76333500
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110296047.7A Pending CN112988423A (en) | 2021-03-19 | 2021-03-19 | Message consumption and message distribution method, device, server and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112988423A (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113407637A (en) * | 2021-07-13 | 2021-09-17 | 上海浦东发展银行股份有限公司 | Data synchronization method and device, electronic equipment and storage medium |
| CN113626217A (en) * | 2021-07-28 | 2021-11-09 | 北京达佳互联信息技术有限公司 | Asynchronous message processing method and device, electronic equipment and storage medium |
| CN113626211A (en) * | 2021-06-29 | 2021-11-09 | 欧电云信息科技(江苏)有限公司 | Switching method and switching system of message queue, electronic equipment and storage medium |
| CN113835905A (en) * | 2021-09-28 | 2021-12-24 | 北京奇艺世纪科技有限公司 | Message queue load balancing method and device, electronic equipment and medium |
| CN113900838A (en) * | 2021-10-20 | 2022-01-07 | 福建天晴数码有限公司 | A method and system for ensuring order status of payment system order flow |
| CN114217875A (en) * | 2021-12-17 | 2022-03-22 | 平安壹钱包电子商务有限公司 | Method, device and equipment for processing order and storage medium |
| CN114237936A (en) * | 2021-12-16 | 2022-03-25 | 成都知道创宇信息技术有限公司 | Data consumption method, apparatus, server, and readable storage medium |
| CN114253747A (en) * | 2021-12-27 | 2022-03-29 | 北京宇信科技集团股份有限公司 | Distributed message management system and method |
| CN114296910A (en) * | 2021-12-02 | 2022-04-08 | 武汉物易云通网络科技有限公司 | Multi-partition queue sequential consumption method and device |
| CN114356609A (en) * | 2022-01-10 | 2022-04-15 | 政采云有限公司 | Message consumption method and message system |
| CN114461415A (en) * | 2021-12-27 | 2022-05-10 | 苏州浪潮智能科技有限公司 | Message data processing method and device, computer equipment and storage medium |
| CN114461407A (en) * | 2022-04-13 | 2022-05-10 | 杭州涂鸦信息技术有限公司 | Data processing method, device, distribution server, system and storage medium |
| CN114726809A (en) * | 2022-06-10 | 2022-07-08 | 北京沃丰时代数据科技有限公司 | Multi-tenant routing method, device, equipment, storage medium and program product |
| CN115174682A (en) * | 2022-06-17 | 2022-10-11 | 中国银联股份有限公司 | Data distribution method, system, center, device and computer readable storage medium |
| CN116132207A (en) * | 2022-09-26 | 2023-05-16 | 青岛海信智慧生活科技股份有限公司 | State data transmission method, device and electronic equipment of smart home equipment |
| CN116846897A (en) * | 2023-06-19 | 2023-10-03 | 同盾科技有限公司 | A message copying method, device, electronic equipment and readable storage medium |
| CN118277116A (en) * | 2022-12-30 | 2024-07-02 | 比亚迪股份有限公司 | Message data processing method, device, storage medium and electronic device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108874562A (en) * | 2018-06-21 | 2018-11-23 | 北京顺丰同城科技有限公司 | Distributed high concurrent message queue supplying system |
| CN110535787A (en) * | 2019-07-25 | 2019-12-03 | 北京奇艺世纪科技有限公司 | Information consumption method, apparatus and readable storage medium storing program for executing |
| WO2020238365A1 (en) * | 2019-05-31 | 2020-12-03 | 深圳前海微众银行股份有限公司 | Message consumption method, apparatus and device, and computer storage medium |
| CN112181683A (en) * | 2020-09-27 | 2021-01-05 | 中国银联股份有限公司 | Concurrent consumption method and device for message middleware |
| CN112363853A (en) * | 2020-11-10 | 2021-02-12 | 平安普惠企业管理有限公司 | Kafka system-based message publishing method, device, equipment and medium |
| CN112445626A (en) * | 2019-08-29 | 2021-03-05 | 北京京东振世信息技术有限公司 | Data processing method and device based on message middleware |
-
2021
- 2021-03-19 CN CN202110296047.7A patent/CN112988423A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108874562A (en) * | 2018-06-21 | 2018-11-23 | 北京顺丰同城科技有限公司 | Distributed high concurrent message queue supplying system |
| WO2020238365A1 (en) * | 2019-05-31 | 2020-12-03 | 深圳前海微众银行股份有限公司 | Message consumption method, apparatus and device, and computer storage medium |
| CN110535787A (en) * | 2019-07-25 | 2019-12-03 | 北京奇艺世纪科技有限公司 | Information consumption method, apparatus and readable storage medium storing program for executing |
| CN112445626A (en) * | 2019-08-29 | 2021-03-05 | 北京京东振世信息技术有限公司 | Data processing method and device based on message middleware |
| CN112181683A (en) * | 2020-09-27 | 2021-01-05 | 中国银联股份有限公司 | Concurrent consumption method and device for message middleware |
| CN112363853A (en) * | 2020-11-10 | 2021-02-12 | 平安普惠企业管理有限公司 | Kafka system-based message publishing method, device, equipment and medium |
Cited By (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113626211A (en) * | 2021-06-29 | 2021-11-09 | 欧电云信息科技(江苏)有限公司 | Switching method and switching system of message queue, electronic equipment and storage medium |
| CN113407637A (en) * | 2021-07-13 | 2021-09-17 | 上海浦东发展银行股份有限公司 | Data synchronization method and device, electronic equipment and storage medium |
| CN113626217A (en) * | 2021-07-28 | 2021-11-09 | 北京达佳互联信息技术有限公司 | Asynchronous message processing method and device, electronic equipment and storage medium |
| CN113626217B (en) * | 2021-07-28 | 2024-08-13 | 北京达佳互联信息技术有限公司 | Asynchronous message processing method, device, electronic equipment and storage medium |
| CN113835905A (en) * | 2021-09-28 | 2021-12-24 | 北京奇艺世纪科技有限公司 | Message queue load balancing method and device, electronic equipment and medium |
| CN113835905B (en) * | 2021-09-28 | 2023-09-05 | 北京奇艺世纪科技有限公司 | Message queue load balancing method and device, electronic equipment and medium |
| CN113900838A (en) * | 2021-10-20 | 2022-01-07 | 福建天晴数码有限公司 | A method and system for ensuring order status of payment system order flow |
| CN114296910B (en) * | 2021-12-02 | 2022-11-22 | 武汉物易云通网络科技有限公司 | Multi-partition queue sequential consumption method and device |
| CN114296910A (en) * | 2021-12-02 | 2022-04-08 | 武汉物易云通网络科技有限公司 | Multi-partition queue sequential consumption method and device |
| CN114237936A (en) * | 2021-12-16 | 2022-03-25 | 成都知道创宇信息技术有限公司 | Data consumption method, apparatus, server, and readable storage medium |
| CN114217875A (en) * | 2021-12-17 | 2022-03-22 | 平安壹钱包电子商务有限公司 | Method, device and equipment for processing order and storage medium |
| CN114217875B (en) * | 2021-12-17 | 2024-08-27 | 平安壹钱包电子商务有限公司 | Method, device, equipment and storage medium for processing order |
| CN114461415A (en) * | 2021-12-27 | 2022-05-10 | 苏州浪潮智能科技有限公司 | Message data processing method and device, computer equipment and storage medium |
| CN114253747A (en) * | 2021-12-27 | 2022-03-29 | 北京宇信科技集团股份有限公司 | Distributed message management system and method |
| CN114356609A (en) * | 2022-01-10 | 2022-04-15 | 政采云有限公司 | Message consumption method and message system |
| CN114461407B (en) * | 2022-04-13 | 2022-08-26 | 杭州涂鸦信息技术有限公司 | Data processing method, data processing device, distribution server, data processing system, and storage medium |
| CN114461407A (en) * | 2022-04-13 | 2022-05-10 | 杭州涂鸦信息技术有限公司 | Data processing method, device, distribution server, system and storage medium |
| CN114726809B (en) * | 2022-06-10 | 2022-09-20 | 北京沃丰时代数据科技有限公司 | Multi-tenant routing method, device, equipment, storage medium and program product |
| CN114726809A (en) * | 2022-06-10 | 2022-07-08 | 北京沃丰时代数据科技有限公司 | Multi-tenant routing method, device, equipment, storage medium and program product |
| CN115174682A (en) * | 2022-06-17 | 2022-10-11 | 中国银联股份有限公司 | Data distribution method, system, center, device and computer readable storage medium |
| CN116132207A (en) * | 2022-09-26 | 2023-05-16 | 青岛海信智慧生活科技股份有限公司 | State data transmission method, device and electronic equipment of smart home equipment |
| CN118277116A (en) * | 2022-12-30 | 2024-07-02 | 比亚迪股份有限公司 | Message data processing method, device, storage medium and electronic device |
| CN116846897A (en) * | 2023-06-19 | 2023-10-03 | 同盾科技有限公司 | A message copying method, device, electronic equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112988423A (en) | Message consumption and message distribution method, device, server and storage medium | |
| KR101400286B1 (en) | Method and apparatus for migrating task in multi-processor system | |
| US10402223B1 (en) | Scheduling hardware resources for offloading functions in a heterogeneous computing system | |
| CN109032803B (en) | Data processing method and device and client | |
| CN111460474B (en) | Method, device, memory and computer for implementing decentralization predictor | |
| CN103034540B (en) | Distributed information system and equipment thereof and coordination approach | |
| CN110753129A (en) | Message transmission method, system, apparatus, device, and computer-readable storage medium | |
| CN109145053B (en) | Data processing method and device, client, server | |
| US7243354B1 (en) | System and method for efficiently processing information in a multithread environment | |
| CN111163130A (en) | A network service system and data transmission method thereof | |
| CN111126613A (en) | Methods, apparatus and computer program products for deep learning | |
| CN110955461B (en) | Computing task processing methods, devices, systems, servers and storage media | |
| CN103034541A (en) | Distributing type information system and equipment and method thereof | |
| Wang et al. | Optimizing big data processing performance in the public cloud: opportunities and approaches | |
| CN115134373A (en) | Data synchronization method and device, storage medium and electronic equipment | |
| CN112416980B (en) | Data service processing method, device and equipment | |
| CN113971098A (en) | RabbitMQ consumption management method and system | |
| CN118140211A (en) | Hardware Accelerator Service Aggregation | |
| CN111176641B (en) | Flow node execution method, device, medium and electronic equipment | |
| CN109614242B (en) | A computing power sharing method, device, equipment and medium | |
| CN115391053A (en) | Online service method and device based on CPU and GPU hybrid calculation | |
| US12210912B2 (en) | Method and system for uniform, consistent, stateless and deterministic consistent hashing for fixed size partitions | |
| US12058201B2 (en) | Read access for computational results of a distributed network | |
| CN109032804B (en) | Data processing method and device and server | |
| US11797342B2 (en) | Method and supporting node for supporting process scheduling in a cloud system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |