CN115756770A

CN115756770A - Request operation scheduling execution method and remote procedure call system

Info

Publication number: CN115756770A
Application number: CN202211275377.9A
Authority: CN
Inventors: 李文坦; 汪翔; 沈春辉; 张为
Original assignee: Alibaba Cloud Computing Ltd
Current assignee: Alibaba Cloud Computing Ltd
Priority date: 2022-10-18
Filing date: 2022-10-18
Publication date: 2023-03-07

Abstract

The present application relates to a request operation scheduling execution method, a remote procedure call system, a computer device and a non-transitory computer readable medium. Wherein, the method comprises the following steps: acquiring request operation of a tenant, and determining the resource amount of each type of resource required by the request operation; determining the next round of resource available time of each type of resource of the tenant according to the current round of resource available time of each type of resource allocated to the tenant and the resource generation speed; allocating expected scheduling time for the request operation of the tenant, wherein the expected scheduling time is the latest time in the next round of resource available time of each type of resource; and scheduling and executing the request operation of the tenant according to the expected scheduling time of the tenant. By the method and the device, accuracy of resource flow control is improved, flow control on various resources is supported, flow control of multi-tenant isolation is supported, and use of thread resources by the flow control is reduced.

Description

Request operation scheduling execution method and remote procedure call system

Technical Field

The present application relates to the field of computers, and in particular, to a method for requesting operation scheduling execution, a remote procedure call system, a computer device, and a non-transitory computer-readable medium.

Background

In a multi-tenant scenario, each tenant has limited server resources allowed to be used at the same time, and therefore, the server resources allowed to be used by each tenant need to be subjected to flow control. The server resources are divided according to types and comprise: the CPU processing resources, the memory resources, the disk storage resources, and the like, and generally needs to complete a certain operation by matching multiple server resources, so that the tenant operation generally needs to be performed with flow control of multiple server resources.

In the flow control scheme in the related art, because a conventional flow controller can only control one resource at a time, when multiple resources are simultaneously subjected to flow control, multiple flow controllers are required to be used for carrying out flow control in series, if one resource is subjected to flow control, the flow controller waits for the current resource, and after the waiting is finished, the logic of the next resource flow controller is executed. If a resource is occupied too much to serve the user in a short time, the system may throw out an exception of insufficient resource. At this time, if the flow controllers corresponding to insufficient resources in the serially executed flow controllers are not executed at the beginning, that is, a plurality of flow controllers have been executed in front, the flow controllers already occupy the corresponding resources, and even if the resources are not actually used because the task is not executed at last, the problems of inaccurate flow control and long waiting time of the flow controllers before the resource application fails will be caused.

Disclosure of Invention

In the embodiment, a request operation scheduling execution method, a remote procedure call system, a computer device and a non-transitory computer readable medium are provided to at least solve the problem of inaccurate resource flow control in the related art.

A method for requesting operation scheduling execution comprises the following steps:

acquiring request operation of a tenant, and determining the resource amount of each type of resource required by the request operation;

determining the next round of resource availability time of each type of resource of the tenant according to the current round of resource availability time and the resource generation speed of each type of resource allocated to the tenant, wherein the next round of resource availability time is determined by the current round of resource availability time and the resource production time, and the resource production time is determined based on the resource amount and the resource generation speed;

allocating expected scheduling time for the request operation of the tenant, wherein the expected scheduling time is the latest time in the next round of resource available time of each type of resource;

and scheduling and executing the request operation of the tenant according to the expected scheduling time of the tenant.

In some embodiments, when determining the next round of resource available time of each type of resource of the tenant according to the current round of resource available time and the resource generation speed of each type of resource allocated to the tenant, if the current round of resource available time is before the current step execution time, setting the current round of resource available time as the current step execution time.

In some embodiments, when determining the next round of resource available time of each type of resource allocated to the tenant according to the current round of resource available time and the resource generation speed of each type of resource allocated to the tenant, if the current round of resource available time is before the current step execution time, setting the current round of resource available time as a target time, where the target time divides a time interval from the current round of resource available time to the current step execution time into two time periods in a set proportion.

In some of these embodiments, the method further comprises:

and under the condition that the time interval of the next round of resource available time relative to the current time exceeds a set threshold, failing to allocate the expected scheduling time scheduled to be executed for the request operation of the tenant, and rolling back the resource available time of each type of resource of the tenant to the current round of resource available time.

In some of these embodiments, scheduling execution of the request operation according to the desired time of the tenant comprises:

generating a task corresponding to the requested operation of the tenant;

taking expected scheduling time corresponding to the task and the request operation as a queue element, and adding the queue element into a tenant queue of the tenant according to the sequence of the expected scheduling time from early to late;

determining a queue group to which a tenant queue of the tenant belongs and a position in the queue group according to a time interval between an expected scheduling time in a queue element at the head of the queue in the tenant queue of the tenant and an execution time of the current step, wherein the queue group comprises an active queue group and a dormant queue group, the queue group comprises tenant queues of a plurality of tenants, and the tenant queues of the tenants are arranged according to an order from early to late of the expected scheduling time in the queue element at the head of the queue in the tenant queue;

and scheduling and executing the tasks according to the active queue group.

In some embodiments, the number of active queue groups is multiple, each of the active queue groups having a different weight value indicating a share of tasks within the corresponding active queue group that are scheduled to be executed; the scheduling execution of the task according to the active queue group comprises the following steps:

determining a probability of each active queue group being scheduled for execution according to a weight value of each active queue group, wherein the probability is determined by the sum of the share and the share;

and scheduling and executing the tasks for each active queue group according to the probability.

In some of these embodiments, the method further comprises:

deleting a tenant queue of the tenant from the active queue group and/or the dormant queue group in the event that the tenant is logged off.

In some of these embodiments, after scheduling execution of the request operation of the tenant according to the desired scheduling time of the tenant, the method further comprises:

determining allocated unused resource time of each type of resource of the request operation of the tenant if the request operation of the tenant is scheduled to execute but fails to execute;

and determining the resource available time of each type of resource of the tenant in the round according to the allocated unused resource time of each type of resource of the tenant and the resource available time of the previous round.

In some embodiments, the total number of thread resources used to schedule execution of tasks for each tenant is a set number.

In some embodiments, in a case where the request operation of the tenant is obtained through a streaming remote procedure call, for a plurality of sequential request operations obtained in one streaming remote procedure call, sequentially determining a desired scheduling time of each request operation in order, and after successfully determining a desired scheduling time of a previous request operation, determining a desired scheduling time of a subsequent request operation.

In some of these embodiments, the method further comprises:

when any request operation in a plurality of sequential request operations acquired in one streaming remote procedure call is scheduled to be executed but fails to be executed, releasing various types of resources allocated to the request operation subsequent to the request operation in the plurality of sequential request operations, and determining expected scheduling time for the request operation and the subsequent request operations again.

A computer arrangement comprising a processor and a non-transitory computer readable medium having stored therein a computer program which, when executed by the processor, performs the method described above.

A non-transitory computer-readable medium, on which a computer program is stored, which, when being executed by a processor, performs the above-mentioned method.

A remote procedure call system comprising: the remote procedure calling device and the request operation scheduling executing device comprise a resource distributor and a scheduling executing center and are used for executing the steps of the request operation scheduling executing method.

Compared with the related art, the request operation scheduling execution method, the remote process calling system, the computer device and the non-transitory computer readable medium are used for obtaining the request operation of the tenant and determining the resource amount of each type of resource required by the request operation; determining the next round of resource available time of each type of resource of the tenant according to the current round of resource available time and the resource generation speed of each type of resource allocated to the tenant, wherein the next round of resource available time is determined by the current round of resource available time and the resource production time, and the resource production time is determined based on the resource amount and the resource generation speed; allocating expected scheduling time for the request operation of the tenant, wherein the expected scheduling time is the latest time in the next round of resource available time of each type of resource; the request operation of the tenant is scheduled and executed according to the expected scheduling time of the tenant, the problem of inaccurate resource flow control in the related technology is solved, and the accuracy of the resource flow control is improved.

The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below to provide a more thorough understanding of the application.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments of the application and, together with the description, serve to explain the application and are not intended to limit the application. In the drawings:

fig. 1 is a flowchart of a request operation scheduling execution method according to the present embodiment.

Fig. 2 is a flowchart of a related art streaming RPC performing a streaming RPC task once.

Fig. 3 is a block diagram of the remote procedure call system of the present embodiment.

Fig. 4 is a flowchart of the streaming RPC of the present embodiment executing a streaming RPC task once.

Fig. 5 is a schematic diagram of a package structure of the task of the present embodiment.

Figure 6 is a flow chart of tenant queue location update of the present embodiment.

Fig. 7 is a flowchart of the resource allocator of the present embodiment allocating resources.

Fig. 8 is a schematic structural diagram of the scheduling execution center of the present embodiment.

Fig. 9 is a schematic diagram of a plurality of active queue groups of the present embodiment.

Fig. 10 is a schematic diagram of the ordered retrievable queue of the present embodiment.

Fig. 11 is a schematic diagram of queue element enqueuing and dequeuing of the tenant queue of the present embodiment.

Fig. 12 is a schematic diagram of the sleep queue group of the present embodiment.

Fig. 13 is a schematic diagram of the task queue of the present embodiment.

Fig. 14 is a diagram showing the execution thread group according to the present embodiment.

Fig. 15 is a flowchart of the operation of the execution thread of the present embodiment.

Fig. 16 is a flowchart of scheduling execution tasks by the scheduling execution center according to the present embodiment.

Detailed Description

For a clearer understanding of the objects, aspects and advantages of the present application, reference is made to the following description and accompanying drawings.

The present embodiment provides a method for performing request operation scheduling, and fig. 1 is a flowchart of the method for performing request operation scheduling according to the present embodiment. The flow can be applied to a multi-tenant system and used for scheduling resources of multiple tenants. As shown in fig. 1, the process includes the following steps:

step S101, acquiring request operation of a tenant, and determining the resource amount of each type of resource required by the request operation.

And S102, determining the next round of resource available time of each type of resource of the tenant according to the current round of resource available time and the resource generation speed of each type of resource allocated to the tenant, wherein the next round of resource available time is determined by the current round of resource available time and the resource production time, and the resource production time is determined based on the resource amount and the resource generation speed.

Step S103, allocating expected scheduling time for the request operation of the tenant, wherein the expected scheduling time is the latest time in the next round of resource available time of each type of resource.

And step S104, scheduling and executing the request operation of the tenant according to the expected scheduling time of the tenant.

In the present embodiment, the process of calculating the next resource availability time based on the current resource availability time for each type of resource is referred to as a round. The available time of each round of resources of each type of resources is independently calculated respectively and does not interfere with each other. The resource availability time of a certain type of resource refers to the latest resource availability time determined by the current time (the time point of calculating the resource availability time of the next round) of the certain type of resource. The previous round of resource availability time refers to a historical resource availability time which is closest to the current round of resource availability time and is used for calculating the current round of resource availability time. The next round of resource available time refers to a future resource available time which is closest to the current round of resource available time and is calculated by taking the current round of resource available time as a time reference.

The resources refer to server resources, and include resources related to processing and storage capabilities of the server, such as CPU processing resources, memory resources, disk storage resources, and the like.

A tenant refers to a user who purchases a rental server resource, and one server can serve multiple tenants simultaneously. Each tenant may purchase a fixed amount of server resources, and at the same time, the amount of server resources used by the user cannot typically exceed the amount that it purchased.

Taking a request operation as an example of a read request operation, if the read data size is 100KB, the resource sizes of the types of resources required by the read operation are: the bandwidth resource is 100KB, and read/write Operations (Input/Output Operations Per Second, abbreviated as IOPS) are performed 1 time Per resource. Correspondingly, if the resource generation speeds of the bandwidth resource and the IOPS resource are 100KB/s and 0.5IOPS, respectively, the current time is 00. Then for 100KB of bandwidth resources, the resource production time is 1s, and the next round of resource availability time of the bandwidth resources is 00. For IOPS resource 1, the resource production time is 2s, and the next resource available time of the IOPS resource is 00. Since the next round of resource availability time of the IOPS resource is later than the next round of resource availability time of the bandwidth resource, for the request operation of the tenant reading 100KB of data volume, its expected scheduling time is determined to be 00. That is, scheduling and executing the request operation for reading data at time 00.

It should be noted that the bandwidth resources 100KB/s and 0.5IOPS are virtual values for the sake of example, and the actual values are usually much higher. The resource generation rate is typically measured in terms of the amount of resources generated per second, such as 100KB/s as described above. In a scenario requiring higher time accuracy requirements, the measurement can be performed in milliseconds or nanoseconds. In addition, if an operation is limited to be performed several times per second, for example 3000 times, it can also be abstracted as a resource whose generation speed is, for example, 3 times/ms.

When the traditional current limiter generates the current limiter, the resource generation speed is given to the resource, the used resource amount is transmitted into the current limiter at the position where the resource is used in the task thread, if no available resource exists at the moment, the resource current limiter waits to enable the current task thread to wait until the resource is available, and the resource current limiter continues to execute. By adopting the embodiment, the resources are not required to be occupied serially by using the flow controller, thread resources are not required to be distributed for each tenant or each resource of each tenant respectively, the resources are planned in advance according to the available time of the resources, and the resources and the threads are really occupied and used when the expected scheduling time is reached, so that the flow control can be performed on various resources by adopting the request operation scheduling execution method, the scheduling of various resources can be accurately realized, and the use of the thread resources is reduced.

In a multi-tenant system, there are some time periods when the tenant has not used a certain resource, and as time passes, the current round of resource availability time of the resource may become a certain time in the past for the tenant. If the next round of resource availability of the tenant is determined directly from a past time as a starting point, the resources that have been purchased by the tenant but are not completely used are compensated, for example, if the tenant purchases a bandwidth resource of 100KB/s and the tenant does not use the bandwidth resource for 1s, the bandwidth resource that can be used for the next 1s is 200KB (equivalent to obtaining a bandwidth resource of 200 KB/s). However, in the multi-tenant system, if a large number of tenants obtain excess resources by compensating for resources that are not fully used, the resources of the system may be insufficient at the same time.

Therefore, in some preferred embodiments, when determining the next round of resource availability time, if the current round of resource availability time is before the current step execution time, the current round of resource availability time is set as the current step execution time, that is, the next round of resource availability time of the tenant is determined with the current step execution time as a starting point, so as to avoid the resource insufficiency of the system caused by the simultaneous excess of resources of multiple tenants.

In other embodiments, in order to allow a certain proportion of excess resources, when determining the next round of resource available time, if the current round of resource available time is before the current step execution time, the current round of resource available time is set as a target time, and the target time divides the time interval from the current round of resource available time to the current step execution time into two time periods with a set proportion, for example, for a time interval of 10s, the set proportion is 9:1, which means that 1/10 of unused resources are allowed as excess resources. The set ratio can be set to different values according to different systems, such as 2; in addition, different setting proportions can be set for different types of resources.

In some embodiments, the request operation is provided with a response timeout time. For example, when the time interval of the next round of resource availability time relative to the current time exceeds the set threshold, allocating the expected scheduling time scheduled for execution to the request operation of the tenant fails, and if the resource of the other types of resources has already determined that the resource of the next round of resource availability time corresponds to the current request operation, rolling back the resource availability time of each type of resource of the tenant to the resource availability time of the current round, so as to recover the unsuccessfully allocated resource, so that the scheduling execution of the other request operations of the tenant is facilitated, the unsuccessfully allocated resource does not affect the resource control of the tenant, and the accuracy of resource control under the condition of unsuccessfully allocated resource is further improved.

In some of these embodiments, the request operation is scheduled for execution by encapsulating it into a task and queuing in step S104. For example, first, a task corresponding to a request operation of a tenant is generated; then, taking expected scheduling time corresponding to the task and the request operation as a queue element, and adding the queue element into a tenant queue of the tenant according to the sequence of the expected scheduling time from early to late; determining a queue group to which the tenant queue of the tenant belongs and a position in the queue group according to a time interval between an expected scheduling time in the queue element at the head of the queue in the tenant queue of the tenant and an execution time of the current step, wherein the queue group comprises an active queue group and a dormant queue group, and the tenant queues of all tenants in the queue group are arranged according to the expected scheduling time in the queue element at the head of the queue in the tenant queue from early to late; and finally, scheduling and executing the task according to the active queue group.

In a tenant queue, a plurality of request operations of the tenant are strictly arranged from early to late according to the sequence of expected scheduling time, and the request operation with the earliest expected scheduling time is arranged at the head of the queue and is dequeued firstly.

In some embodiments, the tenant queues of which the scheduling time is expected to be earlier than the current time in the queue elements of the tenant queue head are placed in the active queue group, and the tenant queues of which the scheduling time is expected to be later than the current time in the queue elements of the tenant queue head are placed in the dormant queue group. Meanwhile, in both the active queue group and the dormant queue group, the tenant queues of all tenants are arranged from early to late according to expected scheduling time in a queue element positioned at the head of the queue in the tenant queue, so that the first task of the first tenant queue in the active queue group is scheduled and executed most preferentially, and whether the tenant queue needs to be placed in the active queue group or not can be judged in the dormant queue group according to the expected scheduling time of the first queue element of the first tenant queue.

Due to the existence of excess resources, server resources may not be available to all users at a time. At this time, if the scheduling is performed randomly, the resources of one part of users may be preferentially satisfied, and the other part of users may be allocated only few resources or even no resources at all. To address the above issues, in some embodiments, certain rules may be used to ensure that users compete fairly for their entitled resources. For example, a plurality of active queue groups are set, each active queue group has a different weight value, and the weight value is used for indicating the share of the scheduled execution of the task in the corresponding active queue group; when the task is scheduled and executed according to the active queue group, determining the probability of scheduling and executing of each active queue group according to the weight value of each active queue group, wherein the probability is determined by the sum of the share and the share; and scheduling and executing the tasks for each active queue group according to the probability.

For example, the weight values of the two active queue groups are 1 and 9 respectively, then the tasks in the first active queue group have a probability of 1/10 and are scheduled to be executed, and the tasks in the second active queue group have a probability of 9/10 and are scheduled to be executed, so that certain tenants occupy a large amount of resources at the same time, and other tenants have no resources available, and fair scheduling of multiple tenants is realized.

In some embodiments, when a tenant is logged off, the tenant queue of the tenant may be deleted from the active queue group and/or the dormant queue group, so that each type of resource allocated by the logged-off tenant is released in time, and the resource utilization rate of the entire system is improved.

Besides the expected scheduling time allocation failure and the resource release after the tenant is logged out in the above embodiments, in some embodiments, a resource returning mechanism is provided for the task that fails to be scheduled and executed, so as to further improve the accuracy of resource management and control. For example, after scheduling and executing the request operation of the tenant according to the expected scheduling time of the tenant, in the case that the request operation of the tenant is scheduled and executed but the execution fails, determining the allocated unused resource time of each type of resource of the request operation of the tenant; and determining the round resource available time of each type of resource of the tenant according to the allocated unused resource time of each type of resource of the tenant and the previous round resource available time. That is, for the allocated unused resources, the resources are returned to the tenant again, so that the task (request operation) whose scheduled execution failed and the task (request operation) whose scheduled execution succeeded have different degrees of influence on resource management and control.

The above embodiments can be applied to a remote procedure call system, that is, the request operation of each tenant can be obtained through a streaming remote procedure call or a non-streaming remote procedure call.

In the above embodiment, the total number of thread resources for scheduling and executing the task of each tenant is a set number, and the set number may be a fixed number or a variable number determined according to the total number of tenant queues in the active queue group. For example, the total number of thread resources may be any number not greater than the maximum number of tenant queues in the active queue group, may be any number less than the sum of tenant queues in the active queue group and the dormant queue group, or may be any number less than the sum of tasks to be performed in the active queue group and the dormant queue group. Compared with the related art, the embodiment adopts the set number of threads, and schedules and executes the tasks of each tenant in the active queue group one by one or according to a certain probability without setting thread resources for each tenant and each resource of each tenant and performing waiting operation of the thread resources, so that the exhaustion of the thread resources is avoided, and the use number of the thread resources is greatly saved.

In some embodiments, in a case where a request operation of a tenant is obtained through a streaming remote procedure call, for a plurality of sequential request operations obtained in one streaming remote procedure call, sequentially determining a desired scheduling time of each request operation in order, and after successfully determining a desired scheduling time of a previous request operation, determining a desired scheduling time of a subsequent request operation.

In some embodiments, since the streaming remote procedure call needs to be executed in sequence, otherwise, a correct result cannot be obtained, when any request operation in the multiple sequential request operations acquired in one streaming remote procedure call is scheduled to be executed but fails to be executed, the types of resources allocated to the request operations subsequent to the any request operation in the multiple sequential request operations are released, and a desired scheduling time is determined for the any request operation and the subsequent request operations.

The present embodiments also provide a computer arrangement comprising a processor and a non-transitory computer readable medium having stored therein a computer program which, when executed by the processor, performs the above-described method.

In particular, the processor may include a Central Processing Unit (CPU), or A Specific Integrated Circuit (ASIC), or may be configured to implement one or more Integrated circuits of the embodiments of the present Application.

Non-transitory computer-readable media may include, among other things, mass storage for data or instructions. By way of example, and not limitation, a non-transitory computer-readable medium may include a Hard Disk Drive (Hard Disk Drive, abbreviated as HDD), a floppy Disk Drive, a Solid State Drive (SSD), flash memory, an optical Disk, a magneto-optical Disk, magnetic tape, or a Universal Serial Bus (USB) Drive, or a combination of two or more of these. Non-transitory computer readable media may include removable or non-removable (or fixed) media, where appropriate. The non-transitory computer readable medium may be internal or external to the data processing apparatus, where appropriate. In a particular embodiment, the Non-transitory computer-readable medium is Non-Volatile (Non-Volatile) memory. In particular embodiments, non-transitory computer-readable media include Read-Only memories (ROMs) and Random Access Memories (RAMs). The ROM may be mask-programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically Erasable PROM (EEPROM), electrically rewritable ROM (EAROM), or FLASH Memory (FLASH), or a combination of two or more of these, where appropriate. The RAM may be a Static Random-Access Memory (SRAM) or a Dynamic Random-Access Memory (DRAM), where the DRAM may be a Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), an Extended data output Dynamic Random-Access Memory (EDODRAM), a Synchronous Dynamic Random-Access Memory (SDRAM), and the like.

The non-transitory computer readable medium may be used to store or cache various data files for processing and/or communication use, as well as possible computer program instructions executed by the processor.

The present embodiments also provide a non-transitory computer readable medium having stored thereon a computer program which, when executed by a processor, performs the method described above.

The following describes and explains the present embodiment by taking the above-described request operation scheduling execution method applied in the remote procedure call system as an example.

Remote Procedure Call (RPC) transmission is a common information interaction mode of modern network services, common RPC transmission is single request and single return, namely, a client transmits a request to a server, the server generates a result after processing and returns the result to the client, and the RPC request is ended. A single RPC request is suitable for most types of applications and is therefore most widely used. However, in the fields of data transmission and the like, one RPC request may include request information and return information with huge data volume, and using a conventional single RPC request may result in long time for transmitting and receiving information, and huge information volume, and may require all transmission and all reception at one time to process, and a long time is required for transmitting and receiving a large amount of information at one time, which increases the delay of request processing, and the request and return information transmitted at one time may exceed the memory size of the client and the server, resulting in the need of a client and the server with larger resources to process the requests.

In order to solve the deficiency of a single RPC request, the streaming RPC request is that a client transmits the request information of the RPC request to a server for one to multiple times, each transmission is an independent data packet which carries identity information indicating the current RPC request, and the server can conveniently identify the packet after receiving the data packet; after the server receives a data packet, the server can start processing immediately without waiting for all request information to be received, and the client sends subsequent request information of the RPC at the time of processing, so that information transmission and processing can be carried out simultaneously, and processing delay is reduced; after the server finishes processing the current request packet, the server can discard the request packet so as to save the memory; meanwhile, the information returned by the server can be sent in packets for multiple times as the request, and the generated advantages are consistent with the advantages of the client-side packet sending request.

The primary streaming RPC is different from continuously sending multiple common RPC requests although the primary streaming RPC is divided into multiple request packets and multiple return packets; multiple common RPC requests are continuously sent, each request is completely independent and has no relation with each other, so the sending, receiving and processing sequence and time of the requests are undefined and can be simultaneously received and processed. The method comprises the steps that a plurality of request packets and return packets of primary streaming RPC share the same context information, the sending, receiving and processing of the request packets and the return packets are executed strictly according to the sequence, the sequence of the packets received by a server is consistent with the sequence of the packets sent by a client, the next packet is processed after the server finishes processing the last packet, if the content of the streaming RPC is abandoned, only from the logic level, one RPC request is a request and one return in the strict sense, and only a physical implementation layer realizes the request and the return as a plurality of requests and a plurality of returns.

The following describes a process of one-time streaming RPC transmission and its advantages, taking file transmission as an example:

suppose 10G data needs to be acquired from the server, MD5 verification is completed, and the data is written into the local disk of the client. The MD5 checks must read and count the data content in order, and the write disks must also be written from head to tail in order.

And sending a request by the client side for 10G data to the server side by using a common RPC request. And the server reads the 10G data into the memory, then completes the calculation of the MD5, and returns the 10G data and the MD5 to the client. And after receiving the 10G data, the client starts to calculate the MD5, checks after the calculation is finished, and writes the data into a local disk of the client. The above processes are completely serial, the required time is equal to the sum of the steps, and the server and the client are required to cache temporary data in a memory of more than 10G.

For a common RPC request, the request can be split into 10 times, numbered 1 to 10, and MD5 is calculated for 1G each time. Since the order and processing sequence of the requests received by the server cannot be guaranteed, in the worst case, the server may start processing from number 10 and sequentially process to number 1, and the client needs to completely receive 10 pieces of return data before writing into the local disk. Or the server may process these 10 requests simultaneously. This still requires the server or client to have 10G of memory to process, and the disk writing process and data reception may be completely serial.

The above problem can be solved if streaming RPC requests are used. The client may send one streaming RPC request to request 10G of data, including 10 request packets, 10 return packets, each request packet requesting 1G of data. The server side can directly send the 1 st 1G return packet to the client side after finishing processing the 1 st request packet, and continues to process the 2 nd request packet, and the like. And after receiving the 1 st return packet, the client calculates the MD5, then writes the MD5 into a local disk, and waits for the 2 nd return packet. The pipeline type sending and processing mode enables the client and the server to simultaneously process the work of checking, receiving, sending and writing the disk, and the client and the server do not need to cache the data of the last packet due to the sequential receiving and sending, so that the memory and the time delay are optimized.

In a multi-tenant scenario (pooling, serverless, etc.), one server may serve many users simultaneously. For example, if 1 ten thousand users purchase the same service at the same time, each user applies for 10 units of resources. Each server has a resource that can support 1000 users simultaneously, i.e. 1 ten thousand units of resource. Ten servers may be set up at this time, each server serving 1000 users. Considering that the 1000 clients on the same server are unlikely to use up their purchased resources at the same time, and therefore there are many times that there will be a margin in the server resources, it can be considered that the amount of resources that can be used by the user is increased, and each user can allocate 20 units of resources, which provides a better customer experience, that is, "excess resources".

In a multi-tenant scenario, the amount of resources allowed to be used by each user at the same time is limited, and in the multi-tenant scenario, the resource limit of each user is 20 units, so when the resource required by the request sent by the user exceeds 20 units, the request of the user needs to be limited, for example, suspension processing, delay processing, and the like, which are called "resource limit". In the embodiment, a mode of performing current-limiting scheduling according to an expected scheduling time is adopted, and the hidden danger that the thread resource is exhausted by the conventional common mode is solved.

In this embodiment, a fair scheduling mechanism is also employed. Due to the existence of excess resources, server resources may not be available to all users at a time. At this time, if the scheduling is performed randomly, the resources of a part of users may be preferentially satisfied, and the other part of users are allocated only a few resources, so that a certain rule needs to be used to ensure that each user competes for the acquired resources fairly.

The streaming RPC is an RPC type, and has unique context information for one streaming RPC request; the client may transmit a plurality of request packets in sequence, the server may process each request packet in sequence, and after finishing post-processing on the last request packet, the server may enter the processing of the next request packet (i.e., the request packets are not processed in parallel); in the processing of each of the request packets, the server side may interact with the context of the current streaming RPC; for each request packet, one or more return packets in the existing order may be generated, and after the request packet is processed, the return packets are sent to the client in order before the next request packet starts to be processed; the client side can receive the return packets corresponding to the request packets in sequence according to the sequence of sending the request packets, and if the corresponding request packets have a plurality of return packets, the return packets are received according to the sequence appointed by the server side.

Fig. 2 is a flow of a streaming RPC framework in the related art when a streaming RPC task is executed once. The processing logic corresponding to each request packet and the logic generating the return packet are located in the green part, and the green part can be developed by service developers. It is known from the characteristics of streaming RPC that for a certain streaming RPC request, there may be multiple executions of the green part, each time corresponding to a different request packet, and they are executed in sequence and non-parallel. Service developers need to define the context of the streaming RPC, and complete the development of the service logic of the green part on the premise that multiple times of green part execution is sequential and non-concurrent in the same streaming RPC request. As for the position of context storage, the guarantee that the green part is logically sequential and non-concurrent is taken charge of by the streaming RPC framework, and service developers do not need to care about the guarantee.

Because server resources are not endless, and the same server may serve multiple tenants at the same time, service developers need to limit the speed of the server resources, even to limit different speeds for different users. Because a service developer can only develop in a green service logic development part, in order to avoid resource shortage, in the related technology, the service developer often uses a traditional current limiter, when the resource is insufficient, the traditional current limiter suspends the current thread by calling a sleep () method and the like, so that the current thread is suspended to occupy the resource, and after the resource is available, the thread is waken up to continue to execute. The related technology can greatly occupy thread resources, and reduce the service capacity of the server; the related technology needs a plurality of traditional current limiters, which can cause the problems of inaccurate current limiting and the like; there are also problems of infinite latency, thread switching, traffic non-uniformity, etc.

With the solution of the related art, when the resource of the server is insufficient, each user may not obtain the resource amount specified by the current limit, and at this time, if the access amounts of different users to the resource are different, for example, a user requests 1000M resources per second, 10000 requests are sent, another user requests 100M resources per second, and 1000 requests are sent, and at this time, the server can only provide 100M resources per second, then ideally, the ratio of obtaining the resource by different users should be controllable, for example, both users obtain 50M resources per second, or the ratio is divided according to 1.

The request operation scheduling execution method applied to the multi-tenant of the embodiment can be applied to a streaming RPC framework or a non-streaming RPC framework. The device realized based on the request operation scheduling execution method can be installed in the service flow part of the RPC framework, after the device is used, the service logic is packaged in the task object of the device, and the device is installed in the service flow part of the RPC framework. That is, for the RPC framework, it performs the functions of the device, and the device proxies the true service logic.

The device has the function of respectively carrying out flow control on different users. The device ensures that the task execution thread is not interrupted or suspended during the period from the beginning of one task execution to the completion of the task execution, thereby saving thread resources. The device supports the flow control of multiple resources simultaneously. The device can theoretically realize absolute accuracy of flow control. The device allows a developer to specify the actual resource occupation condition of the task when the task fails to be executed, so that the condition that the statistics of resource consumption is equal to the task execution success is avoided. The device has the capability of smooth flow control, ensures that a large amount of resources are not suddenly allowed to be used, and the resources are not suddenly allowed to be used, but ensures that the resources in the whole process are distributed at a smoother speed. The device will operate without thread switching overhead. The device supports the allocation of resources to different users according to a preset proportion when the resources of the server are insufficient.

Fig. 3 is a schematic structural diagram of the remote procedure call system of the present embodiment, and as shown in fig. 3, the remote procedure call system includes a remote procedure call device 30 and a request operation scheduling execution device 40, and the request operation scheduling execution device 40 includes a resource allocator 41 and a scheduling execution center 42. The remote procedure call system of the present embodiment is used to execute the request operation scheduling execution method and the steps of the preferred embodiment and embodiment thereof as shown in fig. 1.

And the remote procedure call device 30 is configured to obtain a remote procedure call packet of the tenant, and execute a preset proxy program 31 according to the remote procedure call packet, where the remote procedure call packet carries a request operation of the tenant, and the preset proxy program is configured to send the remote procedure call packet to the request operation scheduling execution device.

The resource allocator 41 is configured to determine a resource amount of each type of resource required by the request operation, determine a next resource availability time of each type of resource of the tenant according to a current resource availability time and a resource generation speed of each type of resource allocated to the tenant, and allocate a desired scheduling time to the request operation of the tenant, where the next resource availability time is determined by the current resource availability time and a resource production time, the resource production time is determined based on the resource amount and the resource generation speed, and the desired scheduling time is a latest time of the next resource availability times of each type of resource.

And the scheduling execution center 42 is used for scheduling and executing the request operation of the tenant according to the expected scheduling time of the tenant.

The execution process of the preset agent program to proxy one service flow (for non-streaming RPC, one service flow refers to one RPC request; for streaming RPC, one RPC request may have multiple request packets and multiple return packets, and one service flow refers to the processing of one request packet) is shown in FIG. 4, and mainly includes the following steps:

1. and calculating the resources required by the request according to the actual request type. For example, if the read data size is 100KB for one read request, 100KB of bandwidth is required, and 1 IOPS resource is required.

2. And (3) packaging the resources calculated in the step (1) into a resource application request, and informing the resource application request and the information of the current user to a resource distributor to apply for the resources.

3. If the resource allocation fails in the step 2 (whether the allocation is successful is judged by the resource allocator), the service process cannot be carried out due to insufficient server resources, failure information is returned to the client, and the process is finished.

4. After the resources are successfully allocated, the resource allocator informs the caller of the time that the resources can start to be used (= the scheduled expected time of the task at this time, = the expected time to be executed), encapsulates the actual service logic code into an executable object, encapsulates the executable object as a master call into a task as shown in fig. 5, transmits the task and the scheduled expected time to the scheduling execution center, and the scheduling execution center takes over and is responsible for processing the task.

4.1 optionally, if the service flow is executed successfully and information needs to be returned to the client, the code of the returned information can be packaged into an executable object, and the executable object is registered as the successful callback of the task. The successful callback is executed by the scheduling execution center after the main call is successfully executed. It should be noted that the function of the successful callback is not limited to the execution of returning information to the client, and all codes that need to be called by the task master after the execution is successful may be registered in the successful callback, and a plurality of successful callbacks may be registered, and the scheduling execution center may execute the successful callbacks in sequence.

4.2 optionally, if the task fails to execute, any code needs to be executed (such as an error is returned to the client), the corresponding code can be encapsulated into a failure callback and registered in the task. The failure callback is responsible for execution by the scheduling execution center after the main scheduling execution fails.

4.3 optionally, if the task fails to execute, the current task is considered not to actually consume the resources or the amount of consumed resources is less than the amount calculated in the step 1, and the value of the unused resources may be returned to the resource allocator and processed by the resource allocator, which is referred to as a resource returning mechanism.

5. In the dispatching execution center, the tenant queue of the corresponding user is updated, and the additional parameters are as follows: if the tenant queue group is not moved in the current update operation and the tenant queue is in the active queue group, the position of the tenant queue in the current queue group is not updated.

After the steps 1 to 5 are finished, for the streaming RPC framework, the processing of the request packet is completed, and the processing of the request packet does not send any return packet to the client. In fact, after the service logic part of the streaming RPC is proxied by the device, only the resource is requested, the request packet is encapsulated into a task, and the task is handed to the scheduling execution center. The actual task execution and the processing mode (for example, sending a return packet to the client) after the task execution are finished are executed by the scheduling execution center according to the attributes (main call, success callback and failure callback) in the task.

The resource allocator may support allocation of a plurality of resources and may set a threshold below which allocation fails when resource availability is below.

The resource allocator measures the amount of server resources using resource production speed and uses latency to represent the time required for the successful application to the resource. That is, when a new resource category is added, the resource production rate of the resource, for example, 100MB/s,200/s, etc., is input to the resource allocator, when the resource category needs to be allocated, the amount of the required resource, for example, 100mb,200, etc., is input, the resource allocator returns a time T, which represents that the resource allocator has allocated the required resource this time, and the resource applicant needs to wait for the time T and then immediately use the resource.

The resource distributor consists of two parts, namely a tenant resource use condition table and a tenant resource table. In the tenant resource use condition table, numbers (number of users × number of resource types) are stored, each number represents a time when a certain resource of one user is ready next time, that is, a resource available time (note: the resource available time can also be understood as a resource non-liability time, that is, a current resource may be borrowed in advance, but the resource is always produced at a resource production speed, when the time reaches the resource available time, the liability is completely reduced, and the resource enters a non-liability state again), and the time is an absolute value (generally, real time of the current reality) at the whole resource distributor level. The tenant resource table stores (the number of users × the number of resource types) numbers, and each number represents a limiting speed of one user for using a certain resource, i.e., a resource production speed.

The process of allocating resources by the resource allocator is shown in fig. 7, and includes the following steps.

1. The resource application party informs the resource distributor of the resource type, the resource quantity and the user identification which need to be distributed.

2. And the resource distributor takes out the tenant resource table and the tenant resource use condition table of the corresponding user.

3. For each resource that needs to be applied:

3.1 in the tenant resource use condition table, acquiring the resource available time and the resource generation speed of the resource.

3.2 if the resource availability time of the resource is the past time, setting the resource availability time as the current time.

3.3 dividing the resource application amount by the resource production speed to obtain the production time required by the resource application.

3.4 the time required by the resource production plus the time when the resource is available in the current round is equal to the time when the resource is available next time after the resource application is finished.

3.5 if the distance between the next available time of the resource and the current time exceeds the threshold value, the resource application fails, and the step 4 is directly entered, otherwise the resource application is successful.

4. And if all the resource applications are successful, taking the next resource available time in all the resources as the resource available time of the current resource application, returning the time to the calling party, and ending.

5. The resource application fails. And for each resource which is already applied in the step 3, subtracting the production time required by the application of the resource from the available time of the resource, namely recovering the modification of the resource use condition of the user in the step 3.

As shown in fig. 8, the scheduling execution center of this embodiment is composed of a plurality of active queue groups, a sleep queue group, a task queue, a scheduling thread, and an execution thread group. The queue group refers to a queue formed by a plurality of tenant queues, and the tenant queues in the queue group are arranged according to a certain sequence.

As shown in fig. 9, each of the active queue groups has a weight value, which represents the priority of a tenant queue in the queue group when performing task scheduling, and generally, the greater the weight value of the active queue group is, the more tasks in the tenant queue should be scheduled preferentially. Each tenant corresponds to a weight value, that is, if the tenant queue of each tenant is located in a certain active queue group, it should be located in the active queue group with the weight value equal to that of the tenant in the active queue groups. By assigning active queue groups with different weights, the proportion of resources which can be obtained by different tenants when requests are piled up and waiting for execution can be controlled, and the server resources cannot respond to the currently received requests.

In particular, the elements in the active queue group (i.e., tenant queue) follow a first-in-first-out rule. The tenant queues in the active queue group should all be active tenant queues. The tenant queue is active, meaning that the expected execution time of the earliest task that should be executed in the tenant queue should be earlier than or equal to the current time, i.e. the task should be executed immediately, rather than waiting.

Alternatively, the active queue group may be implemented using an ordered retrievable queue as shown in fig. 10 in this embodiment. The ordered retrievable queue consists of a first-in first-out queue (typically implemented by a linked list) and a KV map (typically implemented by a red-black tree, a hash table, a skip table).

The FIFO queue may delete an element from the queue by reference of the element on the premise of having a reference (pointer) of the element, and the method for implementing the FIFO queue by a linked list is as follows:

1. the queue maintains two pointers, the head of the queue (the next element to be dequeued), and the tail of the queue (the element that has entered the queue most recently).

2. Each element maintains two pointers to the previous and next elements, and if there is no corresponding element, the pointer points to null.

3. When an object needs to be taken out, taking out the element A corresponding to the pointer at the head of the queue, finding the element B pointed by the next element pointer in the element A, if the element B does not exist, indicating that the queue is empty, and pointing the two pointers at the head and the tail of the queue to be empty; if element B is present, the queue head pointer is pointed to B and the last pointer of element B is pointed to null. The object saved in element a is then fetched.

4. When an object needs to be put in, generating a new element C, putting the object into the element C, finding a queue tail pointer, if the queue tail pointer is empty, indicating that no element exists in the queue, pointing the head pointer and the tail pointer of the queue to the element C, and ending; otherwise, the next element pointer of the element D pointed by the queue tail pointer points to the newly generated element C, and the last pointer of the element C points to the element D. The queue tail pointer is then pointed to element C.

5. On the premise that the element reference E is obtained, the method for deleting the element in the queue comprises the following steps: and if the pointer of the last element of the element E is not null, finding the last element F of the element E, and similarly, finding the next element G of the element E. If F, G is empty, it indicates that element E is the only element in the queue, and points the head and tail pointers of the queue to empty, and ends; if only F is empty, the element E is indicated to be at the head of the queue, the last pointer of the element G is set to be empty, the pointer at the head of the queue points to the element G, and the operation is finished; if the element G is only empty, the element E is indicated to be at the tail of the queue, the next pointer of the element F is set to be empty, the pointer at the tail of the queue points to the element F, and the operation is finished; if F, G is not empty, element E is in the middle of the queue, pointing the next pointer to F to G, pointing the last pointer to G to F, and ending.

The ordered retrievable queue can realize the first-in first-out function of objects and can quickly delete corresponding objects in the queue according to Key. The operation mode is as follows: after a new object is placed in the queue each time, a Key is set for the object (for this example, the tenant queue is an object, and the Key may be an ID of the tenant), and Value pointed by the Key is an element generated by the fifo queue when the object is inserted this time. Key and Value are associated through KV mapping technology, so that the ability of rapidly positioning to Value through Key can be realized. When a certain element needs to be deleted, the reference of the element where the corresponding object is located is found through the corresponding Key, and then the object is deleted from the queue by using a method of deleting the element by using the first-in first-out queue.

The active queue group is realized through the ordered retrievable queue, so that when the tenant logs off, the server can quickly clean the tenant queue corresponding to the tenant, and the tenant queue is prevented from continuously occupying system scheduling resources.

As shown in fig. 11, the tenant queue is a minimum heap, each tenant queue is used by only one tenant, and each tenant also has only one tenant queue. The objects in the queue are doublets consisting of a task and the expected scheduled time for that task as shown in FIG. 5. The task which is taken out first in the tenant queue is certainly the task which is expected to be scheduled with the earliest time in all tasks in the tenant queue, and after a new element is added to the tenant queue, all objects in the tenant queue are arranged in order according to the time which the task is expected to be scheduled.

Alternatively, if the expected scheduling time corresponding to the task that can be put into the tenant queue is guaranteed to be the latest of all tasks in the current tenant queue, the tenant queue can be implemented by using a common first-in first-out queue.

The sleep queue group is a minimum heap, as shown in fig. 12, tenant queues in the sleep queue group are strictly ordered according to the order of the tenant queues, and tenants that need to be executed as soon as possible (queue head element time is earliest) should first come out of the queue before going forward in the queue. This capability can be achieved with a minimum of heaps.

The task queue is a first-in first-out queue, the elements stored in the task queue shown in fig. 13 are tasks shown in fig. 5, and each task in the task queue is a task that should be scheduled immediately.

As shown in FIG. 14, the execution thread group includes one or more threads, and the work of each thread is to take out a task from the task queue, then execute, and if there is no task in the task queue, sleep until there is a task in the task queue.

In this embodiment, the number of threads that actually execute a task may be a set number. The set number may be a fixed number or a variable number. For example, the total number of thread resources may be any number not greater than the maximum number of tenant queues in the active queue group, may be any number less than the sum of tenant queues in the active queue group and the dormant queue group, or may be any number less than the sum of tasks to be performed in the active queue group and the dormant queue group. These threads typically remain active all the time and cyclically fetch tasks from the active set of queues. If there is no task, the thread will sleep until the next time there is a task to execute. Although in the present embodiment, the thread has been created before the time when the resource is available, the number of thread resources is smaller than the sum of the numbers of all tasks waiting to be executed, and the number of thread resources is not increased without limitation due to the increase in the number of all tasks waiting to be executed, so that the thread resources are not easily depleted.

The flow of executing each thread of the thread group is shown in fig. 15, when a task is found in the task queue, the task is taken out and the primary call of the task is executed first, if the primary call is executed successfully, a successful callback is executed (for example, a return packet is sent to the client, etc.), if the execution fails, a failed callback is executed (for example, resources are returned, failure information is sent to the client, etc.), then the next task is continuously searched in the task queue, and the process is repeated.

The scheduling thread is responsible for maintaining the tenant queue, the set of active and dormant queues. If the previous tenant queue in the sleep queue group has an active condition (there is a task that needs to be scheduled soon), it is moved into the corresponding active queue group for scheduling. And if the tasks in the active queue group need to be immediately scheduled, selecting the corresponding tasks to be put into the task queue so as to execute the thread group to acquire the tasks and execute the tasks. The specific process is shown in fig. 16.

With reference to fig. 16, each time it is woken up, the scheduling thread first checks whether there are more tenant queues in the active queue group (1).

If the tenant queue exists in the active queue group, the fact that tasks need to be scheduled immediately is shown in the step (1), at this time, weights of all active queue groups are summed (each active queue group has one weight, the weights of all tenant queues included in the active queue group are equal to the weights of the active queue group, the larger the weight is, the more tasks in the tenant queue are scheduled preferentially, the weights are integers, are discrete and are limited in number), the sum is set to be S, and a random integer R is generated and located between 1 and S (including two sides). The total number of the active queue groups is N, S integers in an interval [1,S ] are divided into N intervals according to the weight proportion of the active queue groups, and then the N queues are divided into corresponding intervals according to the weight proportion. For example, there are 3 active groups of groups, the weights are 1,5, 7 respectively, S =1+5+7=13, the number 1 is assigned to group 1 (weight 1), the numbers 2 to 6 are assigned to group 2 (weight 5), and the numbers 7 to 13 are assigned to group 3 (weight 7). Then, looking at which group the random number R is allocated to, the task scheduled this time is selected from the tenant queue Q at the head in the active queue group. As can be seen from the above, the greater the weight, the greater the probability of being scheduled preferentially. After the tenant queue Q is selected, the element at the head of the tenant queue Q is taken out, wherein the element consists of the time and the task which are expected to be scheduled, and the time which is expected to be scheduled is definitely earlier than the current time. Taking out the tasks in the elements, putting the tasks into a task queue, and then updating a tenant queue Q, wherein the parameters are as follows: if the queue group is not moved and is located in the active queue group, then the location of the tenant queue within the current queue group is "updated". And then the steps continue back to the time the thread is awakened.

And (3) if the step (1) shows that no tenant queue exists in the active queue group, the scheduling thread checks the dormant queue group. As can be seen from the process of updating tenant queues shown in fig. 6, the earlier tenant queue in the sleep queue group has the earliest expected time that the task at the head of the tenant queue group needs to be scheduled, so that it is only necessary to see whether there is a task at the tenant queue at the head of the sleep queue group that needs to be scheduled immediately, and if the expected scheduled time of the task at the head of the tenant queue group that needs to be scheduled is all the future, tasks in other tenant queues in the sleep queue group are also necessarily expected to be scheduled in the future. And circularly taking out the tenant queue at the head of the sleep queue group, and putting the tenant queue into the active queue group corresponding to the tenant weight until the time when no tenant queue exists in the sleep queue group or the first element of the tenant queue at the head is in the future. Then, find the expected scheduled time T (i.e., the time of the first element of the head tenant queue of the sleep queue group) of the task for which the sleep queue group earliest needs to be scheduled, sleep the scheduled thread until that time T (during which the scheduled thread may be woken up by other threads). The next task that needs to be scheduled should be scheduled at time T before no new tenant queue is updated, so the scheduling thread has no things to do until time T if no new tenant queue is updated.

The process of updating the tenant queue is shown in fig. 6, which receives a parameter "if the queue group is not moved and located in the active queue group," update "or not update" the location of the tenant queue in the current queue group, and there are two parameters "update" and "not update". The purpose of updating the tenant queue is to move the tenant queue into the correct set of queues.

In connection with fig. 6, assume that the tenant queue to be updated is Q. First, the tenant queue Q is obtained, and the time of the head element of the tenant queue is obtained, as can be seen from the above, the time is the earliest time expected to be scheduled among all tasks to be executed in the tenant queue. If this earliest expected scheduled time is in the past, the tenant queue should be in the active queue group at this time because there are tasks in the tenant queue that need to be scheduled immediately; if this earliest time expected to be scheduled is in the future, the tenant queue should be in the sleep queue group because the tenant queue has no tasks that need to be scheduled immediately.

In some embodiments, it may also be allowed to set a certain time threshold, for example, if the aforementioned earliest time expected to be scheduled is in the future, but the time interval from the current time is less than a set time threshold (for example, 300 ms), the tenant queue may also be added to the active queue group to wait for scheduling execution. This set time threshold controls the burst speed that the tenant can achieve. For example, if the speed limit of the user is 100MB/s, if our threshold is set to 0ms, assuming that the user receives a 1MB request, it takes 10ms to produce 1MB, and the request is put into the sleep queue immediately, and after at least 10ms, the 1MB request will be executed. If the threshold is 300ms, then the request can be executed immediately (corresponding to a portion of the request being released for execution first). After the flow is stabilized, the threshold value does not affect the overall speed limit.

Next, it is checked whether the tenant queue is currently located in the correct queue group (2), for example, if the tenant queue should be located in the dormant queue group but is currently located in the active queue group, the queue group where the tenant queue is located is incorrect. If the tenant queue is located in the correct queue group and currently located in the active queue group, it needs to determine an input parameter, "if the queue group is not moved and located in the active queue group," whether to update "the position of the tenant queue in the current queue group," if the parameter is "update", the tenant queue is taken out of the corresponding active queue group and then put in, and the property of first-in first-out of the active queue group is known, and the operation is equivalent to placing the tenant queue at the tail of the corresponding active queue group. And (3) if the tenant queue is judged not to be positioned in the correct queue group in the step (2), firstly taking out the tenant queue from the current queue group, and then putting the tenant queue into the corresponding correct queue group. If the tenant queue Q is just located in the sleep queue group and located at the head of the sleep queue group after the tenant queue group is adjusted (as above, the sleep queue group is strictly arranged from small to large according to the earliest expected scheduled time of tasks in the tenant queue), then if the scheduling thread is sleeping at this time, the scheduling thread sleeping deadline may be later than the earliest expected scheduled time of tasks in the tenant queue Q, so that the scheduling thread needs to be awakened, so as to prevent the tasks in the tenant queue Q from not being scheduled in time. Optionally, the sleep time of the scheduled thread may also be adjusted to the earliest expected scheduled time of the task in the tenant queue Q by modifying the time, instead of directly waking up the scheduled thread.

In summary, the above-mentioned embodiments or preferred embodiments of the present application have the following advantages over the related art:

the streaming transmission framework such as GRPC solves the problem of streaming transmission, but does not relate to the work related to multiple tenants, cannot limit the resources of the multiple tenants, and is even impossible to carry out fair scheduling. The scheme supports multiple tenants, and the scheduling execution center supports multiple tenant queues and multiple active queue groups for scheduling respectively. The resource distributor stores resource tables of a plurality of tenants. For the fair scheduling problem, each tenant has a weight, when tasks are to be scheduled in a tenant queue of the tenant, the tenant enters an active queue group corresponding to the weight, when server resources are insufficient, request backlog occurs, scheduling is performed according to the weight proportion, and the tasks of the tenants with large weights have more opportunities to be scheduled.

2. For flow control including but not limited to multi-tenancy, it is common in the industry to achieve this by throttling within the traffic thread. For example, in a certain RPC request, the server side has insufficient resources during execution, and the server side suspends the current thread by using a function similar to a sleep () class, and resumes the thread execution after the resources are sufficient. The method has the problems that after the requests are subjected to flow control each time, the service threads for executing the requests cannot be released, and when a large number of requests are limited in speed, a server end has a large number of threads in a sleep () state, which may cause the resource of the server threads to be exhausted, and cause the server to be down. In the scheme, after the request is subjected to flow control, the request cannot be scheduled, so that the thread resource is occupied, and the server thread resource cannot be exhausted even if a large number of requests are blocked. In the scheme, a request may consume multiple resources, such as QPS/IOPS/Bandwidth, and simultaneously apply for the multiple resources, the resource with the longest waiting time is taken as the time that the task needs to wait for scheduling the current task, and when the waiting time is not up, the task is not scheduled, so the task does not occupy the thread. The scheme is completely free of sleep operation in the task scheduling process, and a large number of threads are not needed to deal with a large number of tasks waiting for scheduling.

3. When flow control is performed, the general flow control can only control one resource, for example, control some index such as current flow and QPS. The scheme supports the scheduling of various resources.

4. In the conventional flow control scheme, when multiple resources are simultaneously subjected to flow control, multiple flow controllers (for example, the 3 rd flow controller can only control one resource at a time) are required to be used to perform flow control in series, and if one resource is subjected to flow control, the current resource flow controller waits for the current resource flow controller, and after the current resource flow controller waits for the current resource flow controller, the logic of the next resource flow controller is executed. If a resource is occupied in a large amount and the user cannot be served in a short time, the system may throw out an exception of insufficient resource. In the conventional scheme, if the flow controller corresponding to the insufficient resource is not executed at the beginning, and a plurality of flow controllers are already executed in front, the flow controllers in front already consume the corresponding resource, even if the resource is not really used because the task is not executed, which results in inaccurate flow control. In the scheme, due to the fact that waiting is not needed, only one flow controller is provided, all resources can be applied in the single flow controller at one time, if a certain resource is insufficient, the task cannot be executed, the whole resource application fails, any resource cannot be occupied, and therefore the flow controller is more accurate. In addition, when the resources are insufficient, the scheme can rapidly give a failure prompt to rapidly inform the user of the failure, in the conventional scheme, the front flow controller may perform a sleep () waiting operation, so that when the flow controller actually enters the resources and returns to the failure, the flow controller waits for a long time, although the task is not really executed. The scheme supports simultaneous scheduling of various resources, supports returning of allocated resources when the task execution fails, and does not perform sleep operation of threads.

5. In the traditional flow control scheme, the resource application is successful, but if the execution of the subsequent task fails, the resource is already applied, so the resource is already consumed, which affects the accuracy of flow control. In the scheme, the application of the resources can be withdrawn through the failure callback, so that the flow control is more accurate.

6. Since the conventional current limiter generates thread switching (sleep) every time the current is limited, the conventional current limiter has overhead every time the current limiting operation is triggered, in order to reduce the overhead, time is divided into time slices in the conventional current limiter, each time slice grants an available resource amount corresponding to the length of the time slice (for example, the current is limited by 100MB per second, the resource amount of one time slice is 100MB if the time slice is 1 second), when the operation of attempting current limiting is called externally, the current limiter checks whether resources exist in the current time slice, if so, the corresponding resources are consumed, no current limiting is performed, and if not, the current limiter starts sleeping to the next time slice. For the reason of thread switching, the time slice should not be set too short, otherwise, the thread switching is too much, and should not be set too long, because the too long time slice has a larger resource amount, that is, a larger resource amount can be allocated in a moment, then the resource of the time slice is used up, and all the subsequent requests enter a long-time waiting, which shows that the flow is suddenly large and suddenly none. Moreover, if a time slice resource of the current limiter is smaller than a resource applied for a certain time, the application can never be applied for the time, and infinite waiting is caused. The above problems are solved by the present solution.

It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to be limiting. All other embodiments, which can be derived by a person skilled in the art from the examples provided herein without any inventive step, shall fall within the scope of protection of the present application.

It is obvious that the drawings are only examples or embodiments of the present application, and it is obvious to those skilled in the art that the present application can be applied to other similar cases according to the drawings without creative efforts. Moreover, it should be appreciated that in the development of any such actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which may vary from one implementation to another.

The term "embodiment" is used herein to mean that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is to be expressly or implicitly understood by one of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

Unless defined otherwise, technical or scientific terms used herein shall have the same general meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The use of the terms "a" and "an" and "the" and similar referents in the context of this application do not denote a limitation of quantity, either in the singular or the plural. The terms "comprises," "comprising," "has," "having," and any variations thereof, as referred to in this application, are intended to cover non-exclusive inclusions; for example, a process, method, and system, article, or apparatus that comprises a list of steps or modules (elements) is not limited to the listed steps or modules, but may include other steps or modules (elements) not listed or inherent to such process, method, article, or apparatus. Reference throughout this application to "connected," "coupled," and the like is not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. Reference to "a plurality" in this application means two or more. "and/or" describes an association relationship of associated objects, meaning that three relationships may exist, for example, "A and/or B" may mean: a exists alone, A and B exist simultaneously, and B exists alone. In general, the character "/" indicates a relationship in which the objects associated before and after are an "or". The terms "first," "second," "third," and the like in this application are used for distinguishing between similar items and not necessarily for describing a particular sequential or chronological order.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the patent protection. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for requesting operation scheduling execution comprises the following steps:

allocating an expected scheduling time for the request operation of the tenant, wherein the expected scheduling time is the latest time in the next round of resource availability time of each type of resource of the tenant;

scheduling the request operation of the tenant according to the expected scheduling time of the tenant.

2. The method according to claim 1, wherein when determining the next round of resource availability time of each type of resource of the tenant according to the present round of resource availability time and the resource generation speed of each type of resource allocated to the tenant, if the present round of resource availability time is before the current step execution time, the present round of resource availability time is set as the current step execution time.

3. The method according to claim 1, wherein when determining the next round of resource availability time of each type of resource of the tenant according to the current round of resource availability time and the resource generation speed of each type of resource allocated to the tenant, if the current round of resource availability time is before the current step execution time, the current round of resource availability time is set as a target time, wherein the target time divides a time interval from the current round of resource availability time to the current step execution time into two time periods with a set proportion.

4. The method of claim 1, further comprising:

and under the condition that the time interval of the next round of available resources relative to the current time exceeds a set threshold, failing to allocate the expected scheduling time scheduled to be executed for the request operation of the tenant, and rolling back the resource available time of each type of resource of the tenant to the current round of available resources.

5. The method of claim 1, wherein performing the request operation according to the desired time schedule of the tenant comprises:

generating a task corresponding to the requested operation of the tenant;

and scheduling and executing the tasks according to the active queue group.

6. The method of claim 5, wherein the active queue groups are multiple, and each of the active queue groups has a different weight value indicating a share of a task scheduled to be executed within the corresponding active queue group; the scheduling execution of the task according to the active queue group comprises the following steps:

7. The method of claim 5, further comprising:

8. The method of claim 1, wherein after scheduling the request operation of the tenant for execution according to the desired scheduling time of the tenant, the method further comprises:

and determining the round resource available time of each type of resource of the tenant according to the allocated unused resource time of each type of resource of the tenant and the previous round resource available time.

9. The method of claim 5, wherein the total number of thread resources used to schedule execution of the tasks of each tenant is a set number.

10. The method according to claim 1, wherein in a case that the request operation of the tenant is obtained through a streaming remote procedure call, for a plurality of sequential request operations obtained in one streaming remote procedure call, the expected scheduling time of each request operation is sequentially determined in order, and after the expected scheduling time of the previous request operation is successfully determined, the expected scheduling time of the next request operation is determined.

11. The method of claim 10, further comprising:

12. A computer arrangement comprising a processor and a non-transitory computer readable medium, in which a computer program is stored, which computer program, when executed by the processor, performs the method of any of claims 1 to 11.

13. A non-transitory computer-readable medium, on which a computer program is stored, which, when executed by a processor, performs the method of any one of claims 1 to 11.

14. A remote procedure call system, comprising: remote procedure call means and request operation scheduling execution means, said request operation scheduling execution means comprising a resource allocator and a scheduling execution center, for performing the method of any of claims 1 to 11.