CN113837660B

CN113837660B - Driving scheduling method, medium and electronic equipment

Info

Publication number: CN113837660B
Application number: CN202111224956.6A
Authority: CN
Inventors: 彭剑; 蔡序庆; 陈媛媛; 李勤英
Original assignee: Shanghai Ouyeel Logistics Co ltd
Current assignee: Shanghai Ouyeel Logistics Co ltd
Priority date: 2021-10-21
Filing date: 2021-10-21
Publication date: 2024-08-13
Anticipated expiration: 2041-10-21
Also published as: CN113837660A

Abstract

The application relates to the technical field of logistics, and discloses a driving scheduling method, a medium and electronic equipment. The method is realized based on a driving scheduling model, the electronic equipment performs multiple times of simulated scheduling on a plurality of orders to be delivered by using the driving scheduling model, a plurality of simulated scheduling schemes are generated, and a scheme with the highest return value of each order in the plurality of simulated scheduling schemes is used as a delivery scheme. And during the period, the electronic equipment completes the simulated scheduling once, namely, the parameters of the driving scheduling model are updated according to the return value and the evaluation value of each order in the simulated scheduling, and the next simulated scheduling is carried out according to the updated parameters. By the crane scheduling method, waiting time of the crane in the process of delivering goods to be delivered can be reduced, the delivering efficiency of the crane for delivering the goods to be delivered can be improved, the steel finished product warehousing operation cost can be reduced, and the overall warehousing operation efficiency can be improved.

Description

Driving scheduling method, medium and electronic equipment

Technical Field

The application relates to the technical field of logistics, in particular to a driving scheduling method, a medium and electronic equipment.

Background

With the rapid development of the logistics industry, the requirements of people on the aging of logistics are higher and higher. In order to improve the speed of warehouse entry/exit of goods in a warehouse, intelligent robots are adopted in the existing warehouse to sort, store and transport the goods in the warehouse. However, for a warehouse for storing heavy goods, such as a warehouse for storing steel coils, due to the high cost of developing an intelligent robot capable of transporting heavy goods such as steel coils, a plurality of travelling cranes are still used to transport the goods in the warehouse to a goods-loading area for delivery or from a warehouse entrance to a warehouse area for storage. However, in the process of carrying out goods warehouse entry/warehouse exit through the travelling crane, the travelling crane body can only move along the same direction, so that the travelling crane is often required to wait for other travelling cranes to finish goods transportation in the process of transporting goods to carry out next goods, part of travelling cranes are idle in the process of transporting goods for warehouse exit, and the efficiency of goods warehouse exit is reduced. Therefore, how to better determine the delivery sequence of each cargo in the warehouse and the travelling crane for transporting the cargo so as to improve the delivery efficiency of the travelling crane is a problem to be solved.

Disclosure of Invention

In view of the above, the embodiment of the application provides a driving scheduling method, a medium and electronic equipment. Multiple times of simulated scheduling is carried out on the order to be delivered based on a driving scheduling model to generate multiple simulated scheduling schemes, and the optimal one of the multiple simulated scheduling schemes is used as a scheme for scheduling the driving to deliver the goods, so that waiting time of the driving in the delivering process of delivering the goods can be reduced, and delivering efficiency of the driving for delivering the goods is improved.

In a first aspect, an embodiment of the present application provides a driving scheduling method, which is applied to an electronic device, and the method includes: performing simulated scheduling on a plurality of orders to be delivered by using a first scheduling model, and generating a first scheduling scheme and a first return value of each order; evaluating the first scheduling scheme, generating an evaluation result of each order, and determining a second scheduling model based on the first return value and the evaluation result of each order; performing simulated scheduling on a plurality of orders to be delivered by using a second scheduling model to generate a second scheduling scheme and second return values of the orders, wherein the sum of the second return values is larger than that of the first return values; and dispatching the travelling crane according to the second dispatching scheme to carry out delivery on the order to be delivered.

The electronic device may perform multiple simulated scheduling on multiple orders to be delivered by using a first scheduling model, for example, the following driving scheduling model 200, to generate multiple simulated scheduling schemes, and schedule the driving order to deliver according to one simulated scheduling scheme with the largest cumulative return value of each order in the multiple simulated scheduling schemes. In the process of carrying out simulated scheduling on a plurality of orders to be delivered through the driving scheduling model 200, the electronic equipment adjusts network parameters of the driving scheduling model 200 according to the return values and evaluation results of the orders in the current scheduling every time in the process of carrying out simulated scheduling on the orders to be delivered through the driving scheduling model 200, and carries out simulated scheduling by using new network parameters in the next simulated scheduling, so that a scheduling scheme with better sum of the return values of the orders can be obtained.

By the crane scheduling method provided by the embodiment of the application, the waiting time of the crane in the process of delivering the goods to be delivered can be reduced, the delivering efficiency of the crane for delivering the goods to be delivered can be improved, the steel finished product warehousing operation cost can be reduced, and the overall warehousing operation efficiency can be improved.

In combination with the first possible implementation manner of the first aspect, the determining the second scheduling model based on the first return value and the evaluation result of each order includes: and adjusting the parameters of the first scheduling model to the parameters of the second scheduling model according to the first return value and the evaluation result of each order.

That is, in the embodiment of the present application, the second scheduling model is obtained by updating the parameters of the first scheduling model, for example, in the case that the first scheduling model is the driving scheduling model 200, the second scheduling model may be obtained based on updating the parameters of the driving scheduling model 200 by the method in step S506 below.

In a second possible implementation manner with reference to the first possible implementation manner of the first aspect, the first scheduling model includes a task scheduling network and a scheduling evaluation network.

That is, the first scheduling model includes two networks for implementing different functions, for example, the following driving scheduling model 200 includes a task scheduling network 201 and a scheduling evaluation network 202, the task scheduling network 201 is used for performing simulated scheduling on orders to be taken out of a warehouse and generating return values of the orders, the scheduling evaluation network 202 is used for evaluating scheduling of the task scheduling network 201 and generating evaluation results of the orders, so that the driving scheduling simulation 200 can adjust parameters of the task scheduling network 201 and the scheduling evaluation network 202 according to the return values and the evaluation results of the orders, and obtain a scheduling scheme with a larger sum of the return values of the orders in the next simulated scheduling process.

In a third possible implementation manner with reference to the second possible implementation manner of the first aspect, the first scheduling scheme is generated by a task scheduling network.

In a fourth possible implementation manner of the third possible implementation manner of the first aspect, the task scheduling network generates the first scheduling scheme by performing the following operations in a loop: and determining the strategy gradient of each order of the undetermined delivery order, and taking the one with the largest strategy gradient as the next delivery order.

For example, referring to the method in step S502 below, the task scheduling network may calculate a policy gradient for each order for which the order of delivery has not been determined, take the one with the largest policy gradient as the order for the next delivery, and mark the order as the order for which the order of delivery has been determined (updated scheduling parameters as follows).

In a fifth possible implementation manner of the second possible implementation manner of the first aspect, each of the first return values is generated by a scheduling and evaluating network, where the scheduling and evaluating network includes a rule for predicting waiting time of each driving when a plurality of orders to be taken out of a warehouse are taken out of the warehouse.

That is, the scheduling and evaluating network (for example, the scheduling and evaluating network 202 below) may be a pre-trained network, and is configured to take as input a job-out task parameter described below, output a predicted value of a driving waiting time corresponding to the job-out task parameter (hereinafter referred to as a predicted value), and use a difference value of the predicted value corresponding to the job-out task parameter before determining a order as an evaluation result of the order by subtracting the predicted value corresponding to the job-out task parameter before determining the order from the predicted value corresponding to the job-out task parameter after determining the order.

In the pre-training process of the scheduling evaluation network 202, a preset scheduling scheme may be used as an input, and an ideal waiting time of each driving in the preset scheduling scheme is used as a training target, for example, 0, so as to fit a rule for calculating a predicted value corresponding to the parameters of the outbound task.

In combination with the first aspect and any one of the possible implementation manners of the first aspect, the second scheduling scheme includes a delivery order of each order, a driving identifier for executing each order, a starting point position and a target position of a cargo corresponding to each order.

In combination with the sixth possible implementation manner of the first aspect, the scheduling the driving according to the second scheduling scheme to take out the order to be taken out includes: and sending a delivery instruction to a vehicle executing each order according to the delivery sequence of each order and the vehicle identifier of the executing each order.

In an eighth possible implementation manner of the seventh possible implementation manner of the first aspect, when an order to be delivered is changed, the first scheduling model is used to perform simulated scheduling on the order to be delivered, so as to generate a first scheduling scheme and a first return value of each order.

That is, in the embodiment of the application, when the electronic device detects that the order to be delivered has a change, for example, when part of the order is cancelled and part of the order is newly added, the electronic device can perform simulated scheduling on the order to be delivered again, so that the running scheduling scheme can be adjusted in real time according to the state of the order to be delivered, and the efficiency of delivering goods by running is further improved.

In a second aspect, an embodiment of the present application provides a readable medium having stored thereon instructions that, when executed on an electronic device, cause the electronic device to implement any one of the methods for driving scheduling provided in the first aspect and possible implementations thereof.

In a third aspect, an embodiment of the present application provides an electronic device, including: a memory for storing instructions for execution by one or more processors of the electronic device; and a processor, which is one of the processors of the electronic device, for executing the instructions stored in the memory to implement any one of the driving scheduling methods provided in the first aspect and possible implementations thereof.

Drawings

FIG. 1 illustrates a schematic view of a scenario in which cargo is transported by driving a vehicle for delivery, according to some embodiments of the application;

FIG. 2 illustrates a schematic view of a scenario of driving schedule, according to some embodiments of the application;

FIG. 3 illustrates a schematic diagram of a task scheduling network 201, according to some embodiments of the application;

FIG. 4 illustrates a flow diagram of a method of scheduling traffic, according to some embodiments of the application;

FIG. 5 illustrates a schematic diagram of a process by which a traffic scheduling model 200 determines a scheduling scheme, according to some embodiments of the application;

Fig. 6 illustrates a schematic diagram of an electronic device 100, according to some embodiments of the application.

Detailed Description

Illustrative embodiments of the application include, but are not limited to, a traffic scheduling method, medium, and electronic device.

The following describes the technical scheme of the embodiment of the application with reference to the attached drawings.

Fig. 1 illustrates a schematic view of a scenario in which cargo is delivered by crane for delivery, according to some embodiments of the application.

As shown in fig. 1, a warehouse 00 is provided with a traveling crane 10 and a traveling crane 20, a vehicle body (cart) of each traveling crane, for example, a vehicle body 11 of the traveling crane 10 can move in the Y direction, and a cart 12 of each traveling crane, for example, the traveling crane 10, can move in the X direction; the lifting mechanism, such as the lifting mechanism 13 of the crane 10, provided on the trolley of each crane can move in the Z direction. The electronic device 100 is coupled to the crane 10 and the crane 20, respectively, so that the electronic device 100 can control the crane 10 and the crane 20 to move the goods from the storage area 01 to the transport vehicle of the loading area 02, thereby realizing the delivery of the goods from the storage area 01.

When the delivery of the goods corresponding to the plurality of orders from the warehouse 00 is required to be carried through the travelling crane 10 and the travelling crane 20, the delivery sequence of each order and the delivery of the goods corresponding to each order are usually determined manually, but if the travelling crane collides with the travelling paths of other travelling cranes in the delivery process of the goods, the goods can be delivered continuously after the delivery of the goods is completed by the other travelling cranes. For example, referring to fig. 1, when the carriage 20 is positioned at the position of the Y-coordinate 2 and the transported goods move in the negative direction of the Y-axis toward the loading area 02, and the carriage 10 is positioned at the position of the Y-coordinate 1 and the transported goods move in the positive direction of the Y-axis toward the position of the Y-coordinate 2, collision occurs if the carriage 20 and the carriage 10 continue to move toward each other. To avoid the collision, the crane 10 needs to retract to the waiting area and wait for the crane 20 to return to the position with the Y coordinate of 3 after transporting the goods to the goods-transporting area 02 and then move to the position with the Y coordinate of 2. The delivery sequence of each order and the goods travelling crane corresponding to each order determined by a manual mode cannot effectively avoid collision between travelling cranes, and the order with the shortest travelling crane waiting time cannot be selected according to the path of the goods currently transported by each travelling crane, so that the delivery efficiency of the goods transported by the travelling crane is lower.

In order to solve the above-mentioned problems, the embodiment of the present application provides a driving scheduling method, specifically, the driving scheduling method is implemented based on a driving scheduling model (first model), where the driving scheduling model (for example, the driving scheduling model 200 below) may perform multiple simulated scheduling on the order to be delivered, to generate multiple simulated scheduling schemes, and take an optimal one of the multiple simulated scheduling schemes as the delivering scheme, for example, determine, as the delivering scheme, one of the simulated scheduling schemes that minimizes the waiting time of each driving, so as to reduce the waiting time of the driving 10 and the driving 20 in the delivering process of the goods, improve the efficiency of delivering the goods, and reduce the overall storage operation cost of the finished goods.

For example, fig. 2 illustrates a schematic view of a scenario of driving schedule, according to some embodiments of the application.

Referring to fig. 2, after receiving the order to be delivered, the electronic device 100 may generate delivery task parameters based on specific information of each order to be delivered, including the number of orders to be delivered, a start position of each order, a target position of each order, an initial position of each vehicle in the warehouse 00, an area where each vehicle in the warehouse 00 can reach in the warehouse 00, and the like, and transmit the delivery task parameters to the vehicle scheduling model 200. After the driving scheduling model 200 receives the ex-warehouse task parameter, the e-commerce order to be ex-warehouse received by the electronic equipment can be subjected to simulated scheduling based on the ex-warehouse task parameter.

Specifically, in the process of performing simulated scheduling on the order to be delivered received by the electronic device 100, the task scheduling network 201 initializes the task parameters of delivery, extracts the characteristics of each order to be delivered, and calculates the policy gradient of each order to be delivered based on the extracted characteristics of each order to be delivered. And selecting one with the largest strategy gradient from the orders to be delivered as the next delivery order, deleting the order from the orders to be dispatched, and calculating a return value (first return value) for selecting the order, wherein the return value is used for representing the waiting time required for the travelling crane for transporting the goods corresponding to the order when the order is selected as the next delivery order, for example, in some embodiments, the return value can be defined as the negative value of the total waiting time required for the travelling crane for transporting the goods corresponding to the order when the order is selected as the next delivery order. The task scheduling network 201 may generate a simulated scheduling scheme by circularly executing the above steps, where the simulated scheduling scheme includes a delivery sequence of each order to be delivered and a driving for executing each order to be delivered, which are received by the electronic device 100.

After the task scheduling network 201 generates a simulated scheduling scheme, the scheduling evaluation network 202 evaluates the simulated scheduling scheme to generate an evaluation result of each order in the simulated scheduling scheme, so that the driving scheduling model 200 can update network parameters of the task scheduling network 201 and the scheduling evaluation network 202 based on the return value and the evaluation result of each order, and the task scheduling network 201 can provide a scheduling scheme with a larger accumulated return value (i.e., the sum of the return values of each order) (i.e., the scheduling scheme with the shortest accumulated waiting time of each driving) in the next simulated scheduling process.

It will be appreciated that, in some embodiments, the scheduling evaluation network 202 may also determine the order to be delivered after the task scheduling network 201 determines the delivery order of the order to be delivered, that is, evaluate the order, and generate an evaluation result of each order, which is not limited herein.

After obtaining the return value and the evaluation result of each order in a model scheduling scheme, the driving scheduling model 200 updates the network parameters of the task scheduling network 201 and the scheduling evaluation network 202 according to the return value and the evaluation result of each order, and performs the next simulated scheduling with the new network parameters. After the driving schedule model 200 is subjected to multiple simulated scheduling, multiple simulated scheduling schemes can be generated, and an optimal one of the multiple simulated scheduling schemes is used as a delivery scheme, for example, the one with the largest accumulated return value is selected as the delivery scheme. After acquiring the delivery plan (including, but not limited to, the delivery order of each order, the starting position and the target position of the goods of each order, and the identification of the vehicle executing each order) generated by the vehicle dispatching model 200, the electronic device 100 controls the vehicle 01 and the vehicle 02 to deliver the goods corresponding to each order one by one.

It can be understood that the scheduling and evaluating network 202 may be a pre-trained network, and may take the foregoing ex-warehouse task parameter as an input, output a predicted value of the driving waiting time corresponding to the ex-warehouse task parameter (hereinafter referred to as a predicted value), and use a difference value of the predicted value corresponding to the ex-warehouse task parameter after determining an order minus the predicted value corresponding to the ex-warehouse task parameter before determining the order as an evaluation result of the order. In some embodiments, during the pre-training process of the scheduling evaluation network 202, a preset scheduling scheme may be used as an input, and an ideal waiting time of each driving in the preset scheduling scheme is used as a training target, for example, 0, so as to fit a rule for calculating a predicted value corresponding to the parameters of the ex-warehouse task.

It can be appreciated that, in some embodiments, in the process of controlling the crane 01and the row 02 to transport the goods corresponding to each order one by the electronic device 100 to be delivered, if the status of the order to be delivered changes, for example, cancel part of the order, add a new order, etc., the electronic device 100 may also perform the simulated scheduling again based on the crane scheduling model 200, so as to determine a more reasonable delivery scheme, and further improve the efficiency of transporting the goods by the crane to be delivered.

It will be appreciated that in embodiments of the present application, the task scheduling network 201 and the scheduling evaluation network 202 of the driving scheduling model 200 may be implemented based on the same type of neural network model, may be implemented based on different neural network models, and may be implemented in combination with multiple types of neural network models. For example, it may be implemented based on at least one of the neural Network models of convolutional neural Network (Convolutional Neural Network, CNN), recurrent neural Network (Recurrent Neural Network, RNN), attention mechanism Network (Attention Mechanism Network), fully connected neural Network (Fully Connected Neural Network), normalized Network (Normalization Network), drop Network (Dropout Network), gated recurrent unit Network (gated recurrent unit, GRU), etc. In some embodiments of the present application, the driving schedule model 200 may be implemented by an actor-critter (actor-critic, AC) structure, where the task scheduling network 201 is an actor network and the schedule evaluation network 202 is a critter network.

Illustratively, FIG. 3 shows a schematic diagram of a task scheduling network 201, in accordance with some embodiments of the application. As shown in fig. 3, the task scheduling network 201 may include a one-dimensional convolution network 201A, a one-dimensional convolution network 201B, a one-dimensional convolution network 201C, GRU network 201D, a fully connected network 201E, a discard network 201F, an attention mechanism network 201G, a normalization network 201H, an attention mechanism network 201I, and a task selection network 201J.

The one-dimensional convolution network 201A, the one-dimensional convolution network 201B and the fully connected network 201E are used for extracting features of the ex-warehouse task parameters of the input task scheduling network 201, so that the policy gradient of each order can be calculated by subsequent calculation.

The one-dimensional convolution network 201C, GRU network 201D and the discard network 201F are configured to delete a part of the networks of the task scheduling network 201 according to the order determined last time by the task scheduling network 201, for example, delete the weight network corresponding to the order determined last time in the one-dimensional convolution network 201A, the one-dimensional convolution network 201B and the fully connected network 201E, so as to improve the computing performance.

The attention mechanism network 201G, the normalization network 201H, and the attention mechanism network 201I may calculate, based on the characteristics of each order output by the fully connected network 201E, a policy gradient of each order, for determining the next order to be delivered. Wherein the policy gradient of the order indicates the probability of selecting the order as the next order to be delivered.

The task selection network 201J may select, according to the policy gradient of each order outputted by the attention mechanism network 201I, the order with the largest policy gradient as the next order to be delivered. In some embodiments, the task selection network 201J may also select a next order to be delivered based on a greedy algorithm, such as selecting a next order to be delivered from the orders based on an epsilon-greedy algorithm, according to a policy gradient for each order.

It will be appreciated that the architecture of the task scheduling network 201 shown in fig. 3 is merely an example, and in other embodiments, the task scheduling network 201 may include more or fewer networks, and may also incorporate or split part of the network, and may also replace part of the network, and embodiments of the present application are not limited.

Further, fig. 4 is a flow chart illustrating a driving scheduling method according to some embodiments of the present application. The flow is performed by the electronic device 100, as shown in fig. 4, and includes the following steps.

S401: and obtaining the parameters of the ex-warehouse task. The electronic device 100 obtains the ex-warehouse task parameter, so that the electronic device 100 determines an ex-warehouse scheme based on the driving scheduling model 200 according to the ex-warehouse task parameter.

It is understood that the job parameters include, but are not limited to, a starting point position of each order, a target position of each order, an identification of each vehicle in the warehouse 00, an initial position of each vehicle, an area that each vehicle can reach in the warehouse 00, and the like.

It will be appreciated that in some embodiments, the job parameters may be entered by the user via the electronic device 100 or may be generated by logistics management software running on the electronic device 100, without limitation.

S402: a plurality of simulated scheduling schemes are determined based on the driving scheduling model 200 and the ex-warehouse task parameters, and an optimal one is determined from the simulated scheduling schemes as the ex-warehouse scheme. The electronic device 100 transmits the ex-warehouse task parameter acquired in step S402 to the driving scheduling model 200, and the driving scheduling model 200 performs the simulated scheduling for the preset times according to the ex-warehouse task parameter based on the principle of minimizing the waiting time of the driving, for example, performs the simulated scheduling for 100 times, generates 100 kinds of simulated scheduling schemes, and selects an optimal one from the 100 kinds of simulated scheduling schemes as the ex-warehouse scheme. The specific simulation scheduling process will be described in detail below, and will not be described in detail here.

It will be appreciated that in some embodiments, the delivery scheme may specifically include a delivery order of each order, an identification of a driving of the goods corresponding to each order that is delivered from the storage area 01 to the loading area 02 (i.e., the driving of each order is performed), a delivery time of each order, a start position of each order, a target position of each order, and the like.

S403: scheduling according to a delivery scheme and the driving is used for executing the ex-warehouse task. After the electronic device 100 acquires the delivery plan, it can schedule the travelling crane to execute the orders according to the delivery sequence of the orders in the delivery plan. For example, in the scenario shown in fig. 1, the electronic device 100 may send an instruction to the crane 10 and/or the crane 20 according to the order of delivery of each order in the delivery scheme, so as to control the crane 10 and/or the crane 20 to transport the goods corresponding to each order from the storage area 01 to the vehicle in the loading area 02 according to the order of delivery of each order in the delivery scheme.

S404: and judging whether the library task is updated or not. In the process of scheduling the vehicle to execute the delivery task according to the delivery scheme, the electronic device 100 may periodically execute step S404 to determine whether the delivery task is updated, for example, periodically determine whether the order is added or cancelled, and if it is determined that the delivery task is updated, go to step S401 to schedule the updated delivery task, and if the delivery task is not updated, go to step S403 to continue to schedule the vehicle to execute the delivery task.

It should be understood that the order of execution of the steps S401 to S404 is merely an example, and in other embodiments, some steps may be combined or split, and the order of execution of at least some steps may be changed, which is not limited herein.

By the driving scheduling method provided by the embodiment of the application, the waiting time of the driving in the process of delivering the goods to the warehouse can be reduced, and the efficiency of delivering the goods to the warehouse by using the driving is improved. And when the delivery task changes, the delivery sequence of each order and the delivery vehicle for delivering the goods of each order can be determined again, so that the delivery efficiency of delivering the goods by using the delivery vehicle is further improved, the steel finished product storage operation cost is reduced, and the storage overall operation efficiency is improved.

The specific process of determining a plurality of simulated scheduling schemes based on the ex-warehouse driving scheduling model 200 and the ex-warehouse task parameter and determining an optimal one from the simulated scheduling schemes as the ex-warehouse scheme in the above step S402 will be described in detail.

Fig. 5 illustrates a schematic diagram of a process by which the traffic scheduling model 200 determines a scheduling scheme, according to some embodiments of the application. As shown in fig. 5, the process includes the following steps.

S501: the task scheduling network 201 initializes scheduling parameters according to the outbound task parameters. That is, the task scheduling network 201 converts the number of the order sheets to be delivered, the position of the goods corresponding to each order sheet in the storage area 01, the loading position of the goods corresponding to each order sheet in the loading area 02, the initial position of each travelling crane in the warehouse 00, and the reachable area of each travelling crane in the warehouse 00 into a data format which can be identified by the task scheduling network 201 according to the delivery task parameters.

For example, in some embodiments, the task scheduling network 201 may initialize the outbound task parameters to a state vector s= { S1, S2}, where S1 represents order information and S2 represents driving information.

Specifically, in some embodiments, s1= { ls, le, to, ts }. Wherein ls and le are attributes of space dimension, ls represents the position of goods corresponding to the order in the storage area 01, and le represents the position of goods corresponding to the order in the goods loading area 02; to and ts are attributes of a time dimension, where to represents a time required to transfer an order from the warehouse area 01 to the loading area 02, and ts represents an earliest time that the order can be executed, for example, an earliest time that a transport vehicle corresponding to the order can reach the loading area 02.

It will be appreciated that in some embodiments, to may be calculated according to the following equation (1):

Referring to fig. 1, t _start represents a time when the crane grabs the goods, for example, t _start may be a time required for the crane 10 to lower the lifting mechanism 13 to a position of each of the goods in the storage area 01, grab the goods, and rise to a preset height; t _end represents the time the crane places the cargo on the vehicle in the cargo area for transporting the cargo, e.g., t _end may be the time the crane 10 lowers the hoist 13 on the vehicle transporting the cargo; x ₁ represents the X-direction coordinate of the goods in the storage area 01, Y ₁ represents the Y-direction coordinate of the goods in the storage area 01, X ₂ represents the Y-direction coordinate of the goods in the loading area 02, and Y ₂ represents the Y-direction coordinate of the goods in the loading area 02; v represents the moving speed of the travelling crane in the X direction and the Y direction; max (X ₂-x₁,y₂-y₁) represents the larger of the X-coordinate difference and Y-coordinate difference of the goods in the warehouse area 01 and the loading area 02.

It will be appreciated that the velocity of the v-ride in the X and Y directions in equation 1 is for ease of calculation, and in other embodiments, the velocity of the ride in the X and Y directions may be different, and is not limited herein.

In some embodiments, s2= { ls1, le1, ts, te, tn }. Wherein ls1 and le1 are attributes of space dimensions, ls1 represents a position of a cargo corresponding to an order in a warehouse, where the cargo is currently executed by a crane, in the warehouse area 01, and le1 represents a position of a cargo corresponding to the order in the warehouse area 02, where the cargo is currently executed by the crane. ts, te, tn are the attribute of the time dimension, ts represents the start time of the current execution task of the driving vehicle, te represents the end time of the current execution task of the driving vehicle, tn represents the current time, and ts, te, tn are used for judging the task which can be executed by the driving vehicle in real time.

It will be appreciated that initializing the outbound task parameters to the scheduling parameters represented by the state vector S is merely an example, and that in other embodiments the outbound task parameters may be initialized to other forms of scheduling parameters.

In some embodiments, the scheduling parameters further include the locations in the warehouse that each row cart can reach, for example, referring to fig. 1, the region that row cart 10 can reach is a region with Y coordinates of 0 to 2, and the region that row cart 20 can reach is all the regions in the warehouse.

S502: the task scheduling network 201 acquires network parameters of the task scheduling network 201, determines a next order from the warehouse according to the scheduling parameters under the network parameters, generates a return value of the order and updates the scheduling parameters.

The task scheduling network 201 obtains network parameters of the task scheduling network 201 for determining a next job order. It may be appreciated that, in some embodiments, the network parameters of the task scheduling network 201 acquired by the task scheduling network 201 may be the network parameters updated by the driving scheduling model 200 in the last simulated scheduling process; in other embodiments, the network parameters of the task scheduling network 201 acquired by the task scheduling network 201 may also be network parameters generated by training the driving scheduling model 200 in advance; the network parameters that the task scheduling network 201 randomly generates may also be, but are not limited to.

After the task scheduling network 201 obtains the network parameters of the task scheduling network 201, the next scheduled driving can be determined according to the end time of the order currently executed by the driving 10 and the driving 20, for example, te, and the driving with the order currently executed is first ended as the next scheduled driving (i.e., the driving with te being the previous driving). After determining the next scheduled driving, the task scheduling network 201 can determine the order that the current driving can execute according to the area that the driving can reach in the warehouse 00 and the position of the goods corresponding to each order in the warehouse area 01, calculate the policy gradient of each order that the current driving can execute, and take the order with the largest policy gradient as the order of the next delivery.

After determining the next order to be delivered, the task scheduling network 201 calculates a return value for selecting the order, for example, a negative value of a time required to wait for a corresponding driving when executing the order after selecting the task may be used as the return value of the order, so as to adjust parameters of the task scheduling network 201.

It will be appreciated that the task scheduling network 201 updates the scheduling parameters after determining the order of the next delivery, for example, deletes the determined order information from the scheduling parameters, so as to avoid selecting the order that has already been determined in the delivery order again at the time of the next selection. For example, the current scheduling parameter is a state vector S1, which contains information of k orders, and after determining the next order m, updates the scheduling parameter to a state vector S2 including k-1 orders, i.e. deletes the information corresponding to the order m in S1, and updates the driving information corresponding to S2. After the task scheduling network 201 updates the scheduling parameters, the scheduling parameters before and after the update, such as the state vector S1 and the state vector S2, are transmitted to the scheduling evaluation network 202.

It may be understood that the foregoing taking the order with the largest policy gradient as the order for the next delivery is merely an example, and in other embodiments, after calculating the policy gradient of each order that can be executed by the current driving, the next delivery order may be determined based on an epsilon-greedy algorithm, which is not limited by the embodiments of the present application.

S503: the scheduling and evaluating network 202 acquires network parameters of the scheduling and evaluating network 202, and generates an evaluating result of the order by taking the scheduling parameters before and after updating as input under the network parameters.

For example, as described above, when the scheduling parameter before updating is the state vector S1 and the scheduling parameter after updating is the state vector S2, the scheduling evaluation network 202 generates predicted values corresponding to the two state vectors with S1 and S2 as outputs, respectively, and uses the difference obtained by subtracting the predicted value of S1 from the predicted value of S2 as the evaluation result of the order.

It will be appreciated that, as described above, the schedule evaluation network 202 uses the ideal waiting time value of the input schedule parameter as the training target in the training process, so that the evaluation result characterizes the waiting time of the driving of the order when the order is selected.

S504: the task scheduling network 201 determines whether or not the simulated scheduling of all the orders is completed. That is, the task scheduling network 201 determines whether the order of delivery of all the orders and the driving of the execution orders have been determined, and if the order of delivery of all the orders and the driving of the execution orders have been determined, goes to step S505 to generate a simulated scheduling scheme; otherwise, go to step S502 to determine the next order for delivery.

S505: the task scheduling network 201 generates a simulated scheduling scheme. That is, the task scheduling network 201 may generate a simulated scheduling scheme after completing a complete simulated scheduling (i.e. determining the order of delivery of all orders and the driving of the goods corresponding to each order).

S506: the traffic scheduling model 200 updates the network parameters of the task scheduling network 201 and the network parameters of the scheduling evaluation network 202 based on the evaluation results and the return values of the respective orders.

As described above, the evaluation result and the return value of each order may indicate the waiting time of each train when the order is delivered by the simulated scheduling scheme. Thus, in some embodiments, the network parameters of the task scheduling network 201 and the network parameters of the scheduling evaluation network 202 may be updated based on the evaluation results and the return values of the orders.

Specifically, in some embodiments, the loss function 1 of the task scheduling network 201 may be calculated by equation (2):

where n is the number of orders.

The loss function 2 of the dispatch evaluation network 202 may be calculated by equation (3):

where n is the number of orders.

The traffic scheduling model 200 may update the network parameters of the task scheduling network 201 and the network parameters of the scheduling evaluation network 202 by a gradient descent method, a gradient ascent method, or the like based on the loss function 1 and the loss function 2. Therefore, the network parameters of the task scheduling network 201 and the network parameters of the scheduling evaluation network 202 can be continuously optimized in the simulated scheduling process, so that a scheduling scheme with a higher return value can be obtained in the next scheduling.

S507: the traffic scheduling model 200 determines whether the preset number of simulated scheduling times has been reached. The driving schedule model 200 judges whether the preset simulated schedule times are reached, if so, the driving schedule model indicates that the simulated schedule is not required to be continued, and the step S508 is performed; otherwise, it indicates that the simulation scheduling still needs to be continued, and the process goes to step S501, where the next simulation scheduling is performed using the updated network parameters.

S508: the driving schedule model 200 selects an optimal one from the generated simulated schedule schemes as a delivery scheme. That is, the driving schedule model 200 selects an optimal one from the generated schemes as the delivery scheme after completing the simulated schedule of the preset simulated schedule times. For example, the driving schedule model 200 may select one of the simulated schedule schemes with the largest cumulative return value (i.e., the one with the shortest waiting time of each driving) as the delivery scheme.

It should be understood that the foregoing execution sequence of steps S501 to S508 is only an example, and in other embodiments, other sequences may be adopted, for example, step S503 may be executed after step S505, which is not limited herein.

By the method provided by the embodiment of the application, the delivery scheme with the shortest waiting time of the travelling crane can be determined, so that the electronic equipment 100 can control the travelling crane to operate the delivery of goods according to the delivery scheme, the waiting time of each travelling crane when the travelling crane is used for delivering the goods to deliver the goods to the delivery is reduced, the delivery efficiency is improved, the steel finished product warehousing operation cost is reduced, and the overall warehousing operation efficiency is improved.

Fig. 6 illustrates a schematic diagram of an electronic device 100, according to some embodiments of the application. As shown in fig. 6, electronic device 100 includes one or more processors 101, a system Memory 102, a Non-Volatile Memory (NVM) 103, a communication interface 104, an input/output (I/O) device 105, and system control logic 106 for coupling processor 101, system Memory 102, non-Volatile Memory 103, communication interface 104, and input/output (I/O) device 105. Wherein:

Processor 101 may include one or more single-core or multi-core processors. In some embodiments, processor 101 may include any combination of general-purpose and special-purpose processors (e.g., graphics processor, application processor, baseband processor, etc.). In some embodiments, the processor 101 may be configured to execute instructions corresponding to the driving schedule model 200 to determine a delivery plan according to the delivery task parameters.

The system Memory 102 is a volatile Memory such as Random-Access Memory (RAM), double data rate synchronous dynamic Random Access Memory (Double Data Rate Synchronous Dynamic Random Access Memory, DDR SDRAM), or the like. The system memory is used to temporarily store data and/or instructions, for example, in some embodiments, the system memory 102 may be used to store instructions, ex-warehouse solutions, etc. corresponding to the traffic scheduling model 200.

Nonvolatile memory 103 may include one or more tangible, non-transitory computer-readable media for storing data and/or instructions. In some embodiments, the non-volatile memory 103 may include any suitable non-volatile memory, such as flash memory, and/or any suitable non-volatile storage device, such as a hard disk drive (HARD DISK DRIVE, HDD), compact Disc (CD), digital versatile Disc (DIGITAL VERSATILE DISC, DVD), solid state disk (Solid-state-STATE DRIVE, SSD), and the like. In some embodiments, the nonvolatile memory 103 may also be a removable storage medium, such as a Secure Digital (SD) memory card or the like. In other embodiments, the nonvolatile memory 103 may be used to store instructions, a delivery scheme, etc. corresponding to the traffic scheduling model 200.

In particular, the system memory 102 and the nonvolatile memory 103 may each include: a temporary copy and a permanent copy of instruction 107. The instructions 107 may include: the execution by at least one of the processors 101 causes the electronic device 100 to implement the traffic scheduling method provided by the embodiments of the present application.

The communication interface 104 may include a transceiver to provide a wired or wireless communication interface for the electronic device 100 to communicate with any other suitable device via one or more networks. In some embodiments, the communication interface 104 may be integrated with other components of the electronic device 100, e.g., the communication interface 104 may be integrated in the processor 101. In some embodiments, the electronic device 100 may communicate with other devices through the communication interface 104, such as obtaining the location of the row cart 10 and the row cart 20 in the warehouse 00 through the communication interface 104, controlling the row cart 10 and the row cart 20 to transport goods from the storage area 01 to the loading area 02 through the communication interface 104, and so on.

Input/output (I/O) device 105 may be an input device such as a keyboard, a mouse, etc., an output device such as a display, etc., and a user may interact with electronic device 100 through input/output (I/O) device 105, such as inputting order information, displaying order status, etc.

The system control logic 106 may include any suitable interface controller to provide any suitable interface with other modules of the electronic device 100. For example, in some embodiments, the system control logic 106 may include one or more memory controllers to provide an interface to the system memory 102 and the non-volatile memory 103.

In some embodiments, at least one of the processors 101 may be packaged together with logic for one or more controllers of the system control logic 106 to form a system package (SYSTEM IN PACKAGE, SIP). In other embodiments, at least one of the processors 101 may also be integrated on the same Chip with logic for one or more controllers of the System control logic 106 to form a System-on-Chip (SoC).

It is understood that the electronic device 100 may be any electronic device capable of implementing the driving scheduling method provided in the embodiments of the present application, including but not limited to a computer, a server, a tablet computer, a handheld computer, etc., which is not limited to the embodiments of the present application.

It is to be understood that the configuration of the electronic device 100 shown in fig. 6 is merely an example, and in other embodiments, the electronic device 100 may include more or fewer components than shown, or may combine certain components, or may split certain components, or may have a different arrangement of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

It will be appreciated that, in some embodiments, to increase the speed of the electronic device 100 for driving scheduling, the processor 101 of the electronic device 100 may be an eighth generation of kuri ^TM i7 processor or a processor with comparable/higher computing power, and the space size of the system memory 102 of the electronic device 100 may be greater than 16GB.

Embodiments of the disclosed mechanisms may be implemented in hardware, software, firmware, or a combination of these implementations. Embodiments of the application may be implemented as a computer program or program code that is executed on a programmable system comprising at least one processor, a storage system (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device.

Program code may be applied to input instructions to perform the functions described herein and generate output information. The output information may be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a Digital Signal Processor (DSP), a microcontroller, an Application Specific Integrated Circuit (ASIC), or a microprocessor.

The program code may be implemented in a high level procedural or object oriented programming language to communicate with a processing system. Program code may also be implemented in assembly or machine language, if desired. Indeed, the mechanisms described in the present application are not limited in scope by any particular programming language. In either case, the language may be a compiled or interpreted language.

In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried by or stored on one or more transitory or non-transitory machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed over a network or through other computer readable media. Thus, a machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including but not limited to floppy diskettes, optical disks, read-only memories (CD-ROMs), magneto-optical disks, read-only memories (ROMs), random Access Memories (RAMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), magnetic or optical cards, flash memory, or tangible machine-readable memory for transmitting information (e.g., carrier waves, infrared signal digital signals, etc.) in an electrical, optical, acoustical or other form of propagated signal using the internet. Thus, a machine-readable medium includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a form readable by a machine (e.g., a computer).

In the drawings, some structural or methodological features may be shown in a particular arrangement and/or order. However, it should be understood that such a particular arrangement and/or ordering may not be required. Rather, in some embodiments, these features may be arranged in a different manner and/or order than shown in the illustrative figures. Additionally, the inclusion of structural or methodological features in a particular figure is not meant to imply that such features are required in all embodiments, and in some embodiments, may not be included or may be combined with other features.

It should be noted that, in the embodiments of the present application, each unit/module mentioned in each device is a logic unit/module, and in physical terms, one logic unit/module may be one physical unit/module, or may be a part of one physical unit/module, or may be implemented by a combination of multiple physical units/modules, where the physical implementation manner of the logic unit/module itself is not the most important, and the combination of functions implemented by the logic unit/module is only a key for solving the technical problem posed by the present application. Furthermore, in order to highlight the innovative part of the present application, the above-described device embodiments of the present application do not introduce units/modules that are less closely related to solving the technical problems posed by the present application, which does not indicate that the above-described device embodiments do not have other units/modules.

It should be noted that in the examples and descriptions of this patent, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

While the application has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the application.

Claims

1. The driving scheduling method is applied to electronic equipment and is characterized by comprising the following steps of:

Simulating and scheduling a plurality of orders to be delivered based on delivery task parameters of the orders to be delivered by using a first scheduling model, and generating a first scheduling scheme and a first return value of each order, wherein the first scheduling model is a model of an actor-critique structure comprising a task scheduling network and a scheduling evaluation network, the task scheduling network is an actor network, and the scheduling evaluation network is a critique network;

Evaluating the first scheduling scheme, generating an evaluation result of each order, and adjusting network parameters of the task scheduling network and the scheduling evaluation network based on the first return value and the evaluation result of each order to obtain a second scheduling model;

Performing simulated scheduling on the plurality of orders to be delivered based on the delivery task parameters by using the second scheduling model, and generating a second scheduling scheme and second return values of the orders, wherein the sum of the second return values is larger than the sum of the first return values;

and dispatching the travelling crane to carry out delivery on the order to be delivered according to the second dispatching scheme.

2. The method of claim 1, wherein the first scheduling scheme is generated by the task scheduling network.

3. The method of claim 2, the task scheduling network generating the first scheduling scheme by cyclically:

And determining the strategy gradient of each order without determining the order of delivery, and taking the one with the largest strategy gradient as the next order of delivery.

4. The method of claim 1, wherein each of the first return values is generated by the dispatch evaluation network, wherein the dispatch evaluation network includes rules that predict a wait time for each trip when the plurality of orders to be outbound.

5. The method according to any one of claims 1 to 4, wherein the second scheduling scheme includes a delivery order of each order, a driving identification for executing each order, a start position and a target position of goods corresponding to each order.

6. The method of claim 5, wherein the scheduling the driving to take out the order to be taken out according to the second scheduling scheme comprises:

And sending a delivery instruction to a vehicle executing each order according to the delivery sequence of each order and the vehicle identifier of each order.

7. The method according to claim 6, wherein:

And under the condition that the order to be delivered is changed, performing simulated scheduling on the order to be delivered by using a first scheduling model, and generating a first scheduling scheme and a first return value of each order.

8. A readable medium having stored thereon instructions that, when executed on an electronic device, cause the electronic device to perform the traffic scheduling method of any one of claims 1 to 7.

9. An electronic device, comprising:

a memory for storing instructions for execution by one or more processors of the electronic device;

And a processor, which is one of the processors of the electronic device, for executing the instructions stored in the memory to implement the traffic scheduling method of any one of claims 1 to 7.