CN116192877A - Computing device, synchronization method, electronic device, and storage medium - Google Patents

Abstract

The present disclosure provides a computing device, a synchronization method for the computing device, an electronic device, and a computer-readable storage medium. The computing device includes: a plurality of computing clusters, the plurality of computing clusters are divided into a plurality of broadcast groups; and one or more intra-cluster interfaces respectively located in each computing cluster of the plurality of computing clusters, wherein each When a computing cluster in the broadcast group is used as a consumer computing cluster, it is configured to send a broadcast synchronization barrier message to the intra-cluster interface when the data space for all producer computing clusters in the broadcast group is ready, so The broadcast synchronization barrier message includes a broadcast indicator and an identifier of the broadcast group, and the intra-cluster interface is configured to expand the broadcast synchronization barrier message to compute clusters for each producer in the broadcast group Synchronize barrier messages, and send each synchronization barrier message to the corresponding producer computing cluster.

Description

Computing device, synchronization method, electronic device and storage medium

technical field

The present disclosure generally relates to the field of processors, and more specifically, to a computing device, a synchronization method for the computing device, an electronic device, and a computer-readable storage medium.

Background technique

At present, the structures of various computing devices are becoming more and more complex, and there are more and more hardware layers inside computing devices, so it is necessary to solve the synchronization problem of data interaction between various layers. For example, for a computing device that contains multiple computing clusters, if the computing cluster that generates the data needs to write the generated data to the same computing cluster so that the computing cluster can use the data to perform subsequent processing, the computing cluster that uses the data needs to Inform the computing cluster that produced the data whether it has space ready to receive the produced data.

In the existing technology, using the synchronization barrier mechanism based on the thread warp (wrap), the computing cluster using the data sends a synchronization barrier message one by one to each computing cluster that generates the data to inform whether the space is ready to receive the computing The data generated by the cluster. However, this synchronization barrier mechanism will consume a large number of instruction cycles for computing clusters using data, especially in the case of broadcasting.

Contents of the invention

In view of the above problems, the present disclosure provides a space synchronization scheme in which a computing cluster that uses data sends a broadcast synchronization barrier message, and the intra-cluster interface of the computing cluster expands it to perform space synchronization, which greatly reduces Instruction cycles required for a synchronization barrier.

According to one aspect of the present disclosure, a computing device is provided. The computing device includes: a plurality of computing clusters, the plurality of computing clusters are divided into a plurality of broadcast groups; and one or more intra-cluster interfaces respectively located in each computing cluster of the plurality of computing clusters, wherein each When a computing cluster in a broadcast group is used as a consumer computing cluster, it is configured to send a broadcast synchronization barrier message to the intra-cluster interface when the data space for all producer computing clusters in the broadcast group is ready, The broadcast synchronization barrier message includes a broadcast indicator and an identifier of the broadcast group, and the intracluster interface is configured to expand the broadcast synchronization barrier message to compute clusters for each producer in the broadcast group The synchronization barrier message, and each synchronization barrier message is sent to the corresponding producer computing cluster.

In some implementations, the intra-cluster interface is configured to determine a corresponding producer computing cluster based on an identifier of the broadcast group in the broadcast synchronization barrier message.

In some implementations, the intra-cluster interface includes a plurality of mask registers, each mask register corresponds to one of the plurality of broadcast groups, and the mask register is used to indicate the plurality of computing clusters in a computing cluster belonging to the broadcast group, and wherein the intra-cluster interface is configured to determine a barrier register corresponding to the broadcast group based on the identifier of the broadcast group in the broadcast synchronization barrier message, and based on The barrier register corresponding to the broadcast group determines the corresponding producer computing cluster.

In some implementations, the consumer computing cluster is further configured to: determine whether data space for all producer computing clusters in the broadcast group is ready; The data space of the cluster is ready, generating the broadcast synchronization barrier message; and in response to the fact that the data space of the computing cluster for some producers in the broadcast group is not ready, continue to wait for the computing cluster for the part of the producers Space is ready.

In some implementations, each producer compute cluster in the broadcast group is configured to: determine whether a synchronization barrier message has been received from all other compute clusters in the broadcast group; sending the generated data to the other computing clusters for synchronization barrier messages from all the other computing clusters in the broadcast group; and continuing to wait in response to not receiving the synchronization barrier messages from all the other computing clusters in the broadcast group.

In some implementations, the intra-cluster interface is configured to sequentially send each synchronization barrier message to the corresponding producer computing cluster through the inter-cluster interface.

According to another aspect of the present disclosure, there is provided a synchronization method for a computing device comprising a plurality of computing clusters and one or more clusters underlying each computing cluster in the plurality of computing clusters An internal interface, wherein the plurality of computing clusters are divided into a plurality of broadcast groups. The synchronization method includes: a computing cluster serving as a consumer computing cluster in each broadcast group sends a broadcast synchronization to the intra-cluster interface when the data space for all producer computing clusters in the broadcast group is ready. a barrier message, the broadcast synchronization barrier message including a broadcast indicator and an identifier of the broadcast group, and the broadcast synchronization barrier message is expanded by the intra-cluster interface to calculate for each producer in the broadcast group The synchronization barrier message of the cluster, and each synchronization barrier message is sent to the corresponding producer computing cluster.

In some implementations, expanding, by the intra-cluster interface, the broadcast synchronization barrier message into a synchronization barrier message for each producer computing cluster in the broadcast group includes: synchronizing by the intra-cluster interface based on the broadcast The identifier of the broadcast group in the barrier message identifies the corresponding producer compute cluster.

In some implementations, the intra-cluster interface includes a plurality of mask registers, each mask register corresponds to one of the plurality of broadcast groups, and the mask register is used to indicate the plurality of computing clusters Computing clusters belonging to the broadcast group, and wherein determining the corresponding producer computing cluster by the intra-cluster interface based on the identifier of the broadcast group in the broadcast synchronization barrier message comprises: by the intra-cluster interface based on The identifier of the broadcast group in the broadcast synchronization barrier message determines a barrier register corresponding to the broadcast group, and determines a corresponding producer computing cluster based on the barrier register corresponding to the broadcast group.

In some implementations, the synchronizing method further includes: determining, by the consumer computing cluster, whether data space for all producer computing clusters in the broadcast group is ready; The data spaces of the computing clusters of the other computing clusters are all ready to generate the broadcast synchronization barrier message; The space for the cluster is ready.

In some implementations, the synchronization method further includes: determining, by each producer computing cluster in the broadcast group, whether a synchronization barrier message has been received from all other computing clusters in the broadcast group; to all other computing clusters in the broadcast group, sending the generated data to the other computing clusters; and in response to not having received a synchronization barrier message from all the other computing clusters in the broadcast group, continuing wait.

According to still another aspect of the present disclosure, there is provided an electronic device, including: a memory storing computer-executable instructions in a non-transitory manner; a processor configured to run the computer-executable instructions; wherein the computer-executable The instructions, when executed by the processor, implement the synchronization method as described above.

According to still another aspect of the present disclosure, there is provided a computer-readable storage medium, on which computer program code is stored, and the computer program code executes the above-mentioned synchronization method when executed.

Description of drawings

The present disclosure will be better understood, and other objects, details, features and advantages of the present disclosure will become more apparent by referring to the description of specific embodiments of the present disclosure given in the following drawings.

FIG. 1A shows an exemplary structural diagram of a computing device.

FIG. 1B shows a schematic diagram of a computing device connecting multiple computing clusters through a multi-level inter-cluster interface.

Fig. 2 shows a schematic diagram of a detailed structure of a computing cluster.

Fig. 3 shows an exemplary schematic diagram of a barrier register in an intra-cluster interface according to an embodiment of the present invention.

FIG. 4 shows a flowchart of a synchronization method for a computing device according to an embodiment of the present invention.

Detailed ways

Preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

As used herein, the term "comprise" and its variants mean open inclusion, ie "including but not limited to". The term "or" means "and/or" unless otherwise stated. The term "based on" means "based at least in part on". The terms "one embodiment" and "some embodiments" mean "at least one example embodiment." The term "another embodiment" means "at least one further embodiment". The terms "first", "second", etc. may refer to different or the same object.

FIG. 1A shows a schematic structural diagram of an exemplary computing device 100 . As shown in FIG. 1A , computing device 100 may include a plurality of computing clusters 110 (eight computing clusters 110-1 , 110-2, . . . 110-8 are schematically shown in FIG. 1A ).

The computing cluster 110 in the computing device 100 can be further divided into a plurality of broadcast groups 120, as shown in FIG. 1A, these eight computing clusters 110 are divided into two broadcast groups 120-1 and 120-2, wherein , 110-2, 110-5, and 110-6 belong to the broadcast group 120-1, and computing clusters 110-3, 110-4, 110-7, and 110-8 belong to the broadcast group 120-2. In a broadcast group, the computing cluster 110 that generates data needs to send the generated data to all other computing clusters in the same broadcast group for consumption by other computing clusters. Here, dividing computing clusters into broadcast groups may be based on physical distribution of computing clusters or application requirements, for example.

Each computing cluster 110 may include one or more intra-cluster interfaces 112 (only one intra-cluster interface 112 is schematically shown for each computing cluster 110 in FIG. 1 ). Each computing cluster 110 , more specifically, the intra-cluster interface 112 of each computing cluster 110 is also connected through an inter-cluster interface 130 . Each computing cluster 110 can exchange data and instructions with other computing clusters 110 through its own intra-cluster interface 112 and inter-cluster interface 130 . The intra-cluster interface 112 may be a sub-module of the computing cluster 110 or a separate hardware module attached to the computing cluster 110 .

Intercluster interface 130 may include one or more layers of hardware interfaces. FIG. 1B shows a schematic diagram of a computing device 100 connecting multiple computing clusters 110 through a multi-level inter-cluster interface 130 . As shown in FIG. 1B , the entire computing device 100 may be composed of multiple device components (device) 140 , each device component 140 may include multiple die 150 , and each die 150 may include multiple computing clusters 110 . Fig. 1B schematically shows a three-level inter-cluster interface 130, that is, a first-level inter-cluster interface 130-1 directly connected to the intra-cluster interface 112 of each computing cluster 110, sometimes also referred to as a network on chip (NOC); The second-level inter-cluster interface 130 - 2 of the chip 150 is sometimes called a D2D interface, and the third-level inter-cluster interface 130 - 3 connecting each device component 140 is also sometimes called a P2P interface.

Note that computing device 100 is shown for example only in FIG. . However, those skilled in the art will understand that the numbers shown in FIG. die 150, and/or each die 150 may include more or fewer compute clusters 110.

In addition, the division of the broadcast group 120 does not necessarily have a definite corresponding relationship with the computing cluster 110 , the chip 150 and the device component 140 . For example, the number of computing clusters 110 in one broadcast group may be greater than the number of computing clusters 110 in one chip 150 , for example, it may reach the level of nodes containing multiple device components 140 (eg, 8 devices). In addition, one computing cluster 110 may be located in multiple broadcast groups 120 , that is, different broadcast groups 120 may contain the same computing cluster 110 .

Herein, the computing cluster 110 that generates data is also called a producer computing cluster, and the computing cluster 110 that consumes data is also called a consumer computing cluster. Furthermore, a computing cluster 110 is both a producer computing cluster and a consumer computing cluster if the computing cluster 110 itself both generates data and consumes the data it generates.

FIG. 2 shows a schematic diagram of the detailed structure of the computing cluster 110 . As shown in FIG. 2 , in addition to intra-cluster interface 112 , compute cluster 110 may include one or more compute units (CU) 114 each of which may include one or more execution units (EU) 116 . One or more warps can run on each execution unit 116 , but a warp can only run on one execution unit 116 . A thread warp is a basic scheduling unit often used in GPUs for performing parallel processing. Each thread warp may contain a specific number of threads, such as 32 or 16, to run various instructions.

In the computing device 100 shown in FIG. 1, in a broadcast group 120 (such as broadcast group 120-1), it is assumed that computing cluster 110-6 is used as a consumer computing cluster, and three computing clusters 110-1, 110-2 and 110-5, as the producer computing cluster, must write the generated data to the consumer computing cluster 110-6. Then, before writing, the producer computing cluster needs to know whether the consumer computing cluster 110-6 has prepared the required data space for them. In the traditional warp-based synchronization barrier mechanism, the consumer computing cluster 110-6 can send a synchronization barrier message to each producer computing cluster 110-1, 110-2, and 110-5 to inform the consumer computing cluster 110 - 6 The required data space has been prepared for the corresponding producer computing clusters 110-1, 110-2 and 110-5, that is, the data space is ready.

However, in this way, the consumer computing cluster 110-6 needs to send a synchronization barrier message, such as bar_to_spc_1, bar_to_spc_2, bar_to_spc_5, etc., to all other computing clusters in the broadcast group 120-1, thus consuming the computing cluster 110 -6 for 3 instruction cycles. For a broadcast group containing N computing clusters, it needs to consume N-1 instruction cycles for space synchronization. If the number N of computing clusters included in the broadcast group is large, the instruction cycle consumption will become very large. In the computing device 100 , the computing cluster 110 undertakes complex computing tasks, and too many instruction cycles consumed in space synchronization will inevitably reduce the instruction cycles available for other computing tasks, resulting in a decrease in the overall efficiency of the computing device.

In view of the above problems, the present disclosure provides a method in which a single broadcast synchronization barrier message is sent by a consumer computing cluster, and the broadcast synchronization barrier message is expanded into individual synchronization barrier messages through an intra-cluster interface, thereby saving inter-cluster Instruction cycles required for spatial synchronization.

Specifically, consumer computing cluster 110-6 may be configured to determine whether data spaces for all producer computing clusters 110-1, 110-2, and 110-5 in broadcast group 120-1 are ready. If it is determined that the data space for some of the producer computing clusters (such as computing cluster 110-1) in the broadcast group 120-1 is not ready yet, the consumer computing cluster 110-6 continues to wait for the space preparation of the part of the producer computing clusters ready. If it is determined that the data spaces of all producer computing clusters 110-1, 110-2, and 110-5 in the broadcast group 120-1 are ready, the consumer computing cluster 110-6 may generate a broadcast synchronization barrier message, And send the broadcast synchronization barrier message to the intra-cluster interface 112 of the consumer computing cluster 110-6.

That is to say, the consumer computing cluster 110-6 does not send a synchronization barrier to the producer computing cluster when the data space of each producer computing cluster is ready, but waits for the space of all producer computing clusters to be ready Only when the broadcast synchronization barrier message is generated and sent.

Since the intra-cluster interface 112 can be used to communicate with all computing clusters 110 in the computing device 100, in order to enable the intra-cluster interface 112 to accurately process the broadcast synchronization barrier message, the broadcast synchronization barrier message should at least contain a broadcast indicator and a broadcast An identifier for group 120-1. For example, the broadcast synchronization barrier message can be expressed as bar_broadcast_groupID, wherein "bar" indicates that this is a synchronization barrier, "broadcast" is used as a broadcast indicator to indicate to the intra-cluster interface 112 that this is a broadcast message, and "groupID" is used to indicate a broadcast An identifier for a group (eg, broadcast group 120-1).

The intra-cluster interface 112 is configured to expand the broadcast synchronization barrier message into a synchronization barrier message for each producer computing cluster in the broadcast group 120-1, and send each synchronization barrier message to the corresponding producer computing cluster .

Specifically, the intra-cluster interface 112 may determine corresponding producer computing clusters, such as producer computing clusters 110-1, 110-2, and 110-5, based on the broadcast group identifier (groupID) in the broadcast synchronization barrier message.

In some embodiments, the intra-cluster interface 112 may utilize mask registers to determine the producer computing clusters in the broadcast group. FIG. 3 shows an exemplary schematic diagram of barrier registers in the intra-cluster interface 112 according to an embodiment of the present invention. As shown in FIG. 3 , the intra-cluster interface 112 may include a plurality of mask registers 1122 (two barrier registers 1122-1 and 1122-2 are exemplarily shown in FIG. 3 ), each mask register 1122 corresponds to One of the broadcast groups 120 is used to indicate the computing clusters belonging to the broadcast group among the computing clusters 110 of the computing device 100 . Therefore, each mask register 140 should contain a number of bits at least equal to the number of computing clusters 110 contained in computing device 100 . More specifically, in the mask register 1122 for a broadcast group 120, the computing clusters 110 belonging to the broadcasting group 120 and the computing clusters 110 not belonging to the broadcasting group 120 should be distinguished, for example, represented by 1 and 0 respectively.

As shown in FIG. 3, assume that as shown in FIG. -7 and 110-8 belong to broadcast group 120-2, and mask register 1122-1 corresponds to broadcast group 120-1 (identifier is group1), and mask register 1122-2 corresponds to broadcast group 120-2 (identifier group2), the bitmap of mask register 1122-1 is 11001100, and the bitmap of mask register 1122-2 is 00110011. Note that the above grouping is only exemplary, in fact, the computing clusters included in each broadcast group may overlap. For example, a computing cluster 110 may belong to both the broadcast group 120 - 1 and the broadcast group 120 - 2 , that is, different broadcast groups 120 may include the same computing cluster 110 . Of course, a certain computing cluster 110 in the computing device 100 may not belong to any broadcast group.

The intra-cluster interface 112 is configured to determine the barrier register corresponding to the broadcast group based on the identifier of the broadcast group in the broadcast synchronization barrier message, and determine the corresponding producer computing cluster based on the barrier register corresponding to the broadcast group.

For example, if the groupID in the broadcast synchronization barrier message bar_broadcast_groupID received by the intra-cluster interface 112 is group1, it means that the broadcast synchronization barrier message is aimed at the broadcast group 120-1 whose identifier is group1, then the intra-cluster interface 112 can determine the same The barrier register corresponding to the broadcast group 120-1 is the barrier register 1122-1. The bitmap of the mask register 1122-1 is 11001100, which means that the broadcast group 120-1 includes four computing clusters 110-1, 110-2, 110-5, and 110-6, so as to synchronize the consumption of barrier messages with the broadcast The producer computing clusters corresponding to the producer computing cluster 110-6 are computing clusters 110-1, 110-2, and 110-5.

Then, the intra-cluster interface 112 can expand the broadcast synchronization barrier message bar_broadcast_groupID into synchronization barrier messages for the three producer computing clusters 110-1, 110-2, and 110-5, such as bar_to_spc_1, bar_to_spc_2, and bar_to_spc_5, and send them to The corresponding producers compute clusters 110-1, 110-2, and 110-5.

Here, the intra-cluster interface 112 may be configured to sequentially send the expanded synchronization barrier message to corresponding producer computing clusters through the inter-cluster interface 130 . For example, as shown in FIG. 1B , when the scope of the broadcast group 120 is less than a chip 150, the inter-cluster interface 130 here may only include the first-level inter-cluster interface 130-1 between the intra-cluster interfaces 112 of the computing cluster 110. . When the scope of the broadcast group 120 is larger than a chip 150 and smaller than a device component 140, the inter-cluster interface 130 here may include the first-level inter-cluster interface 130-1 between the intra-cluster interfaces 112 of the computing cluster 110 and the inter-chip The second-level inter-cluster interface 130-2. When the scope of the broadcast group 120 is larger than one device component 140 and smaller than the entire computing device 100 (also referred to as a node sometimes), the inter-cluster interface 130 here may include the first-level inter-cluster interface 112 between the intra-cluster interfaces 112 of the computing clusters 110. An interface 130 - 1 , a second level intercluster interface 130 - 2 between wafers and a third level intercluster interface 130 - 3 between device components 140 .

In this way, in the case of broadcasting, the instruction cycle used by the computing cluster for space synchronization can be reduced, and at the same time, the broadcast synchronization barrier message is expanded into a synchronization barrier for each producer computing cluster by the intra-cluster interface as a hardware module Messages also place less work on the hardware.

The above describes its synchronization barrier broadcast operation for data space readiness from the perspective of consumer computing clusters. In the case of many-to-many broadcast, each compute cluster in a broadcast group sends a sync barrier message to all other compute clusters in that broadcast group, and only if the warp count for a sync barrier is equal to When the number of computing clusters is calculated, each producer computing cluster sends the generated data to all consumer computing clusters. Therefore, from the perspective of the producer computing cluster, it needs to determine whether it has received the data space ready message of all the consumer computing clusters for the same synchronization barrier before sending the generated data to all the consumer computing clusters.

Specifically, still taking FIG. 1A as an example, it is assumed that the computing cluster 110-1 in the broadcast group 120-1 is used as the producer computing cluster, and it is configured to determine whether it has received data from all other computing clusters in the broadcast group 120-1. Synchronization barrier messages for 110-2, 110-5 and 110-6. Here, the synchronization barrier messages of computing clusters 110-2 and 110-5 may also be sent in the manner described above in conjunction with computing cluster 110-6.

Compute cluster 110-1 may count sync barrier messages for the same sync barrier from all other compute clusters in the same broadcast group. For example, in addition to the broadcast indicator and the identifier of the broadcast group, the synchronization barrier message may also include an identifier of the targeted synchronization barrier (such as barID) and an arrival indication.

If it is determined that the synchronization barrier messages from all other computing clusters in the broadcast group have been received, computing cluster 110-1 may send the generated data to these other computing clusters.

If it is determined that synchronization barrier messages from all other computing clusters in the broadcast group have not been received, computing cluster 110-1 may continue to wait for other synchronization barrier messages.

In some embodiments, computing cluster 110-1 may execute a synchronization barrier wait instruction, such as "bar_groupID_wait barID, N", where "groupID" indicates the identifier of the targeted broadcast group (such as the identifier of broadcast group 120-1 group1), "wait" indicates the waiting instruction, "barID" indicates the targeted synchronization barrier, and "N" indicates the synchronization barrier count value (that is, the number of computing clusters included in the broadcast group). Here, the synchronization barrier count value N includes N-1 thread warps from N-1 consumer computing clusters and a thread warp from the producer computing cluster itself (that is, the thread warp running the synchronization barrier waiting instruction) . In the example shown in the figure, N=3.

FIG. 4 shows a flowchart of a synchronization method 400 for computing device 100 according to an embodiment of the present invention.

As shown in Figure 4, in block 410, by a computing cluster (as computing cluster 110-6) as consumer computing cluster in each broadcasting group (as broadcasting group 120-1), for this broadcasting group 120 When the data spaces of all producer computing clusters in -1 (such as computing clusters 110-1, 110-2 and 110-5) are ready, they send a broadcast synchronization barrier message to the intra-cluster interface 112 of the consumer computing cluster. The broadcast synchronization barrier message includes a broadcast indicator and an identifier of the broadcast group 120-1.

At block 420, the broadcast synchronization barrier message is expanded by the intra-cluster interface 112 into Synchronize barrier messages, and send each synchronization barrier message to the corresponding producer computing cluster.

Those skilled in the art can understand that the computing device shown in the above figure is only illustrative. In some embodiments, a computing device may contain more or fewer components.

The synchronous operation of the computing device and the computing cluster it contains according to the present disclosure has been described above with reference to the accompanying drawings. However, those skilled in the art can understand that the execution of the computing device and its synchronous operations is not limited to the order shown in the figure and described above, but can be executed in any other reasonable order. In addition, the computing device does not necessarily include all the components shown in the figure, and it may only include some or more components necessary to perform the functions described in the present disclosure, and the connection mode of these components is not limited to that shown in the figure. in the form shown.

The present disclosure can be implemented as a method, computing device, system and/or computer program product. A computer program product may include a computer readable storage medium having computer readable program instructions thereon for carrying out various aspects of the present disclosure. A computing device may include at least one processor and at least one memory coupled to the at least one processor, the memory may store instructions for execution by the at least one processor. When the instructions are executed by the at least one processor, the computing device may perform the asymmetric synchronization method described above.

In one or more exemplary designs, the functions described in this disclosure may be implemented in hardware, software, firmware, or any combination thereof. For example, if implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium.

Each unit of the device disclosed herein can be implemented using discrete hardware components, or integrated on one hardware component, such as a processor. For example, a general purpose processor, digital signal processor (DSP), application specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components or for The various illustrative logical blocks, modules, and circuits described in connection with the present disclosure are implemented or performed by performing any combination of the functions described herein.

Those of ordinary skill in the art should also understand that various exemplary logical blocks, modules, circuits and algorithm steps described in conjunction with the embodiments of the present disclosure may be implemented as electronic hardware, computer software or a combination of both.

The above description of the present disclosure is provided to enable any person of ordinary skill in the art to make or use the present disclosure. Various modifications to the present disclosure will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other modifications without departing from the spirit and scope of the present disclosure. Thus, the disclosure is not to be limited to the examples and designs described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (13)

1. A computing device comprising: a plurality of computing clusters divided into a plurality of broadcast groups; and one or more intra-cluster interfaces respectively located on each of the plurality of computing clusters, Wherein one computing cluster in each broadcast group, when acting as a consumer computing cluster, is configured to send a broadcast synchronization barrier to the intra-cluster interface when the data space for all producer computing clusters in the broadcast group is ready message, the broadcast synchronization barrier message includes a broadcast indicator and an identifier of the broadcast group, and The intra-cluster interface is configured to expand the broadcast synchronization barrier message into a synchronization barrier message for each producer computing cluster in the broadcast group, and send each synchronization barrier message to the corresponding producer computing cluster . 2. The computing device of claim 1, wherein the intra-cluster interface is configured to determine a corresponding producer computing cluster based on an identifier of the broadcast group in the broadcast synchronization barrier message. 3. The computing device of claim 2 , wherein the intra-cluster interface includes a plurality of mask registers, each mask register corresponding to a broadcast group in the plurality of broadcast groups, the mask register using in indicating the computing clusters belonging to the broadcast group among the plurality of computing clusters, and wherein The intra-cluster interface is configured to determine a barrier register corresponding to the broadcast group based on an identifier of the broadcast group in the broadcast synchronization barrier message, and determine a corresponding barrier register based on the barrier register corresponding to the broadcast group. The producer computes the cluster. 4. The computing device of claim 1, wherein the consumer computing cluster is further configured to: Determining whether data spaces for all producer computing clusters in the broadcast group are ready; generating the broadcast synchronization barrier message in response to data spaces being ready for all producer compute clusters in the broadcast group; and In response to the fact that the data spaces for some of the producer computing clusters in the broadcast group are not ready, continue to wait for the space for the part of the producer computing clusters to be ready. 5. The computing device of claim 1 , wherein each producer computing cluster in the broadcast group is configured to: determining whether synchronization barrier messages have been received from all other computing clusters in the broadcast group; sending the generated data to all other computing clusters in the broadcast group in response to receiving a synchronization barrier message from the other computing clusters; and Continuing to wait in response to not having received synchronization barrier messages from all other computing clusters in the broadcast group. 6. The computing device according to claim 1, the intra-cluster interface is configured to sequentially send each synchronization barrier message to the corresponding producer computing cluster through the inter-cluster interface. 7. A synchronization method for a computing device comprising a plurality of computing clusters and a plurality of intra-cluster interfaces respectively located in each computing cluster of the plurality of computing clusters, wherein the plurality of computing clusters Divided into multiple broadcast groups, the synchronization method includes: A computing cluster as a consumer computing cluster in each broadcast group sends a broadcast synchronization barrier message to the intra-cluster interface when the data space for all producer computing clusters in the broadcast group is ready, and the broadcast the synchronization barrier message includes a broadcast indicator and an identifier of said broadcast group, and The intra-cluster interface expands the broadcast synchronization barrier message into a synchronization barrier message for each producer computing cluster in the broadcast group, and sends each synchronization barrier message to the corresponding producer computing cluster. 8. The synchronization method as claimed in claim 7, wherein the expansion of the broadcast synchronization barrier message by the intra-cluster interface into a synchronization barrier message for each producer computing cluster in the broadcast group comprises: A corresponding producer computing cluster is determined by the intra-cluster interface based on the identifier of the broadcast group in the broadcast synchronization barrier message. 9. The synchronous method as claimed in claim 8, wherein the interface in the cluster comprises a plurality of mask registers, each mask register corresponds to a broadcast group in the plurality of broadcast groups, and the mask register is used Indicating the computing clusters belonging to the broadcast group among the plurality of computing clusters, and wherein determining the corresponding producer computing cluster by the intra-cluster interface based on the broadcast group identifier in the broadcast synchronization barrier message includes : determining, by the intra-cluster interface, a barrier register corresponding to the broadcast group based on the identifier of the broadcast group in the broadcast synchronization barrier message, and determining a corresponding production or computing clusters. 10. The synchronization method of claim 7, further comprising: determining by the consumer computing cluster whether data spaces for all producer computing clusters in the broadcast group are ready; generating the broadcast synchronization barrier message in response to data spaces being ready for all producer compute clusters in the broadcast group; and In response to the fact that the data spaces for some of the producer computing clusters in the broadcast group are not ready, continue to wait for the space for the part of the producer computing clusters to be ready. 11. The synchronization method of claim 7, further comprising: determining by each producer compute cluster in the broadcast group whether a synchronization barrier message has been received from all other compute clusters in the broadcast group; sending the generated data to all other computing clusters in the broadcast group in response to receiving a synchronization barrier message from the other computing clusters; and Continuing to wait in response to not having received synchronization barrier messages from all other computing clusters in the broadcast group. 12. An electronic device comprising: memory non-transitoryly storing computer-executable instructions; a processor configured to execute said computer-executable instructions; Wherein, when the computer-executable instructions are executed by the processor, the synchronization method according to any one of claims 7 to 11 is realized. 13. A computer-readable storage medium having computer program code stored thereon, the computer program code executing the synchronization method according to any one of claims 7 to 11 when executed.

Images