这是indexloc提供的服务,不要输入任何密码
Skip to content

Introduces a new utility function, MatchPermutedSliceAndPartitionOffset, to detect a pattern where a DynamicSlice consumes the output of an AllGather with a permuted set of offsets. This pattern is equivalent to a CollectivePermute and can be optimized accordingly. #97189

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 20, 2025

Conversation

copybara-service[bot]
Copy link

Introduces a new utility function, MatchPermutedSliceAndPartitionOffset, to detect a pattern where a DynamicSlice consumes the output of an AllGather with a permuted set of offsets. This pattern is equivalent to a CollectivePermute and can be optimized accordingly.

The logic is divided into four main sections:

  1. Initial Checks: Verifies that the AllGather is a suitable candidate for this optimization. It ensures the operation is performed across multiple partitions with a single replica and uses flattened IDs (i.e., use_global_device_ids is enabled).
  2. Locate DynamicSlice: Finds the DynamicSlice user of the AllGather, correctly traversing through any intervening Reshape or Bitcast operations that do not alter the data.
  3. Extract Offset Specifications: Derives the memory offset-to-partition ID mappings from both the AllGather (the source layout) and the DynamicSlice (the destination/permuted layout).
  4. Match Specifications: Compares the source and destination offset maps to derive the permutation. For each memory offset, it pairs the source partition ID from the AllGather with the destination partition ID from the DynamicSlice.

This CL also introduces a few key data structures to support this logic:

  • PartitionOffsetSpec: Represents the mapping from a memory offset to a partition ID for each replica group. This is used to model the data layout produced by the AllGather and the permuted access pattern of the DynamicSlice.
  • PermutationPairs: A type alias for a list of (source_id, destination_id) pairs, representing a permute operation for CP.

@copybara-service copybara-service bot force-pushed the exported_pr_783030292 branch 3 times, most recently from ec5c229 to b02a258 Compare July 20, 2025 16:58
…set`, to detect a pattern where a `DynamicSlice` consumes the output of an `AllGather` with a permuted set of offsets. This pattern is equivalent to a `CollectivePermute` and can be optimized accordingly.

The logic is divided into four main sections:
1.  **Initial Checks:** Verifies that the `AllGather` is a suitable candidate for this optimization. It ensures the operation is performed across multiple partitions with a single replica and uses flattened IDs (i.e., `use_global_device_ids` is enabled).
2.  **Locate DynamicSlice:** Finds the `DynamicSlice` user of the `AllGather`, correctly traversing through any intervening `Reshape` or `Bitcast` operations that do not alter the data.
3.  **Extract Offset Specifications:** Derives the memory offset-to-partition ID mappings from both the `AllGather` (the source layout) and the `DynamicSlice` (the destination/permuted layout).
4.  **Match Specifications:** Compares the source and destination offset maps to derive the permutation. For each memory offset, it pairs the source partition ID from the `AllGather` with the destination partition ID from the `DynamicSlice`.

This CL also introduces a few key data structures to support this logic:

*   **`PartitionOffsetSpec`**: Represents the mapping from a memory offset to a partition ID for each replica group. This is used to model the data layout produced by the `AllGather` and the permuted access pattern of the `DynamicSlice`.
*   **`PermutationPairs`**: A type alias for a list of `(source_id, destination_id)` pairs, representing a permute operation for CP.

PiperOrigin-RevId: 785196052
@copybara-service copybara-service bot force-pushed the exported_pr_783030292 branch from b02a258 to 4e1d4b2 Compare July 20, 2025 20:15
@copybara-service copybara-service bot merged commit 4e1d4b2 into master Jul 20, 2025
@copybara-service copybara-service bot deleted the exported_pr_783030292 branch July 20, 2025 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant