Collapse MaterializeEncodingIntoPadding into the generic pass #20160

hanhanW · 2025-03-04T18:59:16Z

@kuhar has been developing the GPU padding approach with the data-tiling flow. We branched a MaterializeEncodingIntoPadding pass that cooks the padding logic separately.

Now we observe that the divergence is about how we lower flow.dispatch.tensor load/store ops, see patterns for details. I think we should be able to collapse the patterns if we move pad/pack/swizzle logic to attribute interface methods. Because the whole point is about getting correct shape type and dispatch the index/offset/strides.

@pashu123 can you pick this up and expand the subtasks in the issue?

hanhanW · 2025-03-04T19:06:32Z

An additional work is that we need to get the encoding resolver attribute when it shows up in the configuration. This is missing in the generic pass.

The revision updates the converted type for MaterializePadEncodingTypeConverter because it should always return the final types of the dialect conversion. In this context, the padding encoding should be dropped for any tensor type, and the binding type should get padded type. Thus, it implements `MaterializeInterfaceBindingEncoding` pattern for the pass. It is a step towards iree-org#20160, since we now have better understanding about the abstraction. Signed-off-by: hanhanW <hanhan0912@gmail.com>

hanhanW · 2025-03-27T00:46:24Z

There are several divergences between MaterializeEncoding pass and MaterializeEncodingIntoPadding pass. There are too many data-tiling specific logic exposed to patterns and type converter. E.g., we do not care MaterializeEncodingInfo at all in the padding approach. We also don't need the getPackedDimsFor* methods exposed to patterns. They are all data-tiling specific.

What needs to be fixed in my mind are:

We should add convertType to the SerializableEncodingAttrInterface. Because only the serialized encoding attribute knows what the final types are.
We need an interface method (e.g., getOffsetsSizesStrides(), which takes the dynamic dimension sizes, to generate the ranges for load/store. I recently learned that the binding types and the final compute types are different on padding approach. [Codegen] Teach MaterializeEncodingIntoPadding about resolved layouts. #20393 is an example about why we need the interface method.
IMO, the serialized encoding should be able to generate the ops that bring the global layout to the preferred layout and vice versa.
The tensor.empty materialization pattern could use above interface methods, because it only cares the final type and transformed dynamic dimensions.
The other computation op lowering (e.g., elementwise ops) should also be moved to lowerOp implementation. Or we need to add an interface method to transform indexing maps, like those dynamic dimension sizes.

Where should the interface methods be introduced is not clear to me at the moment. I need to think more about the difference between encoding types and encoding resolver. I've been thinking it today, and now I think encoding resolver is a superset of encoding types. Because the encoding resolver can be attached to the encoding types (like EncodingAttr), and they are also used to serialize the encoding types. I'll think more about it. I think the above points are what we're looking for in the work.

#20393) The revision updates the converted type for MaterializePadEncodingTypeConverter because it should always return the final type of the dialect conversion. In this context, the padding encoding should be dropped for any tensor type, and the binding type should get padded type. Thus, it implements `MaterializeInterfaceBindingEncoding` pattern for the pass. It is a step towards #20160, since we now have better understanding about the abstraction. Fixes #20362 --------- Signed-off-by: hanhanW <hanhan0912@gmail.com>

hanhanW · 2025-04-15T23:08:34Z

The other computation op lowering (e.g., elementwise ops) should also be moved to lowerOp implementation. Or we need to add an interface method to transform indexing maps, like those dynamic dimension sizes.

I'm looking at supporting multiple reduction dimensions cases for padding approach, and now I realize that this is a right direction to go again because we'll need to collapse reduction dimensions and then pad the collapsed dimension. I'll try to create an EPIC issue for the multi-reduction support.

EDIT: multi-reduction is no longer needed because we will solve the problem in a different approach.

Instead of using the nop layout resolver, the revision explicitly uses the GPUPadLayoutAttr to resolve all the layouts. If the resolver is present in an executable target, it is prioritized. The revision exposes a "hidden" dependency between padding layout resolver and nop layout resolver. Because the current type converter is not decoupled enough from data-tiling usage. It is a step towards #20160 --------- Signed-off-by: hanhanW <hanhan0912@gmail.com>

Migrates a set of Flow constructs to TensorExt to reduce the dependency in Stream/Codegen on Flow and to avoid a circular dependency from Encoding on Flow encountered during the MaterializeEncodingIntoPadding collapsing work (#20160 (comment)): - IREE::Flow::DispatchWorkloadOrdinalOp - IREE::Flow::DispatchTensorLoadOp - IREE::Flow::DispatchTensorStoreOp - IREE::Flow::DispatchWorkgroupCountFromSliceOp - IREE::Flow::DispatchWorkgroupCountFromDagRootOp -> moved because it shares a base class with DispatchWorkgroupCountFromSliceOp - IREE::Flow::DispatchTensorType Resolves: [#20249](#20249) Note for downstream users, you can use the following commands (in the provided order) to help update your codebase for these changes: ``` # Update the op names in the MLIR files grep -rl flow.dispatch.workload.ordinal | xargs -d '\n' sed -i 's/flow.dispatch.workload.ordinal/iree_tensor_ext.dispatch.workload.ordinal/g' grep -rl flow.dispatch.tensor | xargs -d '\n' sed -i 's/flow.dispatch.tensor/iree_tensor_ext.dispatch.tensor/g' grep -rl flow.dispatch.workgroup_count_from | xargs -d '\n' sed -i 's/flow.dispatch.workgroup_count_from/iree_tensor_ext.dispatch.workgroup_count_from/g' # Add 'TensorExtOps.h' header to files using the ops { grep -rl 'IREE::Flow::DispatchWorkloadOrdinalOp' ./; grep -rl 'IREE::Flow::DispatchTensorLoadOp' ./; grep -rl 'IREE::Flow::DispatchTensorStoreOp' ./; grep -rl 'IREE::Flow::DispatchWorkgroupCount' ./; } | xargs -d '\n' sed -i '/#include "iree\/compiler\/Dialect\/Flow\/IR\/FlowOps.h"/a #include "iree/compiler/Dialect/TensorExt/IR/TensorExtOps.h"' # Add 'TensorExtTypes.h' header to files using 'DispatchTensorType' grep -rl 'IREE::Flow::DispatchTensorType' ./ | xargs -d '\n' sed -i '/#include "iree\/compiler\/Dialect\/Flow\/IR\/FlowTypes.h"/a #include "iree/compiler/Dialect/TensorExt/IR/TensorExtTypes.h"' # Update the op names grep -rl 'IREE::Flow::DispatchWorkloadOrdinalOp' | xargs -d '\n' sed -i 's/IREE::Flow::DispatchWorkloadOrdinalOp/IREE::TensorExt::DispatchWorkloadOrdinalOp/g' grep -rl 'IREE::Flow::DispatchTensorLoadOp' | xargs -d '\n' sed -i 's/IREE::Flow::DispatchTensorLoadOp/IREE::TensorExt::DispatchTensorLoadOp/g' grep -rl 'IREE::Flow::DispatchTensorStoreOp' | xargs -d '\n' sed -i 's/IREE::Flow::DispatchTensorStoreOp/IREE::TensorExt::DispatchTensorStoreOp/g' grep -rl 'IREE::Flow::DispatchWorkgroupCount' | xargs -d '\n' sed -i 's/IREE::Flow::DispatchWorkgroupCount/IREE::TensorExt::DispatchWorkgroupCount/g' # Update 'DispatchTensorType' grep -rl 'IREE::Flow::DispatchTensorType' | xargs -d '\n' sed -i 's/IREE::Flow::DispatchTensorType/IREE::TensorExt::DispatchTensorType/g' ``` Afterwards, you might need to perform following updates manually: - Remove `FlowOps.h` and `FlowTypes.h` include lines from files where they are not needed anymore. - Update `BUILD.bazel` and/or `CmakeLists.txt` files to include the new `TensorExt` dependency and potentially remove `Flow` dependencies if possible. Note that you can use `python build_tools/bazel_to_cmake/bazel_to_cmake.py` to automatically update cmake files once you have updated the `BUILD.bazel` (if using bazel). - Lint your code using `pre-commit run --all-files`, `clang-format` or your own linter if applicable. Signed-off-by: Jorn Tuyls <jorn.tuyls@gmail.com>

…20612) There is no strong dependency of `DispatchTensorType` on the Encoding dialect as the encoding can be just any attribute and is optional, so I don't think there's a need to have the TensorExt dialect explicitly depend on the Encoding just for the interface. Decoupling them would avoid potential circular dependency issues (I ran into that while working on #20160 (comment)), remove the need for other (potential) users of the TensorExt dialect to use the encoding and allow users to bring their own encoding interfaces while avoiding having to bring in the Encoding dialect. Signed-off-by: Jorn Tuyls <jorn.tuyls@gmail.com>

jtuyls · 2025-05-01T18:28:55Z

RFC: Collapse MaterializeEncodingIntoPadding into MaterializeEncoding

Adding more detailed information for the above.

Rationale

The MaterializeEncoding pass should ideally be be generic to support different encoding types with minimal changes and without custom transformations/passes. The current encodings encountered perform some kind of reshaping, packing, swizzling and or padding. There is a lot of common logic for materializing these and it seems very doable to accommodate all existing variations and potential future versions as long as they perform some combination of these operations.

Proposal

Add two methods to the SerializableEncodingAttrInterface that encoding attributes should implement to specialize the materialization logic:

convertType: Given a type, this returns the materialized type for the encoding attribute.
getOffsetsSizesStrides: Given the dynamic dimension sizes, this generates the offsets, sizes and strides for the DispatchTensorLoadOp and DispatchTensorStoreOp

convertType

The SerializableEncodingAttrInterface will have a new interface method:

Type convertype(type type);

Different serializable encoding attributes will provide their own implementation that returns the materialized type. This avoids the need for MaterializePadEncodingTypeConverter and combines all type conversion into MaterializeEncodingTypeConverter.

For example, based on the current implementation in MaterializeEncodingIntoPadding for PadEncodingLayoutAttr, this will look like:

Type PadEncodingLayoutAttr::convertType(Type type) const {
  return TypeSwitch<mlir::Type, mlir::Type>(type)
      .Case<RankedTensorType>(
          [&](auto concreteType) { return concreteType.dropEncoding(); })
      .Case<IREE::TensorExt::DispatchTensorType>([&](auto concreteType) {
        auto rankedTensorType =
            dyn_cast<RankedTensorType>(concreteType.getBoundType());
        if (!rankedTensorType) {
          return concreteType;
        }
        rankedTensorType = getPaddedType(*this, rankedTensorType);
        return IREE::TensorExt::DispatchTensorType::get(
            concreteType.getAccess(), rankedTensorType);
      })
      .Default([&](auto concreteType) { return concreteType; });
}

The MaterializeEncodingTypeConverter will call this interface method to specialize the type conversion for different encoding attributes. This can look like:

MaterializeEncodingTypeConverter::MaterializeEncodingTypeConverter(
    IREE::Codegen::LayoutAttrInterface layoutAttr)
    : layoutAttr(layoutAttr) {
  addConversion([](IntegerType intType) { return intType; });
  addConversion([](IndexType indexType) { return indexType; });
  addConversion([](FloatType floatType) { return floatType; });
  addConversion([](MemRefType memrefType) { return memrefType; });
  addConversion([=](RankedTensorType type) {
    SerializableEncodingAttrInterface serializableEncodingAttr =
        getSerializableEncodingAttr(getLayoutAttr(), type);
    if (serializableEncodingAttr) {
      return cast<RankedTensorType>(serializableEncodingAttr.convertType(type));
    }
    return type.dropEncoding();
  });
  addConversion([&](IREE::TensorExt::DispatchTensorType dispatchTensorType) {
    auto boundType =
        dyn_cast<RankedTensorType>(dispatchTensorType.getBoundType());
    if (!boundType || !boundType.getEncoding()) {
      return dispatchTensorType;
    }
    SerializableEncodingAttrInterface serializableEncodingAttr =
        getSerializableEncodingAttr(getLayoutAttr(), boundType);
    if (serializableEncodingAttr) {
      return cast<IREE::TensorExt::DispatchTensorType>(
          serializableEncodingAttr.convertType(dispatchTensorType));
    }
    Type convertedBoundType = convertType(boundType);
    return IREE::TensorExt::DispatchTensorType::get(
        dispatchTensorType.getAccess(), convertedBoundType);
  });
}

getOffsetsSizesStrides

The implementations for materializing DispatchTensorLoadOp and DispatchTensorStoreOp for CPUEncodingLayoutAttr/GPUEncodingLayoutAttr and PadEncodingLayoutAttr are quite similar except for the offsets, sizes, strides and dynamic dims the new DispatchTensorLoadOp and DispatchTensorStoreOp are created with. The getOffsetsSizesStrides interface method can compute these differently for every encoding based on the type and dynamic dimensions:

/// Returns success if materialized `newOffsets`, `newSizes` and
/// `newStrides` can be calculated and set for the slice specified by
/// `offsets`, `sizes` and `strides` on the dispatch tensor `type` with
/// potential `dynamicDims` sizes.
LogicalResult
getOffsetsSizesStrides(OpBuilder &builder, Location loc,
                         IREE::TensorExt::DispatchTensorType type,
                         IREE::Codegen::LayoutAttrInterface layoutAttr,
                         ValueRange dynamicDims,
                         ArrayRef<OpFoldResult> offsets,
                         ArrayRef<OpFoldResult> sizes,
                         ArrayRef<OpFoldResult> strides,
                         SmallVectorImpl<OpFoldResult> &newOffsets,
                         SmallVectorImpl<OpFoldResult> &newSizes,
                         SmallVectorImpl<OpFoldResult> &newStrides);

This should work for all patterns that convert DispatchTensorLoadOp and DispatchTensorStoreOp into a materialized version of the same op. However, PadEncodingLayoutAttr needs an additional tensor::InsertSliceOp to be inserted for DispatchTensorStoreOp and an additional tensor::ExtractSliceOpto be inserted forDispatchTensorLoadOp`. I think though that this can be derived by comparing the unmaterialized type with the materialized type and inserting slicing in case of padding. This seems quite general to potential future encodings as long as they perform some combination of reshaping, packing, swizzling and/or padding.

Execution Plan

Original plan:

Add convertType to SerializableEncodingAttrInterface: [Encoding] Add convertType interface to generalize type conversion #20700
Add getOffsetsSizesStrides: [Encoding] Add getOffsetSizesStrides interface for load/store materialization #20741

As we discussed and found out in #20741 (comment), the above methods really belong to the LayoutAttrInterface instead of SerializableEncodingAttrInterface as the former is responsible for materialization. Therefore, the plan has been adjusted to accomplish that as well:

Move LayoutAttrInterface from Codegen to Encoding to avoid a dependency in Encoding on Codegen when introducing the new interface methods above.
Move convertType to LayoutAttrInterface
- [NFC][Encoding] Move convertType to LayoutAttrInterface #20794
Add getOffsetsSizesStrides or getMaterializedDynamicDims (if we wouldn't need getOffsetsSizesStrides)
- [Encoding] Add getOffsetSizesStrides interface for load/store materialization #20741
Enable partial slices as a follow up (added for completeness)

@MaheshRavishankar

…20700) Adds the `convertType` interface method to `SerializableEncodingAttrInterface`. This is the first stage for collapsing MaterializeEncodingIntoPadding into MaterializeEncoding, see #20160. Note that this removes the need for MaterializePadEncodingTypeConverter if the `convertType` method is implemented for `PadEncodingLayoutAttr`, however, this is intentionally left out of the PR as @MaheshRavishankar is looking at adjusting the MaterializeEncodingIntoPadding for dynamic padding. To check what it looks like with MaterializePadEncodingTypeConverter collapsed into MaterializeEncodingTypeConverter you can view this branch instead: https://github.com/iree-org/iree/compare/main...jtuyls:materialize-encoding-into-padding-convert-type?expand=1. Second note that while the type conversion for `EncodingNopLayoutAttr` was previously combined with the common implementation for `CPUEncodingLayoutAttr`, `VMVXEncodingLayoutAttr` and `GPUEncodingLayoutAttr`, this PR provides a minimal standalone implementation for clarity on what's going on. `CPUEncodingLayoutAttr`, `VMVXEncodingLayoutAttr` and `GPUEncodingLayoutAttr` still share the same implementation as they did previously through the `convertType` implementation in HostSerializableEncodingAttrInterfaceExternalModelBase. `EncodingNopLayoutAttr` now implements both IREE::Encoding::EncodingLayoutResolverAttrInterface and IREE::Encoding::SerializableEncodingAttrInterface to go down the same path as the above attributes, but provides its own implementation. Signed-off-by: Jorn Tuyls <jorn.tuyls@gmail.com>

…0720) This PR performs some refactoring as part of collapsing MaterializeEncodingIntoPadding into MaterializeEncoding (#20160) and moves the `materializeEncodingValueFn` parameter from OpMaterializeEncodingPattern to the MaterializeEncodingTypeConverter, together with the `getInnerTileSizesOfr` implementation, for following reasons: - The `typeConverter` and `materializeEncodingValueFn` objects are typically passed around together, so now only `typeConverter` needs to be passed around and this simplifies the code. - As part of #20160, I am creating a `getOffsetsSizesStrides` method in the MaterializeEncodingTypeConverter and SerializableEncodingAttrInterface and all encoding specific logic should be moved there. I would specifically like to avoid the need for passing a `MaterializeEncodingValueFn` to `getOffsetsSizesStrides`. This PR moves us in that direction. Signed-off-by: Jorn Tuyls <jorn.tuyls@gmail.com>

Instead of using the nop layout resolver, the revision explicitly uses the GPUPadLayoutAttr to resolve all the layouts. If the resolver is present in an executable target, it is prioritized. The revision exposes a "hidden" dependency between padding layout resolver and nop layout resolver. Because the current type converter is not decoupled enough from data-tiling usage. It is a step towards iree-org#20160 --------- Signed-off-by: hanhanW <hanhan0912@gmail.com>

) Migrates a set of Flow constructs to TensorExt to reduce the dependency in Stream/Codegen on Flow and to avoid a circular dependency from Encoding on Flow encountered during the MaterializeEncodingIntoPadding collapsing work (iree-org#20160 (comment)): - IREE::Flow::DispatchWorkloadOrdinalOp - IREE::Flow::DispatchTensorLoadOp - IREE::Flow::DispatchTensorStoreOp - IREE::Flow::DispatchWorkgroupCountFromSliceOp - IREE::Flow::DispatchWorkgroupCountFromDagRootOp -> moved because it shares a base class with DispatchWorkgroupCountFromSliceOp - IREE::Flow::DispatchTensorType Resolves: [iree-org#20249](iree-org#20249) Note for downstream users, you can use the following commands (in the provided order) to help update your codebase for these changes: ``` # Update the op names in the MLIR files grep -rl flow.dispatch.workload.ordinal | xargs -d '\n' sed -i 's/flow.dispatch.workload.ordinal/iree_tensor_ext.dispatch.workload.ordinal/g' grep -rl flow.dispatch.tensor | xargs -d '\n' sed -i 's/flow.dispatch.tensor/iree_tensor_ext.dispatch.tensor/g' grep -rl flow.dispatch.workgroup_count_from | xargs -d '\n' sed -i 's/flow.dispatch.workgroup_count_from/iree_tensor_ext.dispatch.workgroup_count_from/g' # Add 'TensorExtOps.h' header to files using the ops { grep -rl 'IREE::Flow::DispatchWorkloadOrdinalOp' ./; grep -rl 'IREE::Flow::DispatchTensorLoadOp' ./; grep -rl 'IREE::Flow::DispatchTensorStoreOp' ./; grep -rl 'IREE::Flow::DispatchWorkgroupCount' ./; } | xargs -d '\n' sed -i '/#include "iree\/compiler\/Dialect\/Flow\/IR\/FlowOps.h"/a #include "iree/compiler/Dialect/TensorExt/IR/TensorExtOps.h"' # Add 'TensorExtTypes.h' header to files using 'DispatchTensorType' grep -rl 'IREE::Flow::DispatchTensorType' ./ | xargs -d '\n' sed -i '/#include "iree\/compiler\/Dialect\/Flow\/IR\/FlowTypes.h"/a #include "iree/compiler/Dialect/TensorExt/IR/TensorExtTypes.h"' # Update the op names grep -rl 'IREE::Flow::DispatchWorkloadOrdinalOp' | xargs -d '\n' sed -i 's/IREE::Flow::DispatchWorkloadOrdinalOp/IREE::TensorExt::DispatchWorkloadOrdinalOp/g' grep -rl 'IREE::Flow::DispatchTensorLoadOp' | xargs -d '\n' sed -i 's/IREE::Flow::DispatchTensorLoadOp/IREE::TensorExt::DispatchTensorLoadOp/g' grep -rl 'IREE::Flow::DispatchTensorStoreOp' | xargs -d '\n' sed -i 's/IREE::Flow::DispatchTensorStoreOp/IREE::TensorExt::DispatchTensorStoreOp/g' grep -rl 'IREE::Flow::DispatchWorkgroupCount' | xargs -d '\n' sed -i 's/IREE::Flow::DispatchWorkgroupCount/IREE::TensorExt::DispatchWorkgroupCount/g' # Update 'DispatchTensorType' grep -rl 'IREE::Flow::DispatchTensorType' | xargs -d '\n' sed -i 's/IREE::Flow::DispatchTensorType/IREE::TensorExt::DispatchTensorType/g' ``` Afterwards, you might need to perform following updates manually: - Remove `FlowOps.h` and `FlowTypes.h` include lines from files where they are not needed anymore. - Update `BUILD.bazel` and/or `CmakeLists.txt` files to include the new `TensorExt` dependency and potentially remove `Flow` dependencies if possible. Note that you can use `python build_tools/bazel_to_cmake/bazel_to_cmake.py` to automatically update cmake files once you have updated the `BUILD.bazel` (if using bazel). - Lint your code using `pre-commit run --all-files`, `clang-format` or your own linter if applicable. Signed-off-by: Jorn Tuyls <jorn.tuyls@gmail.com>

…ree-org#20612) There is no strong dependency of `DispatchTensorType` on the Encoding dialect as the encoding can be just any attribute and is optional, so I don't think there's a need to have the TensorExt dialect explicitly depend on the Encoding just for the interface. Decoupling them would avoid potential circular dependency issues (I ran into that while working on iree-org#20160 (comment)), remove the need for other (potential) users of the TensorExt dialect to use the encoding and allow users to bring their own encoding interfaces while avoiding having to bring in the Encoding dialect. Signed-off-by: Jorn Tuyls <jorn.tuyls@gmail.com>

@MaheshRavishankar

…ree-org#20700) Adds the `convertType` interface method to `SerializableEncodingAttrInterface`. This is the first stage for collapsing MaterializeEncodingIntoPadding into MaterializeEncoding, see iree-org#20160. Note that this removes the need for MaterializePadEncodingTypeConverter if the `convertType` method is implemented for `PadEncodingLayoutAttr`, however, this is intentionally left out of the PR as @MaheshRavishankar is looking at adjusting the MaterializeEncodingIntoPadding for dynamic padding. To check what it looks like with MaterializePadEncodingTypeConverter collapsed into MaterializeEncodingTypeConverter you can view this branch instead: https://github.com/iree-org/iree/compare/main...jtuyls:materialize-encoding-into-padding-convert-type?expand=1. Second note that while the type conversion for `EncodingNopLayoutAttr` was previously combined with the common implementation for `CPUEncodingLayoutAttr`, `VMVXEncodingLayoutAttr` and `GPUEncodingLayoutAttr`, this PR provides a minimal standalone implementation for clarity on what's going on. `CPUEncodingLayoutAttr`, `VMVXEncodingLayoutAttr` and `GPUEncodingLayoutAttr` still share the same implementation as they did previously through the `convertType` implementation in HostSerializableEncodingAttrInterfaceExternalModelBase. `EncodingNopLayoutAttr` now implements both IREE::Encoding::EncodingLayoutResolverAttrInterface and IREE::Encoding::SerializableEncodingAttrInterface to go down the same path as the above attributes, but provides its own implementation. Signed-off-by: Jorn Tuyls <jorn.tuyls@gmail.com>

…ee-org#20720) This PR performs some refactoring as part of collapsing MaterializeEncodingIntoPadding into MaterializeEncoding (iree-org#20160) and moves the `materializeEncodingValueFn` parameter from OpMaterializeEncodingPattern to the MaterializeEncodingTypeConverter, together with the `getInnerTileSizesOfr` implementation, for following reasons: - The `typeConverter` and `materializeEncodingValueFn` objects are typically passed around together, so now only `typeConverter` needs to be passed around and this simplifies the code. - As part of iree-org#20160, I am creating a `getOffsetsSizesStrides` method in the MaterializeEncodingTypeConverter and SerializableEncodingAttrInterface and all encoding specific logic should be moved there. I would specifically like to avoid the need for passing a `MaterializeEncodingValueFn` to `getOffsetsSizesStrides`. This PR moves us in that direction. Signed-off-by: Jorn Tuyls <jorn.tuyls@gmail.com>

…lization (#20741) This PR adds the `getOffsetSizesStrides` interface method as part of the effort to collapse MaterializeEncodingIntoPadding into MaterializeEncoding: #20160. This should enable the collapsing of the padding materialization patterns for `dispatch.tensor.load` and `dispatch.tensor.store` into the generic one, however, this is intentionally left out of the PR as the MaterializeEncodingIntoPadding pass is being adjusted for dynamic padding. The declaration looks like this: ```cpp /// Returns success if materialized `newOffsets`, `newSizes` and /// `newStrides` can be calculated and set for the slice specified by /// `offsets`, `sizes` and `strides` on the dispatch tensor `type` with /// potential `dynamicDims` sizes. LogicalResult getOffsetsSizesStrides(OpBuilder &builder, Location loc, IREE::TensorExt::DispatchTensorType type, IREE::Codegen::LayoutAttrInterface layoutAttr, ValueRange dynamicDims, ArrayRef<OpFoldResult> offsets, ArrayRef<OpFoldResult> sizes, ArrayRef<OpFoldResult> strides, SmallVectorImpl<OpFoldResult> &newOffsets, SmallVectorImpl<OpFoldResult> &newSizes, SmallVectorImpl<OpFoldResult> &newStrides); ``` Two notes on this: - The old offsets, sizes and strides are passed, but internally the implementations check for and only implement whole slices. Support for partial slices if left for future work to avoid changing the functionality (and adding corresponding tests) in this PR. - The VMVX ukernel information is retrieved from the layoutAttr (IREE::Encoding::LayoutAttrInterface). Signed-off-by: Jorn Tuyls <jorn.tuyls@gmail.com>

hanhanW added the codegen Shared code generation infrastructure and dialects label Mar 4, 2025

hanhanW assigned hanhanW and pashu123 Mar 4, 2025

This was referenced Mar 10, 2025

Switching to get encoding resolver from configuration #20202

Closed

[DataTiling][GPU] Add layout resolver option to ROCMTarget #20250

Merged

hanhanW mentioned this issue Mar 24, 2025

[GPU] Failed to materialize encodings for a dispatch in padding approach #20362

Closed

hanhanW mentioned this issue Mar 26, 2025

[Codegen] Teach MaterializeEncodingIntoPadding about resolved layouts. #20393

Merged

hanhanW unassigned pashu123 Mar 27, 2025

hanhanW assigned jtuyls Apr 10, 2025

hanhanW mentioned this issue Apr 14, 2025

Move Flow ops that are depended by CodeGen/Stream to TensorExt dialect #20249

Closed

jtuyls mentioned this issue Apr 16, 2025

Migrate Flow dependencies in Stream/Codegen to TensorExt #20564

Merged

hanhanW mentioned this issue Apr 16, 2025

[Codegen] Use GPUPadLayoutAttr to resolve layouts. #20565

Merged

jtuyls mentioned this issue Apr 23, 2025

[TensorExt][NFC] Remove dependency on Encoding through ExternalModel #20612

Merged

hanhanW mentioned this issue Apr 23, 2025

[DT][CodeGen] Use EncodingTypeInterface during the materialization. #20395

Closed

jtuyls mentioned this issue May 1, 2025

[Encoding] Add convertType interface to generalize type conversion #20700

Merged

jtuyls mentioned this issue May 5, 2025

[NFC][Encoding] Move materializeEncodingValueFn to type converter #20720

Merged

jtuyls mentioned this issue May 6, 2025

[Encoding] Add getOffsetSizesStrides interface for load/store materialization #20741

Merged

jtuyls mentioned this issue May 12, 2025

[NFC][Codegen] Move EncodingNop LayoutAttrInterface to external model #20778

Merged

hanhanW mentioned this issue May 19, 2025

[Encoding] Revisit the need of EncodingNopLayout resolver #20846

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Collapse MaterializeEncodingIntoPadding into the generic pass #20160

Collapse MaterializeEncodingIntoPadding into the generic pass #20160

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Collapse MaterializeEncodingIntoPadding into the generic pass #20160

Collapse MaterializeEncodingIntoPadding into the generic pass #20160

Comments

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

RFC: Collapse MaterializeEncodingIntoPadding into MaterializeEncoding

Rationale

Proposal

convertType

getOffsetsSizesStrides

Execution Plan

Uh oh!