8000 use qualname for the serialized nn.Module by TroyGarden · Pull Request #3021 · pytorch/torchrec · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

use qualname for the serialized nn.Module #3021

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

use qualname for the serialized nn.Module #3021

wants to merge 1 commit into from

Conversation

TroyGarden
Copy link
Contributor

Summary:

context

  • original the IR serializer use type(m).name as the key for the module_to_serializer_cls mapping
  • this becomes a problem when there are multiple modules with same name
  • this change uses the full qualname of a class as the key
  • example
# previously
ebc = EmbeddingBagCollection(...)
type(ebc).__name__ == 'EmbeddingBagCollection'

# now
ebc = EmbeddingBagCollection(...)
qualname(ebc) == 'torchrec.modules.embedding_modules.EmbeddingBagCollection'

debug print

{'torchrec.modules.embedding_modules.EmbeddingBagCollection': <class 'torchrec.ir.serializer.EBCJsonSerializer'>, 'torchrec.modules.feature_processor_.PositionWeightedModule': <class 'torchrec.ir.serializer.PWMJsonSerializer'>, 'torchrec.modules.feature_processor_.PositionWeightedModuleCollection': <class 'torchrec.ir.serializer.PWMCJsonSerializer'>, 'torchrec.modules.fp_embedding_modules.FeatureProcessedEmbeddingBagCollection': <class 'torchrec.ir.serializer.FPEBCJsonSerializer'>, 'torchrec.modules.regroup.KTRegroupAsDict': <class 'torchrec.ir.serializer.KTRegroupAsDictJsonSerializer'>}

Differential Revision: D75727469

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 31, 2025
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D75727469

TroyGarden added a commit to TroyGarden/torchrec that referenced this pull request May 31, 2025
@TroyGarden @facebook-github-bot
use qualname for the serialized nn.Module (pytorch#3021)
8be117e 
Summary:

# context
* original the IR serializer use type(m).__name__ as the key for the module_to_serializer_cls mapping
* this becomes a problem when there are multiple modules with same name
* this change uses the full qualname of a class as the key
* example
```
# previously
ebc = EmbeddingBagCollection(...)
type(ebc).__name__ == 'EmbeddingBagCollection'

# now
ebc = EmbeddingBagCollection(...)
qualname(ebc) == 'torchrec.modules.embedding_modules.EmbeddingBagCollection'
```
# debug print
> {'torchrec.modules.embedding_modules.EmbeddingBagCollection': <class 'torchrec.ir.serializer.EBCJsonSerializer'>, 'torchrec.modules.feature_processor_.PositionWeightedModule': <class 'torchrec.ir.serializer.PWMJsonSerializer'>, 'torchrec.modules.feature_processor_.PositionWeightedModuleCollection': <class 'torchrec.ir.serializer.PWMCJsonSerializer'>, 'torchrec.modules.fp_embedding_modules.FeatureProcessedEmbeddingBagCollection': <class 'torchrec.ir.serializer.FPEBCJsonSerializer'>, 'torchrec.modules.regroup.KTRegroupAsDict': <class 'torchrec.ir.serializer.KTRegroupAsDictJsonSerializer'>}

Differential Revision: D75727469
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D75727469

TroyGarden added a commit to TroyGarden/torchrec that referenced this pull request May 31, 2025
@TroyGarden @facebook-github-bot
use qualname for the serialized nn.Module (pytorch#3021)
df0363b 
Summary:

# context
* original the IR serializer use type(m).__name__ as the key for the module_to_serializer_cls mapping
* this becomes a problem when there are multiple modules with same name
* this change uses the full qualname of a class as the key
* example
```
# previously
ebc = EmbeddingBagCollection(...)
type(ebc).__name__ == 'EmbeddingBagCollection'

# now
ebc = EmbeddingBagCollection(...)
qualname(ebc) == 'torchrec.modules.embedding_modules.EmbeddingBagCollection'
```
# debug print
> {'torchrec.modules.embedding_modules.EmbeddingBagCollection': <class 'torchrec.ir.serializer.EBCJsonSerializer'>, 'torchrec.modules.feature_processor_.PositionWeightedModule': <class 'torchrec.ir.serializer.PWMJsonSerializer'>, 'torchrec.modules.feature_processor_.PositionWeightedModuleCollection': <class 'torchrec.ir.serializer.PWMCJsonSerializer'>, 'torchrec.modules.fp_embedding_modules.FeatureProcessedEmbeddingBagCollection': <class 'torchrec.ir.serializer.FPEBCJsonSerializer'>, 'torchrec.modules.regroup.KTRegroupAsDict': <class 'torchrec.ir.serializer.KTRegroupAsDictJsonSerializer'>}

# backward compatibility issue
* a previously serialized model (module) won't be correctly deserialized
* the workaround is to keep the old key name in the `module_to_serializer_cls` map for a while until the old method is fully deprecated

Differential Revision: D75727469
@TroyGarden @facebook-github-bot
use qualname for the serialized nn.Module (pytorch#3021)
816a69d 
Summary:
Pull Request resolved: pytorch#3021

# context
* original the IR serializer use type(m).__name__ as the key for the module_to_serializer_cls mapping
* this becomes a problem when there are multiple modules with same name
* this change uses the full qualname of a class as the key
* example
```
# previously
ebc = EmbeddingBagCollection(...)
type(ebc).__name__ == 'EmbeddingBagCollection'

# now
ebc = EmbeddingBagCollection(...)
qualname(ebc) == 'torchrec.modules.embedding_modules.EmbeddingBagCollection'
```
# debug print
> {'torchrec.modules.embedding_modules.EmbeddingBagCollection': <class 'torchrec.ir.serializer.EBCJsonSerializer'>, 'torchrec.modules.feature_processor_.PositionWeightedModule': <class 'torchrec.ir.serializer.PWMJsonSerializer'>, 'torchrec.modules.feature_processor_.PositionWeightedModuleCollection': <class 'torchrec.ir.serializer.PWMCJsonSerializer'>, 'torchrec.modules.fp_embedding_modules.FeatureProcessedEmbeddingBagCollection': <class 'torchrec.ir.serializer.FPEBCJsonSerializer'>, 'torchrec.modules.regroup.KTRegroupAsDict': <class 'torchrec.ir.serializer.KTRegroupAsDictJsonSerializer'>}

# backward compatibility issue
* a previously serialized model (module) won't be correctly deserialized
* the workaround is to keep the old key name in the `module_to_serializer_cls` map for a while until the old method is fully deprecated

Differential Revision: D75727469
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D75727469

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. fb-exported
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@TroyGarden @facebook-github-bot
0