8000 Changes for PyTorch 2.1.0 and Python 3.11 by agunapal · Pull Request #2621 · pytorch/serve · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Changes for PyTorch 2.1.0 and Python 3.11 #2621

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Sep 29, 2023
Merged

Changes for PyTorch 2.1.0 and Python 3.11 #2621

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
8847b09
Changes for PyTorch 2.1.0 and Python 3.11
agunapal Sep 27, 2023
720d777
Merge branch 'master' into feature/update_pt2.1
agunapal Sep 27, 2023
bb3781d
Updated workflows with cu121
agunapal Sep 27, 2023
a7267bf
Merge branch 'feature/update_pt2.1' of https://github.com/pytorch/ser…
agunapal Sep 27, 2023
a72f1c3
update torch audio version
agunapal Sep 27, 2023
f54c3b7
update torch versions for cpu
agunapal Sep 27, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci_gpu.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ jobs:
uses: actions/checkout@v3
- name: Install dependencies
run: |
python ts_scripts/install_dependencies.py --environment=dev --cuda=cu117
python ts_scripts/install_dependencies.py --environment=dev --cuda=cu121
- name: Torchserve Sanity
uses: nick-fields/retry@v2
with:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/regression_tests_gpu.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ jobs:
uses: actions/checkout@v3
- name: Install dependencies
run: |
python ts_scripts/install_dependencies.py --environment=dev --cuda=cu117
python ts_scripts/install_dependencies.py --environment=dev --cuda=cu121
- name: Torchserve Regression Tests
run: |
python test/regression_tests.py
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,9 @@ Your contributions will fall into two categories:

For GPU
```bash
python ts_scripts/install_dependencies.py --environment=dev --cuda=cu118
python ts_scripts/install_dependencies.py --environment=dev --cuda=cu121
```
> Supported cuda versions as cu118, cu117, cu116, cu113, cu111, cu102, cu101, cu92
> Supported cuda versions as cu121, cu118, cu117, cu116, cu113, cu111, cu102, cu101, cu92
- Install `pre-commit` to your Git flow:
```bash
pre-commit install
Expand Down
4 changes: 2 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ curl http://127.0.0.1:8080/predictions/bert -T input.txt
```
# Install dependencies
# cuda is optional
python ./ts_scripts/install_dependencies.py --cuda=cu102
python ./ts_scripts/install_dependencies.py --cuda=cu121

# Latest release
pip install torchserve torch-model-archiver torch-workflow-archiver
Expand All @@ -31,7 +31,7 @@ pip install torchserve-nightly torch-model-archiver-nightly torch-workflow-archi
```
# Install dependencies
# cuda is optional
python ./ts_scripts/install_dependencies.py --cuda=cu102
python ./ts_scripts/install_dependencies.py --cuda=cu121

# Latest release
conda install -c pytorch torchserve torch-model-archiver torch-workflow-archiver
Expand Down
2 changes: 1 addition & 1 deletion benchmarks/auto_benchmark.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@ def install_torchserve(skip_ts_install, hw, ts_version):

# install_dependencies.py
if hw == "gpu":
cmd = "python ts_scripts/install_dependencies.py --environment dev --cuda cu118"
cmd = "python ts_scripts/install_dependencies.py --environment dev --cuda cu121"
elif hw == "neuronx":
cmd = "python ts_scripts/install_dependencies.py --environment dev --neuronx"
else:
Expand Down
2 changes: 1 addition & 1 deletion binaries/conda/build_packages.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ def conda_build(

os.environ["PYTHON"] = "python"

python_versions = ["3.8", "3.9", "3.10"]
python_versions = ["3.8", "3.9", "3.10", "3.11"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are you sure you wanna keep 3.8 around?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should support it as long as PyTorch does. But I think we should update our default version in workflows to 3.9

packages = [os.path.join(conda_build_dir, pkg) for pkg in PACKAGES]

# Generate conda binaries for linux-64
Expand Down
4 changes: 2 additions & 2 deletions docker/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,11 @@ Use `build_image.sh` script to build the docker images. The script builds the `p
|-bi, --baseimage specify base docker image. Example: nvidia/cuda:11.8.0-cudnn8-runtime-ubuntu20.04|
|-bt, --buildtype|Which type of docker image to build. Can be one of : production, dev, ci, codebuild|
|-t, --tag|Tag name for image. If not specified, script uses torchserve default tag names.|
|-cv, --cudaversion| Specify to cuda version to use. Supported values `cu92`, `cu101`, `cu102`, `cu111`, `cu113`, `cu116`, `cu117`, `cu118`. Default `cu117`|
|-cv, --cudaversion| Specify to cuda version to use. Supported values `cu92`, `cu101`, `cu102`, `cu111`, `cu113`, `cu116`, `cu117`, `cu118`. `cu121`, Default `cu121`|
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you please confirm if cu121 is PT2.1 default cuda?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

confirmed

|-ipex, --build-with-ipex| Specify to build with intel_extension_for_pytorch. If not specified, script builds without intel_extension_for_pytorch.|
|-n, --nightly| Specify to build with TorchServe nightly.|
|--codebuild| Set if you need [AWS CodeBuild](https://aws.amazon.com/codebuild/)|
|-py, --pythonversion| Specify the python version to use. Supported values `3.8`, `3.9`, `3.10`. Default `3.9`|
|-py, --pythonversion| Specify the python version to use. Supported values `3.8`, `3.9`, `3.10`, `3.11`. Default `3.9`|


**PRODUCTION ENVIRONMENT IMAGES**
Expand Down
9 changes: 6 additions & 3 deletions docker/build_image.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,10 +82,10 @@ do
;;
-py|--pythonversion)
PYTHON_VERSION="$2"
if [[ $PYTHON_VERSION = 3.8 || $PYTHON_VERSION = 3.9 || $PYTHON_VERSION = 3.10 ]]; then
if [[ $PYTHON_VERSION = 3.8 || $PYTHON_VERSION = 3.9 || $PYTHON_VERSION = 3.10 || $PYTHON_VERSION = 3.11 ]]; then
echo "Valid python version"
else
echo "Valid python versions are 3.8, 3.9 and 3.10"
echo "Valid python versions are 3.8, 3.9 3.10 and 3.11"
exit 1
fi
shift
Expand All @@ -94,7 +94,10 @@ do
# With default ubuntu version 20.04
-cv|--cudaversion)
CUDA_VERSION="$2"
if [ "${CUDA_VERSION}" == "cu118" ];
if [ "${CUDA_VERSION}" == "cu121" ];
then
BASE_IMAGE="nvidia/cuda:12.1.0-base-ubuntu20.04"
elif [ "${CUDA_VERSION}" == "cu118" ];
then
BASE_IMAGE="nvidia/cuda:11.8.0-base-ubuntu20.04"
elif [ "${CUDA_VERSION}" == "cu117" ];
Expand Down
2 changes: 1 addition & 1 deletion docker/build_upload_release.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
# Upload pytorch/torchserve docker binaries
try_and_handle(f"./build_image.sh -t {organization}/torchserve:latest", dry_run)
try_and_handle(
f"./build_image.sh -g -cv cu118 -t {organization}/torchserve:latest-gpu",
f"./build_image.sh -g -cv cu121 -t {organization}/torchserve:latest-gpu",
dry_run,
)
try_and_handle(
Expand Down
2 changes: 1 addition & 1 deletion docker/docker_nightly.py
10000
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@
# Build Nightly images and append the date in the name
try_and_handle(f"./build_image.sh -n -t {organization}/{cpu_version}", dry_run)
try_and_handle(
f"./build_image.sh -g -cv cu118 -t {organization}/{gpu_version}",
f"./build_image.sh -g -cv cu121 -t {organization}/{gpu_version}",
dry_run,
)

Expand Down
4 changes: 2 additions & 2 deletions docs/code_coverage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@
For GPU
- Install dependencies (if not already installed)
```bash
python ts_scripts/install_dependencies.py --environment=dev --cuda=cu102
python ts_scripts/install_dependencies.py --environment=dev --cuda=cu121
```
> Supported cuda versions as cu118, cu117, cu116, cu113, cu111, cu102, cu101, cu92
> Supported cuda versions as cu121, cu118, cu117, cu116, cu113, cu111, cu102, cu101, cu92

- Execute sanity suite
```bash
Expand Down
4 changes: 2 additions & 2 deletions docs/getting_started.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@
python ./ts_scripts/install_dependencies.py
```

- For GPU with Cuda 10.2. Options are `cu92`, `cu101`, `cu102`, `cu111`, `cu113`, `cu116`, `cu117`, `cu118`
- For GPU with Cuda 12.1. Options are `cu92`, `cu101`, `cu102`, `cu111`, `cu113`, `cu116`, `cu117`, `cu118`, `cu121`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this list is long no? :P

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, we should truncate this to the latest version. Good point. Anyone looking for 0.6.0 can look at the readme page corresponding to 0.6.0 tag


```bash
python ./ts_scripts/install_dependencies.py --cuda=cu102
python ./ts_scripts/install_dependencies.py --cuda=cu121
```

Note: PyTorch 1.9+ will not support cu92 and cu101. So TorchServe only supports cu92 and cu101 up to PyTorch 1.8.1.
Expand Down
7 changes: 7 additions & 0 deletions requirements/torch_cu121_linux.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu121
--extra-index-url https://download.pytorch.org/whl/test/cu121
-r torch_common.txt
torch==2.1.0+cu121; sys_platform == 'linux'
torchvision==0.16.0+cu121; sys_platform == 'linux'
torchtext==0.16.0; sys_platform == 'linux'
torchaudio==2.1.0+cu121; sys_platform == 'linux'
7 changes: 7 additions & 0 deletions requirements/torch_cu121_windows.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu121
--extra-index-url https://download.pytorch.org/whl/cu121
-r torch_common.txt
torch==2.1.0+cu121; sys_platform == 'linux'
torchvision==0.16.0+cu121; sys_platform == 'linux'
torchtext==0.16.0; sys_platform == 'linux'
torchaudio==2.1.0+cu121; sys_platform == 'linux'
9 changes: 5 additions & 4 deletions requirements/torch_darwin.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
#pip install torch torchvision torchaudio
--extra-index-url https://download.pytorch.org/whl/test/cpu
-r torch_common.txt
torch==2.0.1; sys_platform == 'darwin'
torchvision==0.15.2; sys_platform == 'darwin'
torchtext==0.15.2; sys_platform == 'darwin'
torchaudio==2.0.2; sys_platform == 'darwin'
torch==2.1.0; sys_platform == 'darwin'
torchvision==0.16.0; sys_platform == 'darwin'
torchtext==0.16.0; sys_platform == 'darwin'
torchaudio==2.1.0; sys_platform == 'darwin'
10 changes: 5 additions & 5 deletions requirements/torch_linux.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cpu
--extra-index-url https://download.pytorch.org/whl/cpu
--extra-index-url https://download.pytorch.org/whl/test/cpu
-r torch_common.txt
torch==2.0.1+cpu; sys_platform == 'linux'
torchvision==0.15.2+cpu; sys_platform == 'linux'
torchtext==0.15.2; sys_platform == 'linux'
torchaudio==2.0.2+cpu; sys_platform == 'linux'
torch==2.1.0+cpu; sys_platform == 'linux'
torchvision==0.16.0+cpu; sys_platform == 'linux'
torchtext==0.16.0; sys_platform == 'linux'
torchaudio==2.1.0+cpu; sys_platform == 'linux'
9 changes: 5 additions & 4 deletions requirements/torch_windows.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
#pip install torch torchvision torchaudio
--extra-index-url https://download.pytorch.org/whl/test/cpu
-r torch_common.txt
torch==2.0.1; sys_platform == 'win32'
torchvision==0.15.2; sys_platform == 'win32'
torchtext==0.15.2; sys_platform == 'win32'
torchaudio==2.0.2; sys_platform == 'win32'
torch==2.1.0; sys_platform == 'win32'
torchvision==0.16.0; sys_platform == 'win32'
torchtext==0.16.0; sys_platform == 'win32'
torchaudio==2.1.0; sys_platform == 'win32'
12 changes: 11 additions & 1 deletion ts_scripts/install_dependencies.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,17 @@ def get_brew_version():
parser.add_argument(
"--cuda",
default=None,
choices=["cu92", "cu101", "cu102", "cu111", "cu113", "cu116", "cu117", "cu118"],
choices=[
"cu92",
"cu101",
"cu102",
"cu111",
"cu113",
"cu116",
"cu117",
"cu118",
"cu121",
],
help="CUDA version for torch",
)
parser.add_argument(
Expand Down
0