Zhao Dongyu's Blog

A life which is unexamined is not worth living.

0%

P/D 分离

Posted on In
Symbols count in article: 3k Reading time ≈ 3 mins.

记录 Dynamo 学习实践 P/D 分离。

1
git clone https://github.com/ai-dynamo/dynamo.git

根据这个issue可知，需要 Ubuntu24.04。

Ubuntu 22.04 currently is not supported as it has glibc 2.35 which is below minimum required glibc 2.39

否则会报错：

1
2
3
4
5
6
7
# pip install ai-dynamo[all]
Collecting ai-dynamo[all]
  Using cached ai_dynamo-0.1.0-py3-none-any.whl.metadata (5.7 kB)
INFO: pip is looking at multiple versions of ai-dynamo[all] to determine which version is compatible with other requirements. This could take a while.
ERROR: Ignored the following yanked versions: 0.0.0.dev0
ERROR: Could not find a version that satisfies the requirement ai-dynamo-runtime==0.1.0 (from ai-dynamo[all]) (from versions: none)
ERROR: No matching distribution found for ai-dynamo-runtime==0.1.0

CUDA 基础镜像为：

1
2
3
4
5
6
7
ARG BASE_IMAGE="nvcr.io/nvidia/cuda-dl-base"
ARG BASE_IMAGE_TAG="25.01-cuda12.8-devel-ubuntu24.04"
ARG RELEASE_BUILD
ARG RUNTIME_IMAGE="nvcr.io/nvidia/cuda"
ARG RUNTIME_IMAGE_TAG="12.8.1-runtime-ubuntu24.04"
ARG MANYLINUX_IMAGE="quay.io/pypa/manylinux_2_28_x86_64"
ARG GENAI_PERF_TAG="25d0188713adc47868d6b3f22426375237a90529"

打包镜像： 

1
2
3
4
5
6
7
8
https://catalog.ngc.nvidia.com/orgs/nvidia/containers/cuda-dl-base

$ docker pull nvcr.io/nvidia/cuda-dl-base:25.01-cuda12.8-devel-ubuntu24.04
$ docker images
REPOSITORY                    TAG                                IMAGE ID       CREATED         SIZE
nvcr.io/nvidia/cuda-dl-base   25.01-cuda12.8-devel-ubuntu24.04   583d3d06ed9f   2 months ago    11.9GB
$ docker save -o 25.01-cuda12.8-devel-ubuntu24.04.tar nvcr.io/nvidia/cuda-dl-base
/data/team/zhaodongyu/25.01-cuda12.8-devel-ubuntu24.04.tar

安装python3.10的必要性： 

1
2
ERROR: Ignored the following versions that require a different python version: 1.21.2 Requires-Python >=3.7,<3.11; 1.21.3 Requires-Python >=3.7,<3.11; 1.21.4 Requires-Python >=3.7,<3.11; 1.21.5 Requires-Python >=3.7,<3.11; 1.21.6 Requires-Python >=3.7,<3.11
ERROR: Could not find a version that satisfies the requirement pydantic>=2.9 (from ai-dynamo-vllm) (from versions: none)

1
2
3
4
5
6
wget https://www.python.org/ftp/python/3.10.13/Python-3.10.13.tgz
tar -xzf Python-3.10.13.tgz
cd Python-3.10.13
./configure --enable-optimizations --prefix=$HOME/python310
make -j$(nproc)
make install

python 路径： /root/python310/bin/python3

由于无法访问nvidia官网，在 https://pypi.nvidia.com/ 手动下载 ai_dynamo_vllm-0.7.2-cp38-abi3-manylinux1_x86_64.whl

1
2
3
4
5
6
7
8
9
10
11
12
13
apt-get update

DEBIAN_FRONTEND=noninteractive apt-get install -yq python3-dev python3-pip python3-venv libucx0

/root/python310/bin/python3 -m venv venv310

source venv310/bin/activate

pip install -i https://mirrors.aliyun.com/pypi/simple git/ai_dynamo_vllm-0.7.2-cp38-abi3-manylinux1_x86_64.whl

pip install -i https://mirrors.aliyun.com/pypi/simple ai-dynamo[all]

pip install -i https://mirrors.aliyun.com/pypi/simple vllm

Dynamo CLI has the following 4 sub-commands.

  • 🏃 dynamo run: quickly spin up a server to experiment with a specified model, input and output target.
  • 🫴 dynamo serve: compose a graph of workers locally and serve.
  • 🔨 (Experimental) dynamo build: containerize either the entire graph or parts of graph to multiple containers
  • 🚀 (Experimental) dynamo deploy: deploy to K8 with helm charts or custom operators
  • ☁️ (Experimental) dynamo cloud: interact with your dynamo cloud server
Thanks for your support.