Wan视频生成模型部署
Wan是一个开源的AI视频生成大模型,支持T2V(文本生成视频)和I2V(图像生成视频)两种模式。在ComfyUI中,百舸提供了定制化的JSON工作流和API调用方式,帮助您使用Wan模型生成高质量的视频内容。本文将以I2V(图像生成视频)为例,介绍如何通过ComfyUI部署Wan模型,并使用Wan生成视频。
限制条件
仅支持在全托管资源池,单机、单卡部署单实例服务。
部署Wan模型
- 登录百舸AI计算平台,单击 快速开始;
- 在左侧内容筛选区域的模型系列中,勾选 通义万相,搜索 Wan2.1-I2V-14B 模型;
- 单击 部署 按钮,填写资源规格和流量接入方式;
- 服务部署成功后可以在 在线服务部署 中查看服务的 API 调用地址和 WebUI 访问地址。
WebUI使用
服务部署成功后,在服务列表的 调用信息 中获取 WebUI地址,在浏览器中打开。
WebUI服务鉴权
首次登录WebUI,需要设置账户密码,本地会生成一个PASSWORD的文件用来保护密码信息。
设置密码后可以在百舸控制台此服务的输出日志中查看Token信息。
取消鉴权
如果不需要服务鉴权,可以更新服务,在启动命令中增加此参数rm -r /root/workspace/ComfyUI/custom_nodes/ComfyUI-Login
不支持鉴权的启动命令示例如下:
1cd /root/workspace/ComfyUI/models/ && rm-r diffusion_models text_encoders vae
2ln -s /mnt/aihc_model_dir/diffusion_models /root/workspace/ComfyUI/models/diffusion_models
3ln -s /mnt/aihc_model_dir/text_encoders /root/workspace/ComfyUI/models/text_encoders
4ln -s /mnt/aihc_model_dir/vae /root/workspace/ComfyUI/models/vae
5rm -r /root/workspace/ComfyUI/custom_nodes/ComfyUI-Login
6cd /root/workspace/ComfyUI/ && python3 main.py --listen 0.0.0.0 --port 8000
输入参数
LoadImageNode (输入图片)
方式一:从持久化存储中选择图片
平台默认从服务的/root/workspace/ComfyUI/input路径加载图片,您可以将输入的图片存储在持久化存储后挂载到此路径,在ComfyUI中选择该路径下任意一张图片。
方式二:填写URL
可以直接填写图片URL,例如:
方式三:上传本地图片
点击选择要上传的文件会拉起本地资源管理器,上传本地图片即可
PromptNode(填写Prompt)
生成视频
生成的视频默认保存在服务的容器/root/workspace/ComfyUI/output路径中,可以更新服务配置挂载具备写权限的PFS或BOS存储到该路径,生成的视频可以持久化保存。
API调用
查看访问地址
服务部署成功后,可以在 在线服务部署 的服务列表点击服务的 调用地址 获取公网或VPC内网访问地址和Token信息。
获取ComfyUI工作流 API
ComfyUI的API请求体取决于工作流配置。平台会提供基于当前模型的默认工作流导出的API JSON,可以从WebUI中导出API,也可以直接使用文档中的API示例。
T2V
替换请求示例中body里prompt中的内容
1{
2 "1": {
3 "inputs": {
4 "model_name": "wan2.1_t2v_14B_fp16.safetensors",
5 "weight_dtype": "default"
6 },
7 "class_type": "ModelLoaderNode",
8 "_meta": {
9 "title": "ModelLoaderNode"
10 }
11 },
12 "3": {
13 "inputs": {
14 "vae_name": "Wan2.1_VAE.pth"
15 },
16 "class_type": "VAELoaderNode",
17 "_meta": {
18 "title": "VAELoaderNode"
19 }
20 },
21 "4": {
22 "inputs": {
23 "negative_prompt": "low quality, blurry, ugly, poorly drawn hands, deformed face, extra limbs, bad anatomy, low resolution, disfigured, unrealistic, cartoonish, watermark, text, signature, distorted proportions, creepy, glitch, jpeg artifacts"
24 },
25 "class_type": "NegativePromptNode",
26 "_meta": {
27 "title": "NegativePromptNode"
28 }
29 },
30 "5": {
31 "inputs": {
32 "positive_prompt": "A graceful ancient Chinese beauty in flowing white hanfu stands on a moss-covered stone altar within a ruined forest temple under a starlit sky. Glowing magical symbols and celestial energy swirl around her as she gracefully casts a spell, her long black hair cascading like silk, adorned with delicate jade hairpins. She holds an ornate fan with glowing patterns, ethereal light illuminating her serene face. Cinematic lighting and volumetric fog enhance the mystical atmosphere, with her embroidered robe billowing elegantly in the wind. Ultra-detailed, 4K, highly realistic, in the style of greg rutkowski, artgerm, cinematic fantasy. Animation of magical energy swirling, slow motion aura forming, glowing runes pulsing, hanfu sleeves fluttering with every movement."
33 },
34 "class_type": "PositivePromptNode",
35 "_meta": {
36 "title": "PositivePromptNode"
37 }
38 },
39 "6": {
40 "inputs": {
41 "prompt": [
42 "5",
43 0
44 ],
45 "negative_prompt": [
46 "4",
47 0
48 ],
49 "model": [
50 "1",
51 0
52 ],
53 "text_encoder": [
54 "8",
55 0
56 ],
57 "vae": [
58 "3",
59 0
60 ],
61 "num_inference_steps": 30,
62 "seed": 1428,
63 "tiled": true,
64 "fps": 25,
65 "num_frames": 81
66 },
67 "class_type": "ExVideoNode",
68 "_meta": {
69 "title": "ExVideoNode"
70 }
71 },
72 "7": {
73 "inputs": {
74 "video-preview": "",
75 "video": [
76 "6",
77 0
78 ]
79 },
80 "class_type": "PreViewVideo",
81 "_meta": {
82 "title": "PreViewVideo"
83 }
84 },
85 "8": {
86 "inputs": {
87 "text_encoder_name": "models_t5_umt5-xxl-enc-bf16.pth",
88 "type": "stable_diffusion",
89 "device": "default"
90 },
91 "class_type": "TextEncoderLoaderNode",
92 "_meta": {
93 "title": "TextEncoderLoaderNode"
94 }
95 }
96}
I2V
替换请求示例中body里prompt中的内容
1{
2 "1": {
3 "inputs": {
4 "model_name": "wan2.1_i2v_480p_14B_fp16.safetensors",
5 "weight_dtype": "default"
6 },
7 "class_type": "ModelLoaderNode",
8 "_meta": {
9 "title": "ModelLoaderNode"
10 }
11 },
12 "2": {
13 "inputs": {
14 "clip_name": "models_clip_open-clip-xlm-roberta-large-vit-huge-14.pth",
15 "type": "stable_diffusion",
16 "device": "default"
17 },
18 "class_type": "CLIPLoaderNode",
19 "_meta": {
20 "title": "CLIPLoaderNode"
21 }
22 },
23 "3": {
24 "inputs": {
25 "text_encoder_name": "models_t5_umt5-xxl-enc-bf16.pth",
26 "type": "stable_diffusion",
27 "device": "default"
28 },
29 "class_type": "TextEncoderLoaderNode",
30 "_meta": {
31 "title": "TextEncoderLoaderNode"
32 }
33 },
34 "4": {
35 "inputs": {
36 "vae_name": "Wan2.1_VAE.pth"
37 },
38 "class_type": "VAELoaderNode",
39 "_meta": {
40 "title": "VAELoaderNode"
41 }
42 },
43 "5": {
44 "inputs": {
45 "image": "example.png",
46 "url": ""
47 },
48 "class_type": "LoadImageNode",
49 "_meta": {
50 "title": "LoadImageNode"
51 }
52 },
53 "6": {
54 "inputs": {
55 "prompt": [
56 "7",
57 0
58 ],
59 "negative_prompt": [
60 "8",
61 0
62 ],
63 "model": [
64 "1",
65 0
66 ],
67 "text_encoder": [
68 "3",
69 0
70 ],
71 "clip": [
72 "2",
73 0
74 ],
75 "vae": [
76 "4",
77 0
78 ],
79 "num_inference_steps": 30,
80 "seed": 430,
81 "tiled": true,
82 "fps": 25,
83 "num_frames": 81,
84 "image": [
85 "5",
86 0
87 ]
88 },
89 "class_type": "ExImageToVideoNode",
90 "_meta": {
91 "title": "ExImageToVideoNode"
92 }
93 },
94 "7": {
95 "inputs": {
96 "positive_prompt": "The Little Prince, fox, simple poses, gentle movement, Van Gogh style, starry sky, glowing stars, soft focus, dreamy bokeh, fluffy felt textures, vibrant colors, magical lighting, warm glow, minimalistic action, subtle animation, flowing yellow scarf, peaceful atmosphere, glowing lights, 3D felt figurines, enchanting fantasy, serene, soft swirling vortex"
97 },
98 "class_type": "PositivePromptNode",
99 "_meta": {
100 "title": "PositivePromptNode"
101 }
102 },
103 "8": {
104 "inputs": {
105 "negative_prompt": "Fast movement, chaotic, complex actions, modern, technology, industrial, harsh lighting, dark, dystopian, unrefined, unnatural, flat, lifeless, no star effect"
106 },
107 "class_type": "NegativePromptNode",
108 "_meta": {
109 "title": "NegativePromptNode"
110 }
111 },
112 "9": {
113 "inputs": {
114 "video-preview": "",
115 "video": [
116 "6",
117 0
118 ]
119 },
120 "class_type": "PreViewVideo",
121 "_meta": {
122 "title": "PreViewVideo"
123 }
124 }
125}
发送推理请求
参考发送示例,在示例中输入访问地址、Token、Prompt 发送POST请求启动工作流,会返回此工作流执行的prompt_id。
1curl --location --request POST '<访问地址>/prompt' \
2--header 'Authorization: Bearer <TOKEN>' \
3--header 'Content-Type: application/json' \
4--data-raw '{
5 "prompt":{
6 "1": {
7 "inputs": {
8 "model_name": "wan2.1_t2v_14B_fp16.safetensors",
9 "weight_dtype": "default"
10 },
11 "class_type": "ModelLoaderNode",
12 "_meta": {
13 "title": "ModelLoaderNode"
14 }
15 },
16 "3": {
17 "inputs": {
18 "vae_name": "Wan2.1_VAE.pth"
19 },
20 "class_type": "VAELoaderNode",
21 "_meta": {
22 "title": "VAELoaderNode"
23 }
24 },
25 "4": {
26 "inputs": {
27 "negative_prompt": "low quality, blurry, ugly, poorly drawn hands, deformed face, extra limbs, bad anatomy, low resolution, disfigured, unrealistic, cartoonish, watermark, text, signature, distorted proportions, creepy, glitch, jpeg artifacts"
28 },
29 "class_type": "NegativePromptNode",
30 "_meta": {
31 "title": "NegativePromptNode"
32 }
33 },
34 "5": {
35 "inputs": {
36 "positive_prompt": "A graceful ancient Chinese beauty in flowing white hanfu stands on a moss-covered stone altar within a ruined forest temple under a starlit sky. Glowing magical symbols and celestial energy swirl around her as she gracefully casts a spell, her long black hair cascading like silk, adorned with delicate jade hairpins. She holds an ornate fan with glowing patterns, ethereal light illuminating her serene face. Cinematic lighting and volumetric fog enhance the mystical atmosphere, with her embroidered robe billowing elegantly in the wind. Ultra-detailed, 4K, highly realistic, in the style of greg rutkowski, artgerm, cinematic fantasy. Animation of magical energy swirling, slow motion aura forming, glowing runes pulsing, hanfu sleeves fluttering with every movement."
37 },
38 "class_type": "PositivePromptNode",
39 "_meta": {
40 "title": "PositivePromptNode"
41 }
42 },
43 "6": {
44 "inputs": {
45 "prompt": [
46 "5",
47 0
48 ],
49 "negative_prompt": [
50 "4",
51 0
52 ],
53 "model": [
54 "1",
55 0
56 ],
57 "text_encoder": [
58 "8",
59 0
60 ],
61 "vae": [
62 "3",
63 0
64 ],
65 "num_inference_steps": 30,
66 "seed": 1428,
67 "tiled": true,
68 "fps": 25,
69 "num_frames": 81
70 },
71 "class_type": "ExVideoNode",
72 "_meta": {
73 "title": "ExVideoNode"
74 }
75 },
76 "7": {
77 "inputs": {
78 "video-preview": "",
79 "video": [
80 "6",
81 0
82 ]
83 },
84 "class_type": "PreViewVideo",
85 "_meta": {
86 "title": "PreViewVideo"
87 }
88 },
89 "8": {
90 "inputs": {
91 "text_encoder_name": "models_t5_umt5-xxl-enc-bf16.pth",
92 "type": "stable_diffusion",
93 "device": "default"
94 },
95 "class_type": "TextEncoderLoaderNode",
96 "_meta": {
97 "title": "TextEncoderLoaderNode"
98 }
99 }
100 }
101}'
发起推理后可以通过服务日志查看进度。
查询推理结果
推理结束会返回状态、生成视频等信息。结果为空是推理还未结束,请耐心等待。
1curl --location --request GET '<ENDPOINT>/history/<prompt_id>' \ #
2 --header 'Authorization: <TOKEN>'
ComfyUI各指定目录说明
| 路径 | 说明 |
|---|---|
| /ComfyUI | ComfyUI 项目的根目录。 |
| /ComfyUI/output | ComfyUI 生成视频保存目录。 |
| /ComfyUI/input | ComfyUI 输入图片的配置路径 |
| /ComfyUI/custom_nodes | ComfyUI 自定义节点目录。 |
| /ComfyUI/user/default/workflows | ComfyUI 工作流保存目录。 |
| /ComfyUI/models/checkpoints | ComfyUI checkpoints 路径。 |
| /ComfyUI/models/diffusion_models | ComfyUI 扩散模型配置路径 |
| /ComfyUI/models/lora | ComfyUI LoRA 相关插件配置路径。 |
| /ComfyUI/models/vae | ComfyUI VAE 模型配置路径。 |
| /ComfyUI/models/clip | ComfyUI CLIP 模型配置路径。 |
