torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 26.00 MiB. GPU 0 has a total capacty of 4.00 GiB of which 0 bytes is free. Of the allocated memory 3.35 GiB is allocated by PyTorch, and 144.71 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
Model load has failed. Doesn't exist.
GPUのメモリが足りない・・・? 確認してみる
c:\Dev\StreamDiffusion>nvidia-smi
Wed Jan 17 17:13:45 2024
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 522.06 Driver Version: 522.06 CUDA Version: 11.8 |
|-------------------------------+----------------------+----------------------+
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... WDDM | 00000000:01:00.0 Off | N/A |
| N/A 58C P8 9W / N/A | 114MiB / 4096MiB | 22% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 5588 C+G ...ser\Application\brave.exe N/A |
| 0 N/A N/A 7000 C+G ...n64\EpicGamesLauncher.exe N/A |
+-----------------------------------------------------------------------------+
今はありそうな気もするけど実行時に足りなくなってしまったということなのかな。
そして結局
Found cached model: engines\KBlueLeaf/kohaku-v2.1--lcm_lora-True--tiny_vae-True--max_batch-2--min_batch-2--mode-img2img\unet.engine.onnx
Generating optimizing model: engines\KBlueLeaf/kohaku-v2.1--lcm_lora-True--tiny_vae-True--max_batch-2--min_batch-2--mode-img2img\unet.engine.opt.onnx
[W] Model does not contain ONNX domain opset information! Using default opset.
UNet: original .. 0 nodes, 0 tensors, 0 inputs, 0 outputs
UNet: cleanup .. 0 nodes, 0 tensors, 0 inputs, 0 outputs
[I] Folding Constants | Pass 1
[W] Model does not contain ONNX domain opset information! Using default opset.
Only support models of onnx opset 7 and above.
Traceback (most recent call last):
File "C:\Dev\StreamDiffusion\examples\screen\..\..\utils\wrapper.py", line 546, in _load_model
compile_unet(
File "C:\Dev\StreamDiffusion\.venv\lib\site-packages\streamdiffusion\acceleration\tensorrt\__init__.py", line 76, in compile_unet
builder.build(
File "C:\Dev\StreamDiffusion\.venv\lib\site-packages\streamdiffusion\acceleration\tensorrt\builder.py", line 70, in build
optimize_onnx(
File "C:\Dev\StreamDiffusion\.venv\lib\site-packages\streamdiffusion\acceleration\tensorrt\utilities.py", line 437, in optimize_onnx
onnx_opt_graph = model_data.optimize(onnx.load(onnx_path))
File "C:\Dev\StreamDiffusion\.venv\lib\site-packages\streamdiffusion\acceleration\tensorrt\models.py", line 118, in optimize
opt.fold_constants()
File "C:\Dev\StreamDiffusion\.venv\lib\site-packages\streamdiffusion\acceleration\tensorrt\models.py", line 49, in fold_constants
onnx_graph = fold_constants(gs.export_onnx(self.graph), allow_onnxruntime_shape_inference=True)
File "<string>", line 3, in fold_constants
File "C:\Dev\StreamDiffusion\.venv\lib\site-packages\polygraphy\backend\base\loader.py", line 40, in __call__
return self.call_impl(*args, **kwargs)
File "C:\Dev\StreamDiffusion\.venv\lib\site-packages\polygraphy\util\util.py", line 694, in wrapped
return func(*args, **kwargs)
File "C:\Dev\StreamDiffusion\.venv\lib\site-packages\polygraphy\backend\onnx\loader.py", line 424, in call_impl
postfold_num_nodes = onnx_util.get_num_nodes(model)
File "C:\Dev\StreamDiffusion\.venv\lib\site-packages\polygraphy\backend\onnx\util.py", line 41, in get_num_nodes
return _get_num_graph_nodes(model.graph)
AttributeError: 'NoneType' object has no attribute 'graph'
Acceleration has failed. Falling back to normal mode.
>pip3 install torch==2.1.0 torchvision==0.16.0 xformers --index-url https://download.pytorch.org/whl/cu118
ERROR: Could not find a version that satisfies the requirement torch (from versions: none)
ERROR: No matching distribution found for torch