19GB memory usage and 24 sec runtime on RTX 3090
#11
by PartyParrot - opened
I was wondering whether I can run this on an RTX 3090 with 24GB VRAM. The answer is: Yes!
nvidia-smi reports 19356MB peak VRAM usage for image generation. Prompt processing is about 16348MB. PyTorch reports torch.cuda.memory_stats()["active_bytes.all.peak"] of 18987018752.
Time is 35 seconds on first run for pipe and 24 seconds for subsequent runs.
Thanks, I was just wondering about this! It's strange that the model card says it required ~29GB VRAM. I was skeptical because I saw the unsloth gguf quantization at full precision was only 18.2GB. Thanks for confirming.