~ / tag / #ollama $ grep

#Ollama

1 post

Does Muse Glimmer 30B Really Fit on an RTX 3090? Meta Says One Thing, the People Who Tested It Say Another

The press says Muse Glimmer 30B requires a 5090. r/LocalLLaMA is posting screenshots of it running on a used 3090 from 2020. Both are right, and the explanation is in the VRAM budget: 17 GB of weights, 1.7 GB of KV cache, and an attention architecture designed to fit. Here's the math line by line, the tokens-per-second estimate on a 3090 with the work shown, and the verdict on when 24 GB is enough and when it isn't.

11 Aug · 12 min ›