Modified NVIDIA GPUs
-
Some gpu "labs" are modifying NVIDIA GPUs to get way more VRAM on them.
Probably still one of the best options for the money, but fairly risky.
NOTE: AFAIK the 5090 96GB are FAKE, do NOT buy them. Beware of scammers!You can find these GPUs easily on eBay or Alibaba for solid pricing, the ones I know of are:
- RTX 2080ti 22GB
- RTX 3080 20GB
- RTX 4080 32GB
- RTX 4090 48GB
I myself own 2x RTX 3080 20GB and am very impressed with the results, they are real, and do exist, but don't have working ReBAR support (although a fix for this is discussed here, this majorly affects tensor parallel setups (multi-gpu), I highly recommend this tweak).
The performance I get seems right, about 80tk/s under heavy context load on Qwen 3.8 27B using this quant with 256k ctx and vision enabled (with MTP).
No AI used
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login