Drafters are important
-
So, drafters pretty much give (almost) free token generation output, in exchange for a little bit of VRAM (and some very minor prompt processing speed).
There's 3 'popular' architectures for drafters ATM:
- MTP, the most used by far, usually comes with the weights if the model supports it, lowest VRAM usage.
- DSpark, deepseek's current format, performs better but uses more VRAM.
- DFlash2, by far the best option, outstanding performance.
Rough baseline numbers:
No AI used
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login