Avieshek@lemmy.world to Technology@lemmy.worldEnglish · 3 months agoEdward Snowden slams Nvidia's RTX 50-series 'F-tier value,' whistleblows on lackluster VRAM capacitywww.tomshardware.comexternal-linkmessage-square88fedilinkarrow-up1106arrow-down129
arrow-up177arrow-down1external-linkEdward Snowden slams Nvidia's RTX 50-series 'F-tier value,' whistleblows on lackluster VRAM capacitywww.tomshardware.comAvieshek@lemmy.world to Technology@lemmy.worldEnglish · 3 months agomessage-square88fedilink
minus-squareJeena@piefed.jeena.netlinkfedilinkEnglisharrow-up4·3 months agoExactly, I’m in the same situation now and the 8GB in those cheaper cards don’t even let you run a 13B model. I’m trying to research if I can run a 13B one on a 3060 with 12 GB.
minus-squareThe Hobbyist@lemmy.ziplinkfedilinkEnglisharrow-up4·3 months agoYou can. I’m running a 14B deepseek model on mine. It achieves 28 t/s.
minus-squarelevzzz@lemmy.worldlinkfedilinkEnglisharrow-up1·3 months agoYou need a pretty large context window to fit all the reasoning, ollama forces 2048 by default and more uses more memory
minus-squareJeena@piefed.jeena.netlinkfedilinkEnglisharrow-up1·3 months agoOh nice, that’s faster than I imagined.
Exactly, I’m in the same situation now and the 8GB in those cheaper cards don’t even let you run a 13B model. I’m trying to research if I can run a 13B one on a 3060 with 12 GB.
You can. I’m running a 14B deepseek model on mine. It achieves 28 t/s.
You need a pretty large context window to fit all the reasoning, ollama forces 2048 by default and more uses more memory
Oh nice, that’s faster than I imagined.