☆ Yσɠƚԋσʂ ☆@lemmygrad.ml to Technology@lemmygrad.mlEnglish · 1 month agoDeepSeek-V3 now runs at 20 tokens per second on Mac Studio, and that’s a nightmare for OpenAIventurebeat.comexternal-linkmessage-square15fedilinkarrow-up138arrow-down11cross-posted to: technology@beehaw.org
arrow-up137arrow-down1external-linkDeepSeek-V3 now runs at 20 tokens per second on Mac Studio, and that’s a nightmare for OpenAIventurebeat.com☆ Yσɠƚԋσʂ ☆@lemmygrad.ml to Technology@lemmygrad.mlEnglish · 1 month agomessage-square15fedilinkcross-posted to: technology@beehaw.org
minus-squareCriticalResist8@lemmygrad.mllinkfedilinkarrow-up4·1 month agoBut imagine what you’ll be able to run it on in four more months. But yeah, it’s stretching the definition of consumer hardware a bit.
minus-squarepinguinu [any]@lemmygrad.mllinkfedilinkarrow-up1·1 month agoYou can use the smaller models on (beefy) consumer hardware already. That’s something, right? 😅
minus-squareCriticalResist8@lemmygrad.mllinkfedilinkarrow-up3·1 month agoI want the full 1TB model running on my 10 year old linux laptop
minus-squarepinguinu [any]@lemmygrad.mllinkfedilinkarrow-up2·1 month agoJust put your persistent memory as swap. Easy
But imagine what you’ll be able to run it on in four more months. But yeah, it’s stretching the definition of consumer hardware a bit.
You can use the smaller models on (beefy) consumer hardware already. That’s something, right? 😅
I want the full 1TB model running on my 10 year old linux laptop
Just put your persistent memory as swap. Easy