Some local LLMs tested on an average gaming PC (github.com)
from donatas@feddit.uk to selfhosted@lemmy.world on 07 Apr 10:08
https://feddit.uk/post/27047749

#selfhosted

threaded - newest

slazer2au@lemmy.world on 07 Apr 10:41 next collapse

Average gaming PC.

128GB ram and a 4070 12GB. Doesn’t sound average.

Duke_Nukem_1990@feddit.org on 07 Apr 10:42 next collapse

Yeah, not even a 5090? Pf, nice Dwarf Fortress machine I guess.

superkret@feddit.org on 07 Apr 10:45 collapse

A dwarf Fortress machine would have an i9-14900KS, 192GB DDR5 RAM and no graphics card.

Nibodhika@lemmy.world on 07 Apr 18:31 collapse

You still need a GPU, otherwise your CPU is wasting precious operations drawing the screen instead of simulating dwarves.

archonet@lemy.lol on 07 Apr 11:05 next collapse

I was gonna say, 4x the amount of RAM I have, made me feel inadequate. like, “am I really that poor now? is my 2080 now considered potato gaming?”

it’s simply been so long since I’ve gone “ok I need to upgrade” that I’m starting to get suspicious

surewhynotlem@lemmy.world on 07 Apr 11:55 next collapse

is my 2080 now considered potato gaming?"

I’m going to keep using this 1070 until it fails it I get bored of Skyrim.

GnuLinuxDude@lemmy.ml on 07 Apr 16:33 collapse

At this rate it seems like I will also never be upgrading from the 1070. Miners, NFTs, supply chain, AI, tariffs… how is an honest guy supposed to just buy a decent GPU when squeezed between all that bs?

lka1988@lemmy.dbzer0.com on 07 Apr 23:27 collapse

I got my 7900XTX for MSRP in 2023 and have no plans on “upgrading” any time soon. The same PC’s 5800X3D is also showing no signs of letting up, and given the tariff bullshit going on right now, I plan on keeping it that way for as long as possible.

donatas@feddit.uk on 07 Apr 12:10 collapse

128GB won’t make any difference in gaming when compared to, say, 32GB, so it’s just to fit larger LLMs into memory. However, 0.5t/s is hardly usable for every day tasks, so 32GB will still be more realistic combination with the RTX4070.

Keelhaul@sh.itjust.works on 07 Apr 18:06 next collapse

So it is not running llms on an average gaming pc

Snazz@lemmy.world on 07 Apr 18:29 collapse

Nuh-uh! You can configure Minecraft to use all that memory to run buttery smooth. Just make sure to not play longer than 15 minutes so you don’t get the mother-of-god lagspike when the garbage collector finally runs.

iAmTheTot@sh.itjust.works on 07 Apr 11:53 next collapse

No kidding. Steam survey definitely does not agree with this definition of average lmao

False@lemmy.world on 07 Apr 14:57 collapse

I thought I had a lot of RAM with 64

Valmond@lemmy.world on 07 Apr 16:54 next collapse

Well, you do.

sus@programming.dev on 07 Apr 17:28 collapse

64 gigs of ram costs less than $200 these days

Blue_Morpho@lemmy.world on 07 Apr 12:00 collapse

I didn’t see tokens/s anywhere?

donatas@feddit.uk on 07 Apr 12:06 collapse

Performance/vision based tests have tokens/s at the end of tables.