Claude: papers please?
from SuspciousCarrot78@lemmy.world to privacy@lemmy.ml on 17 Apr 03:25
https://lemmy.world/post/45705254

[deleted by user]

#privacy

threaded - newest

eager_eagle@lemmy.world on 17 Apr 03:38 next collapse

I hope people in the us wake the fuck up and these companies start losing users to foreign ones by the millions

unitedwithme@lemmy.today on 17 Apr 04:04 next collapse

We spread word to migrate to federated and non US tech platforms to start taking their financial leg, then ad revenue drops and slowly they weaken. The weaker they are, the less political influence (aka purchasing power) they have!

SuspciousCarrot78@lemmy.world on 17 Apr 04:34 collapse

[deleted by user]

Sims@lemmy.ml on 17 Apr 04:02 next collapse

Yucky… The identity partner looks nasty. I bet all these corps are linked to the Epstein class and their think tanks. The 1% are taking control of their cattle…

trailee@sh.itjust.works on 17 Apr 04:18 next collapse

Even if the tools are not yet there, “they” want to know exactly who asks for code to things like a DIY radar station or autonomous drone control. We’re well into “first they came” territory.

SuspciousCarrot78@lemmy.world on 17 Apr 04:31 collapse

[deleted by user]

trailee@sh.itjust.works on 17 Apr 06:02 collapse

I hope I’m wrong too. But I’m pretty pessimistic.

utopiah@lemmy.ml on 17 Apr 05:19 next collapse

IMHO LLM usage isn’t coherent with independence. That being said I wrote quite a bit on self-hosting LLMs. There are quite a few tools available, like ollama itself relying on llama.cpp that can both work locally and provide an API compatible replacement to cloud services. As you suggested though typically at home one doesn’t have the hardware, GPUs with 100+GB of VRAM, to run the state of the art. There is a middle ground though between full cloud, API key, closed source vs open source at home on low-end hardware : running STOA open models on cloud. It can be done on any cloud but it’s much easier to start with dedicated hardware and tooling, for that HuggingFace is great but there are multiples.

TL;DR: closed cloud -> models on clouds -> self-hosted provide a better path to independence, including training.

SuspciousCarrot78@lemmy.world on 17 Apr 05:34 collapse

[deleted by user]

steel_for_humans@piefed.social on 17 Apr 05:32 next collapse

Say I have a GPU with 32GB VRAM and I am on Linux, what local LLM would be good for coding?

Currently I just have an iGPU ;) but that’s always an option, albeit a very expensive one.

SuspciousCarrot78@lemmy.world on 17 Apr 05:48 next collapse

[deleted by user]

steel_for_humans@piefed.social on 17 Apr 06:32 collapse

I meant that I can buy one of those Radeons dedicated to AI work, like the ASRock Radeon AI PRO R9700 Creator 32GB GDDR6. If I need to.

Currently my Ryzen iGPU is all I need, because all I need is to see the graphical desktop environment on my screen ;) It does the job well.

I use Claude Code as well and I am slightly concerned with that ID verification news, even more so because of the technology partner that they chose.

SuspciousCarrot78@lemmy.world on 17 Apr 06:54 collapse

[deleted by user]

andrew0@lemmy.dbzer0.com on 17 Apr 06:27 collapse

Get llama.cpp and try Qwen3.6-35B-A3B. Just came out and looks good. You’ll have to look into optimal settings, as it’s a Mixture of Experts (MoE) model with only 3B parameters active. That means that the rest can stay in RAM for quick inference.

You could also try the dense model (Qwen3.5-27B), but that will be significantly slower. Put these in a coding harness like Oh-My-Pi, OpenCode, etc. and see how it fares for your tasks. Should be ok for small tasks, but don’t expect Opus / Sonnet 4.6 quality, more like better than Haiku.

lsjw96kxs@sh.itjust.works on 17 Apr 05:56 next collapse

Personally, I would like to use AI, but I don’t due to it being non local. I know there are local AI that could do things, but I don’t know which models are the good one for each task. If someone can give me pointers for it, I’d be grateful, for exemple a good model for local coding :)

SuspciousCarrot78@lemmy.world on 17 Apr 06:09 next collapse

[deleted by user]

lsjw96kxs@sh.itjust.works on 18 Apr 06:14 collapse

Thanks for the pointers. For the hardware, I have a 9070 XT with 16 Gb of VRAM. It’s sure that it can be very expensive. As I only do this as a hobby, I don’t want to pay that amount of money. I’m okay with having a slow llm as it wouldn’t be a tool I’d use often. I prefer to try doing things on my own and use the ai to help for little tasks first, such as checking why this one line of code didn’t want to work correctly or things like that.

lime@feddit.nu on 17 Apr 06:11 collapse

depends on your hardware and your preferred language. i think wizardcoder is a pretty common choice but the smallest useful version is around 14GB so you need the vram to accommodate it.

lsjw96kxs@sh.itjust.works on 18 Apr 06:08 collapse

Thanks, I’ll dig into this. BTW, I have a 9070 XT, with 16 Gb VRAM, so it should do the job I guess.

lime@feddit.nu on 18 Apr 08:52 collapse

you need space for the context and runtime parameters too, but i think it should work. worst case there are some offloading settings you can do depending on the server you use. only way to knew is to try, really.

ropatrick@lemmy.world on 17 Apr 06:56 next collapse

I love the sound of this but can I ask, if the net goes down and you hardly notice, where do you get your ‘net’ from? Or is it that your intranet doesn’t need internet as such and everything is just local?

I might have answered my own question there but I’m interested to understand it a bit more.

Thanks!

SuspciousCarrot78@lemmy.world on 17 Apr 07:26 next collapse

[deleted by user]

ropatrick@lemmy.world on 17 Apr 07:54 collapse

OK I have you. You dont need the internet because you have the internet in your terabyte farm. Pretty cool.

Thanks for the detailed reply.

One final question, I’m sure its dark at the bottom of the deep rabbit hole you are in, what do you do for batteries for your head torch?! 😀

SuspciousCarrot78@lemmy.world on 17 Apr 08:23 collapse

[deleted by user]

webghost0101@sopuli.xyz on 17 Apr 16:11 collapse

Thats weird, i don’t remember having an alt account called SuspiciousCarrot78 but surely you must be me, same project, same neurology… same fixation pattern.

SuspciousCarrot78@lemmy.world on 17 Apr 17:00 collapse

[deleted by user]

webghost0101@sopuli.xyz on 17 Apr 17:15 collapse

Yep, definitely talking to myself again

Jokes aside, i haven’t seen you mention anything for media streaming.

I highly passionately recommend Navidrome for music. It is my absolute favorite and most used self hosted service.

For acquiring media like film and music depending where you live ripping those from your local library is in some places arguably a protected fair use. (Comes from the time mp3 players became common and runners used to take rented cds in their walkman outside before). In my experience, 480p dvd is much higher quality then internet 480p streams and the total size is much smaller then what you find in downloads.

ARM can help you automatically rip these as long as you have a drive in your pc. I got ARM running in a proxmox lxc with drive passtrough but that honestly was a pain to setup so not sure you should go that exact route, either way the moment arm is functional its smooth sailing and your only concern becomes storage space.

SuspciousCarrot78@lemmy.world on 17 Apr 18:13 collapse

[deleted by user]

webghost0101@sopuli.xyz on 17 Apr 19:54 collapse

Currently my setup for YouTube In jellyfin is pretty basic (but it works well)

Its one complex yt-dlp script that runs ecery hour, reads a list of channels from a txt file. Rss to check the last 2 vids per channel, if there not downloaded already (log with vid ids per channel) it starts to download on high quality preset, filter out shorts and “live”, if the result is getting above a certain size it aborts and tries again with lower quality preset.

I don’t have a system to auto dele yet cause the script is only for my favorites. For everything else or that uploads to regularly i host a local invidious.

But google really hates invidious and it breaks frequently, even got my original residential ip banned. I may take a look at tube archive cause i would prefer download and delete as a more stable flow.

Something else i am experimenting with is using the same data in multiple systems. My music collection is a proxmox dataset but as long as the read rights are good i see no reason why jellyfin can’t also read those files.

The output folder of ARM used to be its own library in jellyfin on my first server.

SuspciousCarrot78@lemmy.world on 17 Apr 16:43 collapse

[deleted by user]

maegul@lemmy.ml on 17 Apr 07:11 next collapse

Probably a shallow response …

But I always figured AI/LLMs are basically apocalyptic for all sorts of individualistic values in computing (including privacy but also independence and diversity).

Whether they’re good or useful etc, I just struggle to see how they will ever be justifiable against these sorts of values.

Sure, local models and our hardware will get better … but better than the state of the art from the big labs and providers? Given that data and training are the big bottlenecks on quality … I struggle to see how AI isn’t a complete feudal capturing of information computing and processing. Not to mention what happens to the pipeline that produces information content if everyone is only consuming it through the models that train on it.

So for me the big question is, what’s our call on a possible (likely even?) future where we are forever stuck using cloud provided AI along with all of its negatives, in the same way that basically all of us has been and still is stuck using MS windows, Google and the big-social-media hellscape?

For me, I baulk at this.

SuspciousCarrot78@lemmy.world on 17 Apr 08:18 next collapse

[deleted by user]

maegul@lemmy.ml on 17 Apr 08:54 collapse

Oh I hear you (and appreciate the response).

For me, I can’t help but think of another alternative, which I’m surprised I haven’t heard of yet …

stripping down one’s personal technological cognitive load to a stack of systems that can fit into one’s brain (like the Python mantra), focusing on learning that stack well building sustainable and stable systems, and then just detoxing from the increasingly polluted digital information stream (protected commons, traditional formats such as books and in person engagement … dunno).

Depends on what the end goal is, but AIs seem to be about using tech more or just opting out of sovereignty. Something like the above seems to me to be about using tech less (in the end) and pushing toward being a secondary tool rather than an end of its own.

SuspciousCarrot78@lemmy.world on 17 Apr 09:00 collapse

[deleted by user]

maegul@lemmy.ml on 17 Apr 09:20 collapse

Ha yes … on the other hand, it was easy to forget how good damn expansive non-internet information is: the whole world ran on that shit for millennia.

[deleted] on 17 Apr 18:53 collapse

.

pound_heap@lemmy.dbzer0.com on 17 Apr 16:55 collapse

Being responsible with powerful technology starts with knowing who is using it.

The fuck it does. Claude is already censored - you can’t get a recipe of a poison, schematics for a bomb, an advice on how to hide a body. If you can, then it’s Anthropic engineers didn’t do their job.

Knowing who is using it helps either with conditional censorship, or helping governments to track people based on their prompts, or just plainly lying and using data for analytics and training. All these easons are shit.

And don’t tell me this is to protect the kids again. Let the parents do their job.