I've Got 'Night Of The Living Dead' On My Homelab Server
from irmadlad@lemmy.world to selfhosted@lemmy.world on 23 Apr 13:00
https://lemmy.world/post/45971651

=> There are 90 zombie processes.

On one of my Homelab servers running Ubuntu Jammy, I always seem to get zombie processes. A quick check with ps -eo pid,ppid,stat,cmd | grep -w Z shows them all <defunct>. It just bugs me. I shut down the server in the most nicest of ways I know how with sudo shutdown -h now but I always get zombie processes shown on start up.

Am I missing something? Do these show up on your servers? How do you deal with them besides just ignoring them if they are <defunct>?

#selfhosted

threaded - newest

frongt@lemmy.zip on 23 Apr 13:27 next collapse

I’ve never run into a situation where it’s a problem. But I just checked one of my Ubuntu servers and don’t have any. What were the processes doing? What do they belong to?

irmadlad@lemmy.world on 23 Apr 13:36 collapse

@BakedCatboy@lemmy.ml @frongt@lemmy.zip Most of them are from [curl] but one is from [health.sh]. I am assuming one of the docker containers uses curl in it’s processes as I have not initiated any curl commands.

frongt@lemmy.zip on 23 Apr 13:47 next collapse

What is their parent process?

irmadlad@lemmy.world on 23 Apr 13:51 collapse

Lots of these: Zombie PID: 230650 | Parent PID: 7791 | Parent Name: spawn-unnamed

Two of those: Zombie PID: 61072 | Parent PID: 7791 | Parent Name: bash

Lots of these: Zombie PID: 56798 | Parent PID: 7791 | Parent Name: health.sh

Lots of these: Zombie PID: 16761 | Parent PID: 7646 | Parent Name: curl

…and a box of naked lady tee’s

surewhynotlem@lemmy.world on 23 Apr 14:01 collapse

What is 7791?

irmadlad@lemmy.world on 23 Apr 14:33 collapse

7791 7768 201 /usr/sbin/netdata -u netdat 8.8 0.5 01:49:50

frongt@lemmy.zip on 23 Apr 14:52 collapse

Sounds like netdata doing health checks but not always reaping its children. If you can reproduce it, I’d file a bug report.

NotEasyBeingGreen@slrpnk.net on 23 Apr 15:02 collapse

Seems like someone already did this:

github.com/netdata/netdata/issues/20565

Maybe upgrading will fix it?

irmadlad@lemmy.world on 23 Apr 15:08 collapse

Hey bro, thanks for the lead! I will read the issue report and check if netdata is current.

@ilyam8 - adding to it here, netdata does leave tons of zombies around, this is not a “cannot reproduce” (not sure why that tag was added without even the minimal response, and then removed needs triage on top of that, so this ticket just dies off) One simply needs a running netdata instance and let it run a while. With time, 1, three and then tens of zombies will be listed at login.

someguy@lemmyland.com on 23 Apr 17:47 collapse

If you use containers with health checks (including with curl), you need to tell docker (or podman) to provide an init process to reap child processes. For docker that means providing –init when running a container. It’s a pretty common problem.

BakedCatboy@lemmy.ml on 23 Apr 13:32 next collapse

I got a few on my laptop but none on either of my long running homelab boxes (70-80 days uptime). On my laptop they all seem related to espeak, the tts program. Is there any pattern in what processes yours are from?

irmadlad@lemmy.world on 23 Apr 13:56 collapse

my long running homelab boxes (70-80 days uptime)

Ahh see I shut down my servers at night. I just couldn’t justify having them run while I was sleeping, and since I am the only user.

BakedCatboy@lemmy.ml on 23 Apr 16:05 collapse

Makes sense. I share my media library with 10-15 friends so there’s usually a few streams late at night, and scrubs, container updates, and backups run early morning at like 2-4am.

tychosmoose@piefed.social on 23 Apr 13:33 next collapse

Zero zombies here. I have a couple of Debian servers and one repeatedly upgraded Ubuntu on noble numbat that I’m too lazy to migrate to Debian. None have zombies.

Do you run a DE? Mine are headless.

irmadlad@lemmy.world on 23 Apr 13:57 collapse

Do you run a DE?

I’m not familiar with the acronym. I’m going to assume Desktop Environment. My servers are all headless as well

tychosmoose@piefed.social on 23 Apr 17:08 collapse

Sorry. That is what I meant.

irmadlad@lemmy.world on 23 Apr 17:34 collapse

s’ok bro. I’m not on all cylinders today.

non_burglar@lemmy.world on 23 Apr 13:50 next collapse

Zombies that stick around for more than a few seconds indicate a signal problem in the parent process, where its init is stuck in the “wait” state, so the entry remains in the PID table.

It could be harmless, but it could become a problem if you need the resources. Curl shouldn’t be doing this on its own.

irmadlad@lemmy.world on 23 Apr 13:59 collapse

It could be harmless, but it could become a problem if you need the resources.

That’s the thing. None of them are consuming resources. I guess I should just ignore them, but it irritates me when I start my server, to see zombie processes. Makes me think something is askew.

GreenKnight23@lemmy.world on 23 Apr 15:19 next collapse

that sounds like poor garbage collection in an application.

I’ve written software that had similar issues when writing to files and I failed to “close” the file after writing.

processes stay open, files stay open. 500 byte processes times 10000 orphaned process can make for a bad time.

jj4211@lemmy.world on 23 Apr 18:27 collapse

Zombie processes do not use resources, well, a little, it’s basically an entry describing how it exited.

The parent process is the thing keeping the zombie entry open. Killing it’s parent should work if they bother you.

non_burglar@lemmy.world on 23 Apr 19:36 collapse

Zombie processes do not use resources, well, a little, it’s basically an entry describing how it exited.

Agreed, but a very poorly-written program having a hanging memory or disk write, or a file lock could become a problem, especially if hundreds or thousands of zombies are waiting for something.

gravitas_deficiency@sh.itjust.works on 24 Apr 11:53 collapse

disturbingly pervasive shuffling noises intensify

folekaule@lemmy.world on 23 Apr 14:17 next collapse

No idea about your specific case, but in the past when I’ve had frequent zombies it’s been due to some blocking network process; typically NFS mounts.

They don’t use any resources, but I agree they’re annoying and could indicate some underlying issue.

I would check for issues with I/O and network access, especially anything that happens at kernel level. Look for anything suspicious in dmesg.

irmadlad@lemmy.world on 23 Apr 14:35 collapse

due to some blocking network process; typically NFS mounts.

I’ll check that out.

zewm@lemmy.world on 23 Apr 15:22 collapse

Awww man. Here I was thinking we were having a movie night.

melfie@lemmy.zip on 23 Apr 16:01 collapse

Right? I’ve got the original and the 90s version in Jellyfin on my home lab server. 🧟‍♂️

zewm@lemmy.world on 23 Apr 16:13 collapse

<img alt="" src="https://lemmy.world/pictrs/image/092e3d3a-ce3c-42b9-addb-d9af615b1052.gif">

irmadlad@lemmy.world on 23 Apr 16:16 collapse

LOL