from monty33@lemmy.ml to selfhosted@lemmy.world on 25 Jan 16:16
https://lemmy.ml/post/25194318
I have been having a few issues recently, and I can’t quite figure out what is causing this. My setup:
- gigabit WAN up and down. Run speed tests regularly and get 800+ mbps up and down.
- opnsense router VM (proxmox) running on a lenovo m920x. Installed an intel 2x10gbe card.
- Sodola 10gbe switch
- TrueNAS server (bare metal) w/ 10gbe serving the media files over NFS, stored on a ZFS mirror.
- Jellyfin LXC
- debian LXC running the arr stack w/ qbittorrent
- NVidia Shield w/ ethernet
First issue is extremely slow downloads on qbittorrent. Even if I download an ubuntu iso with hundreds of seeders will sit around 1 mibps. Media downloads with ~10 seeders, I’ll sit around 200kibps. Running this through gluetun and protonvpn wireguard with port forwarding enabled and functioning.
Second issue I’m having is if I am downloading anything on qbittorrent, and attempt to play a 4k remux on Jellyfin, it is constantly buffering. If I stop all downloads, immediately the movie plays without issue. 1080 files play without issue all the time.
I tried spinning up a new LXC with qbittorrent, and can download ubuntu isos at 30+ mibps locally and not over NFS.
Any idea what could be causing this? Is this a read/write issue on my TrueNAS server? Networking issuing causing the NFS to be slow? I’ve run iperf to the TrueNAS and getting 9+gbps.
threaded - newest
You need to remove a bunch of variables here to even start getting down to what may be the issue, but I’m guessing it’s just the networking.
From your title it’s also hard to understand if you’re trying to stream an unfinished torrent or not. If you’re doing that over NFS, of course it’s not going to work. You’re introducing a waterfall effect of client read/write verification from the source down to the consumer where the contents of the source file are changing and need to be updated and verified all the way down the line.
Hey pal super helpful follow up questions! Here is where I am:
Have not run speed tests, but yes it is a paid plan
Yes I have tried bypassing gluetun with qbittorrent, and no change to the download speed.
No I have not. What info could I gather here? Same file being downloaded in both qbittorrent instances I’ve tried.
No but if you read my post, I did try a second instance of qbittorrent that does not use my NAS.
None.
TrueNAS is serving the NFS, and the LXCs are mounting it via fstab entry.
Sorry I was not clear. I am NOT trying to play media that is unfinished downloaded. I am trying to play files I already have, and they are impacted by other files being downloaded on qbittorrent.
edit: formatting
Are all your containers running in bridged mode or host mode?
Also, how many cores are assigned to these containers?
The Jellyfin LXC has 4 core, and the Arr stack w/ qbittorrent LXC also has 4 cores. The containers are running in bridge mode.
You have more than enough cores for each then. Probably too many. As a test, try the qbit or jellyfin one in host mode and see if the network performance changes. I’d start going down the rabbit hole of tuning bridged mode network in LXC, or just keep them on host mode.
One other thing I changed recently is the motherboard on the NAS. The new one is DDR5 and I didnt have another machine that takes ddr5 to run the new ram through men test, and I didnt want any downtime so I didnt do it. I just powered down the NAS and started memtest. Do you think a bad stick of ram could be the culprit? At this point in just trying to rule things out
Bad RAM wouldn’t present like this. You’d more than likely never get past boot with a DDR5 board having caught it with POST tests, or you’d have thrown a kernel exception by now.
I saw you mentioned that a new LXC container didn’t have the traffic problem, so this is definitely something with config somehow.
Good points. I will finish the memtest thats running if only to have something ruled out. After it finishes I will try attaching the NFS share to the new qbt lxc and see if i get the same slow download speeds.
OK so I have done some additional testing:
So I believe I have ruled out memory issues, NIC issues, datapool issues, and switch issues.
The problem is I don’t know exactly when this started.
I did change out the motherboard on TrueNAS, and just installed the existing NVMe drives into the new motherboard and booted off of them. I did not install a new TrueNAS OS and restore a backup. Could this be an issue?
Shortly after the motherboard change, I upgraded to Electric Eel.
Try a test download without NFS and see what happens.
I tested that and I get full speeds. Upwards of 40-60mbps compared with the 1mbps I get when downloading to the NFS share
Problem solved then. You know where the bottleneck is.
Yes I’m pretty sure I’ve got it narrowed down to issues with NFS shares from TrueNAS. What I can’t figure out is how to fix it. I may do a backup, reinstall truenas, import backup, and see of that fixes it. I’m thinking potentially its an issue from reusing my old installation with the new motherboard, processor, and corresponding hardware.
Just don’t use NFS for large files. It’s not good for that.
I’ve accidentally turned on speed limits in Qbit before
confirmed all speed limits are off
What resources does your qbittorrent have access to? CPU cores / Memory. I have tried running it on a Rpi and it CHUGS so I had to aggressively apply seeding limits and general # of connection limit too.
The qbittorrent docker container runs on an LXC. The LXC has 4 cores and 8 GB memory.
What are the disks and how full is the pool?
The pool is a mirrored pair of 14TB drives. Pool is 56% full. SMART tests all pass, but the last scrub took over a week which was odd.