Self-hosting Upgrade: planning my first 3-node cluster for deployment in a data center (discuss.livingcartoon.org)
from kiol@discuss.online to selfhosted@lemmy.world on 30 Apr 22:11
https://discuss.online/post/39090322

cross-posted from: discuss.online/post/39090314

Full layout on my forum here. Basic idea is 3 thin clients that fit within:

  • 1u space requirement
  • Possibly adding 2.5gb nic to the pci-e low profile slots of each hp t740 for Ceph shared storage.
  • Kubernetes and/or Proxmox experiments
    • Adding some sort of remote management via jetkvm or pikvm or similar.
    • Additional expansion no doubt needed for connecting to the three nodes.
+----------------------------------------------------------+
|                    CLUSTER TOTALS                        |
+----------------------+-----------------------------------+
| TOTAL CORES          | 12 physical / 24 threads          |
| TOTAL RAM            | 192 GB DDR4                       |
| TOTAL STORAGE        | 1.5 TB NVMe (local only)          |
| TOTAL NODES          | 3                                 |
| TOTAL NICs           | 3–7 (depending on 4-port NIC)     |
| GPU                  | None                              |
| KVM/IPMI             | None                              |
+----------------------------------------------------------+

All suggestions greatly appreciated! Never done this before, but hoping to save myself from the future hassle of having to drive to the data center… would much rather log in on Wireguard and address things by remotely rebooting, etc.

#selfhosted

threaded - newest

frongt@lemmy.zip on 30 Apr 23:15 collapse

What’s your plan for when you reboot and it doesn’t come back up? I strongly recommend having some kind of IPMI and virtual console, whether that’s pikvm or whatever. Far too many times I’ve had a server go down and it’s saved me from having to drive down to the datacenter and stand in a loud, frigid room and troubleshoot the issue.

kiol@discuss.online on 01 May 00:06 collapse

I thought I listed it in the description, but this is something I’m actively wondering about. A KVM over IP, such as PiKVM, Jetkvm, openkvm that allows access to all 3 nodes and saves future drives to the data center by using a wireguard connection instead. PiKVM + HDMI and USB switch is an option. Recommended option from PiKVM docs is easycoolav.com/…/ezcoo-hdmi-20-switch-4x1-with-us…

femtek@lemmy.blahaj.zone on 01 May 01:07 collapse

I really like my Jetkvm, has helped me out.

vinnymac@lemmy.world on 01 May 03:00 next collapse

Same, I use Jetkvm for this too, has worked great

kiol@discuss.online on 01 May 05:21 collapse

Cool, have you tried using it with multiple devices on an hdmi switch?