Amazing, thanks again for all for all of this! I’ll start keeping my eyes peeled for any good deals on 3090s that pop up (though will probably end up prioritizing the NAS build first just to get my feet wet before diving straight into the localLLM world). But thanks again for taking the time to share!
Thanks for flagging this! I’d just passively absorbed second hand the mining rig fears, but you’re totally right that it’s not as though a regularly used overclocked gaming GPU isn’t going to also be subject similar degradation (especially if the miner is intentionally underclocking). I guess the biggest fears then are just physical damage from rough install and then potential heat damage (though maybe swapping thermal pads and paste helps alleviate that?) And of course checking benchmarks for any weirdness if possible I guess…
(Also very curious about all of the HA stuff; it’s definitely on my list of things to experiment with, but probably down the line once I’ve gotten some basic infrastructure in place. Very excited at the prospect though)
This is super interesting, thanks so much for sharing! In my initial poking around, I’d seen a lot of people that suggested virtualizing TrueNAS within Proxmox was a bit of a headache (especially when something inevitably goes wrong and everything goes down), but I hadn’t considered cutting out TrueNAS entirely and just running directly on Proxmox and pairing that virtualization with k8s and robust backups (I am pleasantly shocked that PBS can manage that many backups without it eating up crazy amounts of space). After the other comments I was sort of aligning around starting off with a TrueNAS build and then growing into some of the LLM stuff I mentioned, but I have to admit this is really intriguing as an alternative (even if as something to work towards once I’ve got some initial prototypes; figuring out k8s would be a really fun project I think). Just out of curiosity, how noisy do you find the old Dell servers? I have been hesitant both because of power draw and noise, but would love to get feedback from someone who has them. Thanks so much again for taking the time to write all of this out, I really appreciate it!
Thanks so much for sharing! I just poked around for the Ironwolf 8TB drives I was thinking of an it unfortunately looks like they’re sold out for now (as are the 8TB WD Reds it looks like), but I’ll definitely keep an eye out for them here (and honestly maybe explore some different size options honestly; the drive costs I was seeing on other sites was more than I expected, but wasn’t sure if that was just the new normal; glad to have another option!) And thanks so much for the awesomeselfhosted list!! I don’t think I’d seen everything collected in one place like that before, that will be super helpful!
Thanks so much for all of this info! You’re almost certainly correct that I’m overthinking this (it’s definitely a talent of mine). I had been leaning z2 on the NAS only because I’d heard that the resilvering process can be somewhat intensive on the drives, especially when they’re larger, but I had also seen folks say that this was probably overkill for most home settings and so I’m glad someone with experience on it could chime in. I think my biggest takeaway from what you shared is that it sounds like keeping the file system baremetal and fiddling with it as little as possible is the strategy (vs. virtualizing it and running it out of one large Proxmox machine, for instance). And I think you’re totally right on the LLMs being the real sticking point; I’d had no idea just how resource intensive they were not just to train but even to operate until I started looking into running one locally. It’s honestly making me think that maybe trying to roll this out in phases starting with the NAS (while also doing some other infrastructure upgrades like looking at running cat 6a and swapping out my router from the ISP all-in-one to something that can run OPNSense paired with some WAPs), might be a better place to start. Then, if I can get some early successes under my belt, I can move onto the LLM arena and see how much time, money, and tears I want to spend getting that up and running. Oh, and thanks also for mentioning TiB; it sent me down a very interesting rabbit hole on the base 10 vs. base 2 byte measurement and how drive companies use the difference to pump up the number they get to advertise; I had no idea that accounting for the discrepancy in drive size, but is definitely not surprising.
Thank you! I think I am just at the “Valley of Despair” portion of the Dunning-Kruger effect lol, but the good news is that it’s hopefully mostly up from here (and as you say, a good finished product is infinitely better than a perfect idea).
Thanks so much for flagging that, the above 4g decoding wasn’t even on my radar. And I think you and another commenter have sold me on trying for an EPYC mobo and dual 3090 combination. If you don’t mind my asking, did you get your 3090’s new or used? I feel like used is the way to go from a cost perspective, but obviously only if it wasn’t used 24/7 in a mining rig for years on end (and I am not confident in my ability to make a good call on that as of yet. I guess I’d try to get current benchmarks and just try to visually inspect from photos?) But thanks again!
Thanks, will do!
Thanks for this! The jet engine sound level and higher power draw were both what made me a little wary of used enterprise stuff (plus jumping from never having a home server straight to rack mounted felt like flying a little too close to the sun). And thanks also for the epyc rec; based on other comments it sounds like maybe pairing that with dual 3090s is the most cost effective option (especially because I fear you’re right on prices not being adjusted downward; not sure if the big hit Nvidia took this morning because of DeepSeek might change things but I suppose that ultimately unless underlying demand drops, why would they drop their prices?) Thanks again for taking the time to respond!
This is a great point and one I sort of struggled with tbh; I think you’re right that if I built it out as a gaming PC I would probably use Windows (not to say I am not very excited about the work Steam is doing for Linux gaming, it’s just hard to beat the native OS). I was leaning toward a Linux build for the server form though just to try to embrace a bit more FOSS (and because I am still a little shocked that Microsoft could propose the Recall feature with a straight face). Maybe I could try a gaming setup that uses some flavor of Linux as a base, though then I am not sure I take advantage of the ability to use the AI stuff easier. Will definitely think more on it though, thanks for raising this!
Thank you! Honestly I am probably going way overboard myself (I think I’ve tried to convince myself that it might make sense given the likelihood of tariffs around the corner, but honestly might still end up downscaling to more like 10-20TB of storage and radically reduce my LLM expectations). And thanks also for the Crafty Controller rec, I hadn’t heard of them and will definitely check them out!
Thanks so much! Appreciate the DDR4 and DRAM thoughts, and great point on secondaries. I have actually been debating the right place to put this as well. My ONT is in the basement (which is I feel like is probably the best place to put this from a noise perspective), though my sad cable company router is in a spare bedroom that I was considering as well (this option would require a little less rewiring, though honestly I’m probably going to have to either figure out how to run my own ethernet or hire out for it regardless of where I put it). No worries if not, but do you have a sense of what noise I might expect from the TrueNAS machine I am thinking of running 24/7 vs. the Proxmox that I won’t be using all the time? I think I could live with occasional noise spikes, but having something loud 24/7 in a bedroom would probably be cruel. And huge thank you for the warning on power draw: I have not been considering amperage at all and will need to look into that to figure out what I can sustain without burning the house down. Are there any other secondary variables you’d recommend I should consider? Appreciate all of your thoughts!
Thank you for this! Honestly maybe it’s just been all of the Youtubers I watch but I constantly feel like I have no idea about how to make things work (and also, to be fair, basically everything I wrote is just me reading what other people who seem to know what they’re talking about think and then trying to fit all the pieces together. I sort of feel like a money at a typewriter in that way.) Really appreciate you commenting though! It’s given me a little more confidence :)
This is exactly the sort of tradeoff I was wondering about, thank you so much for mentioning this. I think ultimately I would probably align with you in prioritizing answer quality over context length (but it sure would be nice to have both!!) I think my plan for now based on some of the other comments is to go ahead with the NAS build and keep my eyes peeled for any GPU deals in the meantime (though honestly I am not holding my breath). Once I’ve proved to myself I can something stable without burning the house down, I’ll on something more powerful for the localLLM. Thanks again for sharing!