I’ve been a Software Engineering Student for 2 years now. I understand networks and whatnot at a theoretical level to some degree.
I’ve developed applications and hosted them through docker on Google Cloud for school projects.
I’ve tinkered with my router, port forwarded video game servers and hosted Discord bots for a few years (familiar with Websockets and IP/NAT/WAN and whatnot)
Yet I’ve been trying to improve my setup now that my old laptop has become my homelab and everything I try to do is so daunting.
Reverse proxy, VPN, Cloudfare bullshit, and so many more things get thrown around so much in this sub and other resources, yet I can barely find info on HOW to set up this things. Most blogs and articles I find are about what they are which I already know. And the few that actually explain how to set it up are just throwing so many more concepts at me that I can’t keep up.
Why is self-hosting so daunting? I feel like even though I understand how many of these things work I can’t get anything actually running!
You must log in or register to comment.
Part of working with tech is knowing what to search for und using the right keywords. If you could give an example what guide / information you are unable to find, someone could give you an example how to search for it. I personally know a few junior devs and junior devops which use llama2 / chatgpt since they dont know how to search for it or read the docs.
I actually hate GPT, dislike it’s answers and find myself knowing better than it most times.
I’ve been trying to setup a DNS server to create my own domains internally within my VPN but I keep finding info on how DNS servers work, and how to make a records on registrars, but nothing on what I actually need to install and run to have my own DNS for example. Same thing goes for many other services, but that’s the one bugging me for the longest time because it should be so simple.
I’ve found plenty of tutorials on how to make a cache DNS, just not an authoritative name server btw, and I’ve searched for both DNS and name server to no avail. If it was Linux I’d write some custom rules in my hostfiles and be done with it, but it’s so much harder to do on Windows and that’s my daily use OS for now…
“self hosted DNS server” is what I searched for, but I also knew what I was looking for “Unbound” with PiHole is what I use.
Also, Windows has a host file that it uses. %windir%\System32\Drivers\etc\hosts
%windir% is the environment variable for your windows install directory. Usually C:\Windows
You can add your entries in there, just like Linux, though I am sure the syntax might be slightly different. I really haven’t played with host files on Linux, yet.
I hate the fact you gave your honest opinion and othets have to downvote you. I myself rarely use ChatGPT to find a direct answer, instead I use it to give me resources to find the answers for the questions I didn’t think about asking or knew existed. More like thinking outside the box or brainstorming. And always do your due diligence to verify any answer it gives.
“Selfhost authorative dns” returns this for example https://wiki.selfhosted.show/DNS/
And we have two guides for the most common dns implementantations. Depending on the vpn software you can push the dns ip as part of the configuration or by using DHCP.
I also found the pihole docs for unbound rather helpful https://docs.pi-hole.net/guides/dns/unbound/.
For me, I have that as the 4th result, after some Reddit and IBM which probably would’ve discouraged me from continuing my search. I’d have to read on it.
Also, TIL PiHole doesn’t necessarily need to run on a Raspberry Pi. I guess assumptions really do come back to bite me in the ass haha
There’s a lesson in here somewhere about patience. Get good at skimming. I was looking for how to do something I was unfamiliar with the other day and I had to sift through 15 results across four different search strings before I found the solution that was going to work for me. But because I’m good at skimming it only took me 1/2 hour to discover and implement. Google isn’t magic and this is why someone else recommended Chat GPT to help with some of the sifting, especially early on.
Pi runs Raspbian which is just Debian with customisation applied. So of course it can run elsewhere. You don’t know as much as you think you do perhaps 😉
I seriously thought it was a product, rather than software tbf. The name always sounded so “corporate” I never considered it.
I definitely know more about the theory than the practice. I’m clueless as to what my options even are so I can’t argue with that.
But I did know about the Linux “inheritance” of distros if you wanna call it that, and I’m fully aware of what that entails.
Just honestly didn’t look at it twice cause I thought “there must be an FOSS option” without realizing what PiHole really is. Just a case of prejudice biting me in the ass I guess.
Sounds like the next step in your journey is combing through this list and seeing what’s out there: https://github.com/awesome-selfhosted/awesome-selfhosted
So much great stuff! But most of it has drawbacks, like missing features or less attractive UI. But it’s free and open source so we love it all the same.
I’ve read that repo a million times! My self-hosting needs are more esoteric and I mostly play around with it. I’ve no need for media services or 90% of what that repo offers yet!
I mostly want to end up self-hosting my own apps, but I need some foundational knowledge
I use Pihole for this in my home network. It has the capability to configure local domains for your internal network resolution as well as create whitelist/blacklist and use as a DHCP server alternative to your home router – you just need to configure your router to point to the Pihole instance for DNS on your LAN network configuration to get started with the DNS piece
Because most people who develop these things are, frankly, terrible at good documentation, or understanding the end-user perspective.
There’s also a downward spiral effect when you start getting into these things, because lots of them require dependencies, or ask you to do things but don’t explain why, and you’re just left wondering why you added that line to a config file somewhere, but if you don’t put it there, nothing works.
A vertical slice of the amount of knowledge you need passes through so many different disciplines, operating systems, GUIs, and programming languages that it would look like a Milhojas cake.
I’ve been a technical writer in the software industry for 17 years. The number one challenge in my work is extracting all of the information I need to write good documentation from the experts elsewhere in my company.
You’re kidding?
I’m an avid home-labber trying to get into software engineering and I’m finding the software dev work to be super daunting lol
I can setup a reverse proxy in my sleep - but write a JS function without double checking google? forget it!
It comes with experience I guess, I’ve got a bad habit of researching to the core and many times have a hard time grasping things like containers without understanding how it’s setup technically. Sometimes I find a decent explanation, but specially for libraries that do “magic” I gotta go diving into the source to understand what’s going on, else I have trouble understanding what I am doing and what I should be doing.
Which makes it so hard because networking is very low level and I’m very unfamiliar with this environment
NetworkChuck on utube
Each one of these things is a separate topic/project. There is of course, overlap, but concentrate on just one at a time. If you try to do everything, you’ll go mad. Application Administrator, Network Engineer, Sysadmin, Network Security, are often separate full time jobs. Just concentrate on your own needs, one at a time, then focus on how simple you can pare down that one thing you need/want to self host. What is the minimum set of capabilities you must have for this one thing? Then, expect to have as much research for this one thing as taking a 1 credit college course, at least. Maybe more, maybe less, depending what it is. That’s a more realistic set of expectations.
I’ve done a lot of web design and web hosting, and messing with Linux server stacks for around 30 years, now. You absorb a LOT over time, but it is all always changing. When I set up my first server, Cloudflare didn’t exist yet. VPNs weren’t a thing yet. The only people I knew with a network in their home were Computer Science professors. Wifi was not a thing yet. It isn’t you being inadequate, it’s you trying to do multiple jobs that each require constant professional development.
Check YouTube. I haven’t found a lot of written out guides but someone has made a video on it, I bet. Christian Lempa, Techno Tim, Db Tech, NetworkChuck come to mind for self hosting tutorials on containers and new applications. I’m sure there’s a lot more as well.
Thanks for the suggestions! The algorithms keep feeding me people who just explain what stuff is and it drives me nuts. You wouldn’t believe the amount of videos, articles and blogs I’ve seen on setting up a DNS server just for it to be about either a cache or an explanation of how it works. I’ll look into these later!
Jeff Geelong is another good resource on youtube, especially if you ever want to get into infrastructure as a career.
Github is also an amazing resource and always RTFM.
Edit: spelling
You don’t have to do it, 99.9999%^([citation needed]) of the population don’t. If it doesn’t interest or excite you then it’s not worth it.
I can barely find info on HOW to set up this things. Most blogs and articles I find are about what they are which I already know. And the few that actually explain how to set it up are just throwing so many more concepts at me that I can’t keep up.
This is big part of why I, even as an IT professional, don’t do much homelabbing or selfhosting. So often I’ll be scrolling through these subs and see something someone has done and I’ll decide I want to do that too, but so often I end up abandoning whatever it is because doing x requires googling for y and z because without y and z, x doesn’t work. And after a day of dealing with issues and tinkering on work stuff, do I want to spend more time troubleshooting? Often not.
Maybe you should read the documentation for the programs you try to install
Because it’s a complicated topic, you’re just feeling the actual responsibilities of doing it right that many are ignoring now.
Because security isn’t easy. Know you know why system admins have a problem with programmers that just expect things to work.
Most blogs and articles I find are about what they are which I already know.
Exactly… I neither want “what z is” nor all the history behind z, the Z inventors life, his grand parents lives etc. I just want a solution to the problem im having…
Honestly, I get they’re trying to be educational for beginners way more clueless than me. But after two years of an IT degree I know some stuff, and the sheer amount of internet text I’ve read just to find absolutely nothing new and no solution even though the title is exactly my problem is unreal
Maybe you have to go to a 4 year university where they really teach you. A 2 year It degree is from where - might not be enough to get you started. I remember taking lots of programming courses at the Harvard Extention school while working as a programming during the day. I would spend nights and weekends at the School’s lab programming my assignments. You gotta get your feet wet. Find a really good mentor or teacher that will guide you as you seem a little bit lost with all the complexity. Practice the KISS principle.
Enjoy it man! All your problems can be solved with enough effort. THis is the best part, the part where you are forced to learn a bunch of new stuff to move up in your skills :)
Best of luck! I love this shit!
I’d just install Docker and add a CloudFlare Tunnel to securely host your stuff. No need to open any ports, your WAN IP remains hidden and you have the benefits of DDoS protection. It’s free as well; all you need is a domain name for which you can configure the nameservers.
Putting a server in front of my own defeats the whole purpose of self-hosting for me.
I didn’t say CloudFlare “bullshit” so aggressively for no reason.
I want to learn, because I feel like I should know how to deploy stuff and my uni is not teaching me.
Do you have a purpose to host whatever you happened to name here?
There is a reason to host reverse-proxies, however, one can do without them in a self-hosted environments. First, one needs to understand the point behind reverse-proxies (this is an example btw, you might very well know the how and why behind them), and only then would the instructions to set it up start to make sense.
I don’t have a need but I do have a goal for the things I want to setup.
I got downvoted to oblivion for saying I didn’t even find what kind of software I could use to make an internal authoritative DNS service for example, where I want to create a custom internal TLD for my VPN.
But apparently people took offense I’d never heard of bind and assumed PiHole was proprietary…
assumed PiHole was proprietary
Did it look proprietary on their website?
Personally, I wouldn’t even run
bindin my homelab. I don’t see the point.dnsmasqis good enough for me. But if you’d like to run it, go ahead!Creating a new TLD for a VPN with an authoritative DNS for your local network isn’t the hardest thing, I’m sure you’ll find documentation on how to do so BIND’s website. If you don’t understand something just ask your search engine or ChatGPT/Bing (do not rely on LLMs for factual information, but they’re good at summarising information, from Wikipedia for example). Maybe get in the habit to RTFM, it does help.
BTW here’s a good list of internal TLDs one might want to use: https://serverfault.com/questions/17255/top-level-domain-domain-suffix-for-private-network
The things of “how to set up” are usually covered by software’s documentation, further supplemented by googling certain specific questions and going on various forums. Never just use the first page you find but crossreference information as well.
Having gone into self-hosting as a hobby with the intention of supplementing knowledge in my career path (currently still studying), it can become quite daunting quite fast, especially if you try looking at a big picture, rather than taking it step-by-step. The best thing to do would be to follow a tutorial within the documentation of something you’re trying to host, then expanding per step as far as dependencies goes, and then go from there. Compartmentalizing is a very effective way to not get lost.