I'd love to see you do a collaboration with Lawrence systems.

so much gold being spilled for free man, thanks uncle Wendel, you make us better IT Masters

Nice video, thank you! Couple little remarks: HAProxy-WI doesn't take all stuff from SSH. Main part information provides from sockets. Also GUI can install haproxy, nginx and keepalived and configures your hosts, so you don't need install them by manually

OMG I was literally wanting to do this and was research. Was going to make a forum post asking about software.

thanx for this, I had just started investigating what I could use to set up this exact same senario for 2x seperate private networks, both on RPi's - the fact that it has already been prototyped for two seperate users/networks, AND documented, has already taken a load off my mind, especially the securty aspect of it all - cheers, and thanx again

Dude this is ace. I can’t believe I haven’t done IT in like 20years but I understood everything you just explained. Last time I did this was at uni of Canberra in 2003 using Apache as a reverse proxy facing the internet and everything else was behind it. I was actually employed as a web dev and when I rocked up I was like “hey reverse proxy” and they (systems) were like “oh fuck, why didn’t we think of that” lol.

Thanks for making this video, Wendell! You're a legend, extremely interesting stuff!

I was trying to do something like this yesterday! I'll revisit this video during the weekend. Thanks Wendell!

Man Wendell, no wonder your Linux videos take awhile! That was a work of art! Its like if Leonardo DaVinci was a computer janitor/plumber. Keep it up! Im definitely gonna try this

@Level1Techs

4 жыл бұрын

Wait till I explain how to add snort/securicata for moar awesomeness

I've been thinking of hosting my own email for a while this is certainly a good starting spot.

Wendell, this video ís awesome! Thank you.

I have been looking for some of these answers for almost as long as your video has been posted... Thank you!

Amazed, great explanation. Great tutorial.

We use HAProxy at work. I setup SSL termination and reverse proxy services for web apps. It's an excellent program. I didn't know there was a GUI out there for it.

When Wendell comes out with a Linux video I don't even need a notification my Nerdy Sense just tingles.. 😏 By the way check Nginx Proxy Manager stupid simple to use with Docker.

@Xantioss

4 жыл бұрын

Also check the nginx-proxy-manager assistent on github 👍

@dayvie9517

4 жыл бұрын

No don't

Great content as always...

Just gave it a try. Good stuff!

I use Haproxy in my day job. Pretty cool piece of work.

Great video, for a company i created a drag and drop GUI tool to create ha configs that could be directly deployed to proxy servers.

This is pretty neat. I've done a similar thing for email, except rather than using a proxy, using postfix to forward via SMTP to my home server. This way I get more control over the email protocol, and messages can queue up for days rather than hours if there is a network issue.

@autohmae

4 жыл бұрын

Depends what you prefer, your VPS provider reading your incoming mail or forwarding the encrypted TCP-connection from the VPS to your home machine.

Damn - just noticed the Digital Unix box in the background. Used to have that in miy office. Looonnnnggg time since I saw that!

This is very similar to what Helm is doing, but they are selling a complete solution for ~$500.

Amazing

The amount of ACLs on my pfsense box is getting insane for haproxy, that coupled with the dynamic dns service is wonderful.

Wooooo! HA proxy!

Been running like this for years, but imo in a easier way. Could be more secure and easy to set up if between the VPS and your home connection you use a VPN, then use SSL termination on the haproxy before forwarding. This makes things easy to manage imo, the cert used is in 1 location and the haproxy config is pretty simple. In my case there is a VM with docker containers that serve anything from web to tftp, pxe servers and more on the LAN. For the VPN, you can use pfsense or in your NAS Additionally for the letsencrypt cert, when using pfsense the renewal can be handled for that by it. A tip for this, add a post script to put the cert on your NAS. That location can then be mounted on your VPS due to having the VPN, run an automatic reload script when the cert changes. This way there is 0 down time and no intervention needed.

I use a similar principle at my homelab. Instead of HA proxy you could also use docker + wireguard to do the same. My setup runs a wireguard server on linode which forwardes the traffic to port 80 on my homeserver where I've setup letsencrypt + Reverse Proxy to serve the services from my server. All the docker containers are installed on my homeserver and only packet forwarding and point to point vpn is setup on my public server.

MORE of these!

Nice video Wendell, I literally just got done doing this for my home. Minus the linode server. I was just considering doing this, and was looking at AWS and Google's offerings, when this video popped up! I may give Linode a shot. I'm curious do you run a VPN as well? For access to less "securable" services/appliances? Or if you lean more towards this method of publishing what you can and staging them behind the HAproxy?

I used to do something similar with SSH to make a webserver on my notebook/phone available(behind an authenticating proxy) via my VPS. Also that security by obscurity thing. I used to host a small website at home. But traffic was getting annoying. My solution? Drop ICMP pings at my router. My website was still available, but almost all scanners ping'ed before trying HTTP.

Dude from the picture at 2:38 reminded me of Qain a little. Should have asked him to reenact it for you as a cameo if you still talk

If I wanted to forward client headers through the proxy to the backend, how would I do that with HA Proxy? I'm using OPNSense rather than PFSense because PF doesn't support the NICs in my hardware.

Nice setup, I've been using a Raspberry Pi running the swag docker container. My AWS Route53 domain CNAME entries all point to duckDNS (DDNS) which is kept up to date by the same Pi. Ports 80 and 443 forwarded to the Pi's 180 and 1443 ,which Docker then maps back to 80 and 443 for certificate validation. I can then point any subdomain.mydomain.com registered in Route53 to something in my local network using proxy-conf files for nginx. I'm not suggesting it's better than the HAProxy-WI setup, but it's a low-effort alternative for simple home hosting.

I have a similar setup between a public-facing proxy on DigitalOcean and services running locally at home. I use OpenVPN to connect my home machine to the public server and route all traffic through the VPN. This requires zero open/forwarded ports on my PC since I'm running the OpenVPN server on the Droplet.

QUESTION: Why couldn't you also use NGinx as a load-balancer also? Why go to HA-Proxy?

I've done a similar thing, but I don't have any ports open on my home firewall. I have wireguard setup where home is the client and my hosted virtual machine is the server. So home exstablishes the connection out to the data center and the proxy connections come back over the VPN.

Nice video and idea! Have you thought of protecting this with a WAF?

Would a plex server load the content through the HA proxy or straight to the client from the server isp?

Wendell, is a GOD!

This didn't quite work for me as published. I've posted a comment in the forum link above about the changes I had to make to get it to work.

Linode also blocks port 25, 587 and 465 unless you send them support ticket to open it.

So what do you recommend to setup your own email server?

Hey @level1linux, Have you tried this with Cloudflare Tunnels? They're free now, and it only requires outbound connections.

Thinking about doing this, but using WireGuard to encrypt the traffic between my home and the VPS

And I am here sitting with my traefik... i like my gopher ;D

@charlese2833

4 жыл бұрын

Old school, gopher was cool

@vgamesx1

4 жыл бұрын

I'm not saying it wasn't partially on me but traefik was a huge PITA for me to get going, not a big deal once it's setup since you can more or less forget about it but that really sucked... If I had to do it again, I'd probably go with haproxy on pfsense. Also note that I used traefik v2 which isn't quite as well covered and I routed mine through cloudflare for protection which added some extra difficulty, more specifically pages not automatically being upgraded to https, so my way of solving this was going to cloudflare "page rules" and adding a wildcard for my domain and setting it to "Always use https".

You will be just fine as long as decent passwords are used and not displayed in plain text config files on your remote hosting service

Was just about to setup a nginx reverse Proxy to my NAS. Great timing Edit: Just watched the thing, two question remain for me: 1. Why should i not just use a standard nginx reverse Proxy, doesn't it do the same? (Except for TCP I suppose) 2. Anyway to do this with a dynamic home ip? THe only thing i can come up. Is to check on my nas as cronjob, and then automaticly update the IP in the conifs via SSH. Any better way?

@kassim3

4 жыл бұрын

In regards to question 2. Look into Dynamic dns(ddns). You can use something like duckdns or your router company might provide a free ddns because you're using their router. I used to have an Asus router and they gave a free ddns

@damian007567

4 жыл бұрын

@@kassim3 But i can only provide a IP for the hlproxy thingy right?

@LampJustin

4 жыл бұрын

@@damian007567 you can use hostnames for the services you expose ;

@LampJustin

4 жыл бұрын

As for question 1 there's a great webui for a nginx reverse proxy called nginxproxymanager.com It ticks all the boxes and yes running nginx as a rev proxy works pretty much the same. You can also forward tcp as well

@JzJad

4 жыл бұрын

In your case no point in swapping.

Just wondering why you didn't go down or at least an include an option to use DNS API-keys in-place of the http(s) challenge response? I know it's not supported by all DNS hosting services, but at least with the option, people would have a possibility of doing cert updates with 0 down time? Or the alternative is use http host based challenge responses, which is something nginx is quite good for.

This could be useful and interesting. I've used haproxy once before on a hosting service for potential ddos protection for my home network. For a dumb amateur admin (which I still am) the documentation for haproxy is overwhelming and difficult to comprehend.

@ 19:50 : Please do a video on setting up a user for doing haproxy config.

But what about Docker-Kubernetes rancher install for all this stuff?

Can you explain why you're using nginx as a backend on :81, only to redirect traffic to HTTPS? Can't HAProxy do that itself?

could HA Proxy load balance PCoIP and VMWare Blast ?

Suggestion: On CentOS, use certbot-auto. It’s not available through the package manager but avoids all the weird Python version issues that you may have.

@LampJustin

4 жыл бұрын

Run it in a container ;)

I think I might be doing something similar with traefik is HAproxy comparable to traefik?

@RealDids

4 жыл бұрын

Traefik has more features, including Let's Encrypt support. Just be aware that Traefik v2 doesn't do scaling without the commercial version, at least not with Let's Encrypt support.

This is the kind of kung fu I'm about. I was using digital oceans vm machines for a long time, Iptables kungfoo for the win.

I prefer acme.sh over certbot for wildcard certificates. As the name suggests it is a shell script without any dependencies. If your nameserver isn't compatible with ACME v2 wildcard certs, you can create a subdomain which is handled by a compatible and free nameserver like zonomi.com.

Watched briefly (i will go into details later). So external proxy just maps your subdomains to local adresses right? What if you want to expose (nextcloud) to outer World? You would need to redirect request nextcloud.wendel.com to yours networks router WAN IP. And this requires external IP (not behind ISPs nat) and static one or dyndns domain like free duckdns. I already have my nextcloud setup done like that but i dont have proxy. But i want to add one, internal.

To solve the issue of passing root ssh credentials that allow a remote user to change your haproxy config without creating the unnerving threat of such a user doing bad stuff there is a very easy solution available - run the haproxy server in a separate container (I use LXD) - then yes, the haproxy can be changed by root but root access is restricted to an haproxy container only. Thre's nothing else to run. To break the security on that, you first have to ssh into the container (easy - but only to those who have the ssh key, i.e. your remote 'root' user) but then you need a working vulnerability/exploit that can break you out of the container into your host machine (where root access is a much more serious breach), which is very hard to do, even for experts and nation states, especially if you keep the container up to date automatically (since exploits normally gets patched very quickly, often before the vulnerability is publicly identified). My haproxy server is thus an LXD container, and it sends traffic to my different home servers. It has no other means of accessing the servers on my home network - it can't find them via root acccess of the container.

Any reason not to just run HAProxy-WI locally on your own network?

I use Traefik + Let's Encrypt with Cloudflare DNS

@jgould30

4 жыл бұрын

My biggest issue with traefik is that, from what I could tell, it's built expecting docker. I managed to get it working on Linux container and/or VM to serve static web hosts. But it was hard and sort of hacked. Without the docker auto discovery the usefulness of traefik is questionable.

@JackmeMe

4 жыл бұрын

@@jgould30 Yes because it was made for load-balancing docker web services lol

Whats IE ?

No idea whats going on as usual but watched it anyway.

I want to build a CCTV system from an old PC. 2500K 16Gb ram. I have a couple of 8 port BNC camera cards. I tried Linux but 3 times in a row had to be rebuilt within a week due to kernel stack errors. Anyone have any better software options for a CCTV system? Or know why the kernel stack problem? Can you restore from a kernel stack error without having to rebuild the entire system from scratch?

@MrBiky

4 жыл бұрын

Some cameras (HikVision in particular) have options to save recordings to a FTP server (or SMB, or NFS). Set the cameras to record 5 to 25 MB chunks and save them to your FTP. From there you can see your recordings by using something like VLC or whatever. I would suggest you to look for another Linux distro (I'm a fan of Void Linux), but I'm not a Linux evangelist, if you don't want to use free software and free yourself from proprietary shackles, you can use Windows on your old PC and install something like FileZilla server or whatever and run it 24/7. It does the job, albeit poorly. And if you're using a version of Windows newer than 8.1 (which is the last one supported, I don't recommend running Windows 7, since it's unsupported), ie Windows 10, you may have trouble with Windows auto-restarting for updates. From what I hear, it isn't so bad as of lately, but I heard mixed feelings (for some it restarts, for some it doesn't). Not sure exactly what Linux you used, but usually there may be a way to save your old system if you live boot and repair some stuff (depending on what exactly broke).

Why not just install a second pfsense box in the cloud instead of haproxy-wi?

I do this with pfsense on cloud and vpn back home

I am doing the same thing with just nginx!

@myselfremade

4 жыл бұрын

Me too!

@DantalionNl

4 жыл бұрын

But can you load-balance the same service behind multiple instances of nginx running on different machines and easily see how many requests are routed to which servers and corresponding response times?

@Fahdalrabeayah

4 жыл бұрын

me too nginx proxy manager

@geogmz8277

4 жыл бұрын

Me too, using Docker Swarm and Nginx Proxy Manager..

@Level1Techs

4 жыл бұрын

nginx has a lot of cool features and that's totally fine. I used to have to use nginx plus for something.. I forget

woohoo linux stuff

I've always wanted to be able to run my Nextcloud instance on my home server. Unfortunately, my ISP is Comcast and they suck with their outdated cable infrastructure, overpriced bullshit, and 1.2TB datacap.

i mean, i just used apache to proxy all my services to 80 and 443, on one system, and then have them proxied through cloudflare

How would this deal with your home not having a static IP

@Level1Techs

4 жыл бұрын

use a ddns hostname for the proxy config, or write your own script. no waiting for dns to roll over (if not using ddns). Once you change your haproxy config, the change is immediate.

@andljoy

4 жыл бұрын

@@Level1Techs i was thinking of ddns , i guess a script that could update the ip directly would be better tho. I don't have THAT much need to host stuff internally ( well i do host lots of stuff but not much of it is external). Prob just a nextcloud so i can sync my photos from my phone without google spy shit. Hmm i wonder , could i do some janky arse crap with say a externally hosted nextclud but with the storage internal and pass just the storage over , a terrible idea i know , but a fun .... can it be done :)

@vgamesx1

4 жыл бұрын

This can be done super easy with cloudflare if you use this: github.com/oznu/docker-cloudflare-ddns You just make an API key for it to be able to update your IP address.

If you use a server in a datacenter to act as a gateway like this, isn't your home internet speed and bandwidth simply now limited to whatever the server host is offering? I get that this still gives you control of your own storage.

@LampJustin

4 жыл бұрын

True as all traffic goes through that server but not an issue since almost all providers give u simmetrical gigabit connections which is definitely fast enough.

@heckyes

4 жыл бұрын

@@LampJustin Yes, but then that server (unless metal dedicated) is prone to privacy concerns no? Even a good container vm solution like XEN/KVM can still have memory dumps and such right? Also, I can't seem to find a good price for a VPS or greater server package that has an UNMETERED gigabit port. I wish though.

I'd just use nginx to proxy my web traffic... but HAproxy is good too

interesting, but I would be more interesting in running this on an actual razzberry pi (along with pi-hole) locally.

@Mr.Leeroy

4 жыл бұрын

locally > pfSense

Why are you forwarding port 80 to an nginx server to handle https redirect? HAProxy can do this itself.

302 redirect? Why not 301?

acme.sh is amazing for let’s encrypt

tldr: your own cloudflare

2:04 EMBY? I'm disappointed. Use Jellyfin after Emby showed the open source community the big middle finger

Before I would've recommended Traefik over HAProxy, but now with the v2 I'm not so sure. I still love Traefik, but the free version of v2 doesn't so scaling anymore. That said, Traefik still has too many features that HAProxy doesn't, such as built-in Let's Encrypt support.

@DantalionNl

4 жыл бұрын

The main problem with Traefik is that it is incapable of dealing with client certificates it will break as soon as it encounters one. Luckily, that is not a common problem as most things don't require client certificates anymore but if you have a service that does, Traefik won't be a solution.

@LampJustin

4 жыл бұрын

@@DantalionNl at least they're working on that aws! But yes that's a bummer. Even though they changed pretty much everything with v2 the way it's done is now much better. It's more like Kubernetes which just blew my mind at first but now after looking into it, it makes so much sense!

Got it guys? Good.. Coz i didnt :)

Great Video, but you could do this similar task a lot easier with Nginx Proxy Manager as a docker service.

Wendell, why Plex and not Jellyfin?

@LampJustin

4 жыл бұрын

Because Plex just works better... I like jellyfin better myself but sadly most if not almost all viewers r running Plex.

@kelownatechkid

4 жыл бұрын

Run both in parallel IMO. Plex is a horrible company but some people can't change their client devices. Jellyfin can be at the ready for switching whenever possible.

why not just use ubuntu or at least Debian?

I run nginx myself.

I use HAProxy on pfsense. Way too easy.

You look tired sir. Please take rest for your body, better care of yourself. I can see your stressing. Please. Please remember to stretch and go for a 2.5 mile walk every day for basic health benefits.

you can setup redirect to https inside haproxy just add something like redirect scheme https code 301 if !{ ssl_fc } in frontend config and that's it. in case your nginx doesn't know how to work with that just add: http-request add-header X-Forwarded-Proto https if { ssl_fc } in backend config also no need to use acs to redirect traffic to another backend, you can simply use: use_backend cloud if { hdr(host) -i cloud.wendell.tech }

9:53 actually, not 10 years, but 20 years. 14:04 4 hours ? more like 4 days you mean. 19:20 euh... can't you just use volume mounts to the host ? haproxy socket and haproxy config, etc. ? 23:46 Starting from 1 September 2020 you won't be able to get 3 year valid certs anymore, only 1 year.

In this particular scenario HA Proxy is unnecessary. You can even do TCP/UDP Proxy with nginx.

@kelownatechkid

4 жыл бұрын

I agree, a lot easier to use nginx IMO.. nginx + certbot problem solved for 99% of home users in like 15min. Just gotta make sure you have ddns which is easy

ENGAGEMENT

Can't all this be achieved with just Nginx?

@myownsite

4 жыл бұрын

Nginx can do high availability, but HAProxy is a breeze to configure once you get the hang of it. No idea about the webUI, though.

@sp00k1es

4 жыл бұрын

@@myownsite There's been some recent UIs for Nginx, though I'm not fond of them. Personally I prefer the Nginx config syntax, that's why I asked, especially if you add the javascript plugin module to branch out into more complicated scripts for things if you need to.

@myownsite

4 жыл бұрын

@@sp00k1es I highly dislike GUI fronts for software which have robust text configs available and similarly overly complicated configurations. Nginx is a great web server and haproxy is a great load balancer. I think utilising strengths of both tools is the best approach, with as simple configs as possible. That way maintenance and deployments stay feasible.

You really should explain this in more detail it's way too complicated

The virgin maintainer made it difficult to install because he's too poor to get another job that pays.

A guide without the use of the haproxy tool would be great.

I watched the whole video. To bad I don’t know Chinese.

Don't recommend god-ddy to anyone, they're a -terr- not a nice company!*

just way too long, format sucks, don"t want any more vids like this thanks