I would really appreciate a full setup video with k3s, traefik and some other tools like bitwarden, nextcloud etc on a HA raspberry pi cluster that is reachable from the outside to some degree and everything using docker volumes via nfs, e. g. from TrueNAS Core. I tried setting it up, but failed because it was too complicated for me and I switched over to docker swarm.

I'm new to Devops, very enjoyable to learn from your content. Thanks for creating such excellent IT tutorials, that are incredibly beneficial for newbies. and the clarity of your presentation.

@christianlempa

Жыл бұрын

Thank you so much :)

Even though I already have a k3s cluster at home, this was still an interesting and informative video. Looking forward to other videos.

@christianlempa

2 жыл бұрын

Glad you enjoyed it!

Amazing video. Great quality, very useful. Covered some issues I was having whilst deploying

@christianlempa

Жыл бұрын

Thank you so much 😊

More K8s pleas😊 You and TechnoTim have convinced me to take the jump.

Awesome video, thank you so much for it! Would be great if you can show how to deploy k3s nodes with terraform.

You are great, man. Please do more I enjoy watching your content

@christianlempa

2 жыл бұрын

Thanks of course Ill do

I’ve set up k3s HA on Alpine at work. Running on top of vmware with first class storage. Perfectly smooth!

@christianlempa

2 жыл бұрын

Nice!

I want to do something by running kubernetes on a proxmox cluster however I still have a lot to learn about kubernetes and I assume the storage aspect will be more complicated with a Proxmox cluster. Thank you for the content I look forward to seeing more!

love this channel. k8s is indeed a magic blackbox 😂

Great content, thanks! There are actually very few videos on these particular topics.

@christianlempa

6 ай бұрын

Glad you like them!

Great video, as always :D

@christianlempa

2 жыл бұрын

Thank you bro!

I'm on the same journey now, but planning to tackle the hardware HA, external load balancer HA, and physical switch HA. I also have a UPS, and considering getting a generator and transfer switch. Then all that remains is redundant home internet connections. The next logical step after this is to then have redundant and geographically dispersed homes where I can run my home labs and finally not have to worry about an outage. Or so you'd think, because so far we're just based on technology based on transistors which could be wiped out from an EMP. Once I get my geographically dispersed homes racked and stacked I will then begin designing a mechanical computer which can be driven from multiple energy sources including wind, water, solar and coal. We might have to go nuclear.

This is awesome! thank you

@christianlempa

Жыл бұрын

Glad you like it!

28:15 you can apply anti affinity rule (with labelling of node) to make sure that pods get deployed always on all 3 and not together

@christianlempa

2 жыл бұрын

Yeah great idea! I admit I haven't fully understood this topic yet but I'll start running some tests with that!

@TheOnlyEpsilonAlpha

Жыл бұрын

@@christianlempa another method of ensure it’s running on all available nodes it to run it as a daemon set, this should be exactly what you want: Not multiple instances on one machine, but on each machine a pod of it.

This was a spectacularly well made intro to kubernetes/K3s for dummies like me who do not know anything about kubernetes. Very, very well done, sir!

@christianlempa

2 жыл бұрын

Thank you! Makes me happy to see it helps beginners 😀

Hi. Thank you so much, but i need to add: if you using the old sophos UTM its not wokring just with an availability group and dnat(couldnt get through except ping. My workaround was to use the feature "generic proxy", then the utm firewall is pass the traffic over that 6443 port. Sophos Xg is working like in the video described. Its actually a pretty easy on premise load balancer. Why didnt i came up with the idea...also good for many other things

That’s awesome!

@christianlempa

2 жыл бұрын

Thx! :)

So helpful

I'd like to see the K3S home lab video. Dooooo it!

Excellent video

@christianlempa

Жыл бұрын

Thx!

Thanks for this, I had no idea XG could be used for load balancing. I've just created alias's for my 2x internal DNS servers (Alias 1: primary, secondary, external | Alias 2: secondary, primary external | ICMP ping every 5s) - this should prevent any internet outages when I take down hosts.

@christianlempa

Жыл бұрын

You're welcome :) Nice project!

Awesome video!!! Would be awesome to do one with only pi's. Like 3 or more Raspberry Pi for the cluster and other Pi's for other things :) 8)

@christianlempa

2 жыл бұрын

Thank you! I'm currently waiting for more hardware to come, this includes some Rasberry Pi Compute Modules, that would be perfect for K8S :)

Have you used k3sup to install instead of a bash script. I'm curious about those tradeoffs. I've also been trying the rancher Harvester HCI as an underlay for my clusters. Seems pretty cool so far.

Really great video.... very well explained. Just out of curiosity.... why not separate the roles - server nodes and worker nodes? Is there any benefit to having deployments on your server nodes?

@christianlempa

2 жыл бұрын

Thanks! You could do that, but I think it's more efficient in a small setup to run it on the same server

Nice videos. I have a question : I did the installation without sudo, I mean withour root level. Does it works without issue ?

I would like to see something like HAProxy-Ingress instead of Traefik. This could also be used as a HA Load Balancer with Health Check.

@sebastiengauthier58

2 жыл бұрын

It’s possible to use Traefik in HA mode and it does have health checks

What are you using to get all diagrams like ASCII in markdown?

@christianlempa

2 жыл бұрын

I'm using asciiflow.com and some hand work to add nerdfonts icons

How would you go about adding encryption between nodes (servers/masters as well as agents/workers)? As far as I can tell this traffic isn't encrypted.

Any idea how to configure the network with Unifi equipment?

You should be able to bring down 2 of your nodes and still access Nginx. Your API and etcd are part of the control plane and not the data plane. Traefik and Nginx are part of the data plane. When you loose too many etcd nodes, they stop accepting updates. This means you can’t deploy any new workloads to your cluster and Kubernetes can’t heal any of the existing workloads such as deploying a new instance of Nginx that was lost when the other node was shut down. It has no affect on the existing instances of Traefik and Nginx. The inability to heal was probably why you wanted to deploy 2 instances of Traefik. If etcd was healthy, Kubernetes would have scheduled a new replica to an existing node.

how did you generate a new secret for k3s? Are there requirements for length etc?

At 3:00 you already start with prepared Proxmox VMs. Is there another without regarding Proxmox setup on new machine (let's say NUC8 or those 1L ThinkCentre boxes some of us have here sitting around)? Additional question, what cheap, small footprint alternative to Sophos XG we can use to start with this? Any software solution or it need to be hardware LB&FW? Thanks

great!

Hi! really useful information I will try it... Question, how do you made that diagram on the terminal???

@christianlempa

2 жыл бұрын

Using asciiflow.com :)

Really nice video! Just a question, is this enough to deploy ha stateful apps? How is ha storage problem solved? This topic can be very useful to handle it on a video.

@christianlempa

2 жыл бұрын

Thanks mate! Storage is another topic I'll do a video in a few months

Great video. I'd like to try K3S Cluster with two IoT Nodes, one node redundant of the other other, but the Rancher link states 3 nodes required for this mode. Three questions: It sure looks like I can try TWO following the video, i.e. skip the third node. What do you think? Even with your video, example, what happens to the Servers running on the second and third node? How would go about setting up a two Raspberry Pis (only two) so that one is fully redundant, copy, of the other?

@bluebobx

8 ай бұрын

As video says, use an external DB. Most decent NAS boxes can happily run Maria/PSQL. No need for this to be HA itself because you've just got lab use. Same here, but I wanted HA so I can upgrade server nodes without having to down my cluster (rolling upgrades via Ansible)

Should take a look at techno Tim's ansible playbooks he recently shared, automated K3s setup using ha etcd

@christianlempa

2 жыл бұрын

Yes I know, still need to find some time to walk through it :) thanks for sharing!

Not a Kubernetes expert, I just watched a few videos on it, but I think you can load balance inside Kubernetes itself. You can set a shared (cluster) IP for the three nodes and configure how the traffic is balanced between them.

@christianlempa

2 жыл бұрын

How would you set up a shared IP for the nodes? Every node needs a unique public IP address in your network.

@jmnimety

2 жыл бұрын

@@christianlempa look at MetalLB for one possible solution.

Note that if the k8s master goes down then the workloads continue to run, so having a single master node is a SPOF only for control plane, not workloads.

@christianlempa

Жыл бұрын

That's why the nodes all run the k8s master role.

Hi that's really great!! every k3s server is controlplane and worker at the same time, is it right?? many thanks

@christianlempa

Жыл бұрын

Yes, exactly ;)

Hello there, great presentation and demo of k3s. I am trying to find the Kubernetes installation notes that you are referring to at 11:24, but I couldn't locate them on your GitHub repository listed in the video description. Please help find the referred doc. I think over the years you may have re-organized the folder structure 🙂 Thanks.

@christianlempa

5 күн бұрын

Thank you :) I'm planning to make an updated video to k3s with new commands, stay tuned

you should do a separate video on setting up HA etcd

Hi Christian, thanks a lot for your wok on this theme 🙂 I tried to install k3s, but I don´t know what machine you mention at 19:28. You are only talking about 3 virtual machines and then you copy the config to your local machine? Ho did you install k2s on this machine? As Agent? I would like to test everything like you did but it seems that I missed something?!? Thanks.

@Glatze603

Жыл бұрын

And unfortunately I can´t find the files of this video any more in your github-profil?!?

Hi! Can't seem to find the link you referred to, regarding the firewall setup. Could you please share it? :)

@christianlempa

2 жыл бұрын

Oh yeah I forgot it :D Updated the link!

I really liked the ascii flows , how did you manage to add icons within ascii? it should be some ascii code but with what font?

@christianlempa

2 жыл бұрын

I used nerdfonts.com for that, requires some hand work, but is awesome

@lioandgatoula

2 жыл бұрын

@@christianlempa Thanx again, I managed to understand most of Starship and codes related to files, but the problem is that my Windows Terminal Version: 1.12.10982.0 does not render proper the Emoji of NerdFonts, that's a whole story on the internet either from the same JetBrains that a lot of users have. I have also try to use chcp to change the codepage to utf8 but still not working. I am not sure if you faced this, but actually the problem is that all Special characters within Nerdfonts with Unicodes and especially Emojis like Pizza Icons or GLass or whatever, cannot be rendered in my console, instead it shows Gibberish. Maybe it's more stable on Windows 11? I don't know

Nice video, thank you! Quesiton: do you use a tool to create those ASCII diagrams?

@christianlempa

2 жыл бұрын

Thank you! :) I'm using asciiflow.com and some hand work

@farzadmf

2 жыл бұрын

Nice, thank you for letting me know

I'm wondering to know how you draw your k3s architecture in Markdown, and I saw you can put an ubuntu icon in Markdown, I feel that's so cool, please shared how to do that.

@kearneyIT

9 ай бұрын

its called Excalidraw and its a plugin for Obsidian. very handy indeed. Hope that helps.

Hello everyone :) i tried it myself but sadly when i set the NAT Rule (i do everything the same like he does) the alias interface is not reachable (pingable) anymore, when i disable the nat rule everything is working again. Anyone an Idea why this could happen?

I'm a old IT guy looking towards a new DevOps Blockchain related career. I've been studying both Containers, Kubernetes, Portainer and Proxmox. But it seems I'm presently at a fork debating whether to go further towards Kubernetes or Proxmox. So far with ProxMox showing it's ability to manage Kubernetes clusters I'm thinking what rabbit hole would you choose?

@christianlempa

Жыл бұрын

I’d always chose to run a server virtually and then deploy Kubernetes on that vm, because it just makes things easier and you can run other systems in parallel such as windows, other Linux distros, etc.

@mikemorris5944

Жыл бұрын

@@christianlempa Thanks Chris your definitely one of the best resources in this subject. I'm a big fan 👍

Great Video!! What about securing the K3s host machines? How do I secure the machines when the cluster is cloud hosted, are there any best practices? I am planing to set up an k3s cluster on the Hetzner cloud, distributed over three databases. Then using Longhorn for the high availability storage. Also I am thinking if I should use the load balancer from hetzner, it is high available. The other option is to use Traefik hosted in the cluster so it should be high available. Also i thought about using both. The smallest hetzner load balancer can handle 5 services to „increase“ the potential services I cloud use traefik as an high available reverse proxy behind the load balancer. This way I can Host multiple websites and services using the same 5 available service ports. What are you guys thinking about it? Has anyone done something simlar, this would be my first Kubernetes project😅 I was using docker before

@christianlempa

Жыл бұрын

I guess k3s is really more practical for bare metal deployments, in cloud it might make sense to just run a managed Kubernetes solutions, as this usually integrates natively with cloud providers load balancers, k3s doesn't. As for the security, it's hard to find best practices here you need to take a look at different layers separately, the servers itself (my linux server security video might help here) and k3s + container deployments. Seems like a big project, but there will be more videos about security in a home lab, might be also helpful to you ;)

Hi @christianlempa, thanks for such a helpfull video. I'm going to try this setup out, but the only question I have: wouldn't it be possible to run second load balancer for services also on sophos and expose services as NodePort (and disable k3s traefik instances)? I have a separate hardware router and could configure HAProxy on it to distribute the traffic to all k8s nodes on given port for each service, but not 100% sure whether that would work.

@christianlempa

Жыл бұрын

Hmm actually that’s not a bad idea, I’m sure you can do that. However I think it’s easier to manage traefik than Sophos xg for exposing apps

@emilhuseynli

Жыл бұрын

@@christianlempa Then I think I'll give it a try, in spite almost every kubernetes tutorial/resource is always against exposing a service on node port. My case is a bit special because I have HAProxy installed on my router and it acts as a reverse proxy for other applications in my homelab, i.e. I already have some routing configuration there, why not registering all kubernetes services there as well.

Wie hast du die Grafik in markdown erstellt?

possibly I missed it but why did you pick K3s vs say Microk8s or other options? Great information, just always curious why people pick one solution.

@christianlempa

2 жыл бұрын

I picked it because it seems like a solid system that's also very efficient and easy to setup. F.e. the Cloud Provider Civo also built their SaaS Kubernetes Solution on k3s, which confirms this is a very solid system.

@Weirlive

2 жыл бұрын

@@christianlempa that’s awesome thanks for responding

What is your terminal and settings? very nice

@christianlempa

Жыл бұрын

Made some videos about that, check out the windows and Mac OS terminal videos on my channel

What is your terminal tool??? Is very cool! with multi windows

@christianlempa

2 жыл бұрын

I've done a video about it ;) kzread.info/dash/bejne/c39mrKdridaxpNI.html

Installation is similiar to rke2

Hi! Any reason not to use k0s or microk8s?

@itsvrl1856

2 жыл бұрын

K3s is just better, I live Rancher Labs and their easy to understand for a normie. K3s is an amazing entry gateway to Kubernetes. Where K0s, microk8s and minikube is way more thinking and with how minimal frustrating blockers for most teenager now a days, will end in giving up on learning Kubernetes.

I am really digging all these HA homelab videos, but if your cluster is on a handful of VMs on the same proxmox is it really HA? I ask because I don't understand why everyone runs their cluster as a handful of VMs would you not be more likely to have a hardware issue than to have a VM issue? I just moved over to proxmox in my homelab but only have 1 hardware server, so I've been reluctant to "do" much of anything on it that requires any sort of HA or prominence. Am I looking at this wrong?

@christianlempa

Жыл бұрын

No, you’re absolutely right, guess I mentioned it in the video as well, it isn’t a „true“ HA. However, the goal in my home lab is always learning and experimenting first. So what I’m doing is never because I really need it this way for myself, but to understand how the technology works, so I get an understanding of how it could be used in these scenarios. Hope that makes sense ;)

@JohnWeland

Жыл бұрын

@@christianlempa OK I can totally understand the learning aspect. I just wanted to make sure that I wasn’t missing a technical benefit of using VM’a for HA.

Sorry if this is a silly question, how can you make diagram like that on 8:38? did you create it manually or there's a tool for it?

@christianlempa

2 жыл бұрын

No silly question, mate! I'm using asciiflow.com ;)

@adenaziz3600

2 жыл бұрын

@@christianlempa ah, thank you mate! you just made my day 😁

There is an error when creating the first server you must not use --token=YOUR-SECRET or installation fails

This may be off topic, but I've been interested in exposing some web apps that I'm hosting on my home network to the internet. I'm a software engineer, but I don't have a lot of experience in infrastructure. Is there any reasonably safe way to expose servers to the internet? Could anyone recommend any learning materials on how to properly setup IT infrastructure? I've always been interested in setting up my homelab properly. Maybe using some combo of Ansible, Kubernetes, maybe some private code repositories and setting up a CICD pipeline locally. This is a very large endeavor and the sheer scale and lack of Infrastructure knowledge on my part makes me hesitant to even try. It's also discouraging to me as a beginner that all the existing tutorials out there are either outdated, don't work, or skip difficult topics.

@christianlempa

2 жыл бұрын

I will soon make a video ob CI CD and more

Side note, how are you displaying the block chart markdown in a terminal? Is that Mermaid?

@christianlempa

Жыл бұрын

No it’s asciiflow and some text editor magic ;)

Why do you need all the Sophos stuff? Love to see a guide that’s not tied to a product like Sophos. I believe the config can be without touching the FW yeah ?

@christianlempa

4 ай бұрын

Simple reason: I get it for free for testing :D

@Mishanw

4 ай бұрын

@@christianlempa love your content. I’m new to homelabbing and it’s been a heck of a ride. Running into walls and finding ways around. I’m a Uniquiti guy. So it’s a little tough to follow something’s. Thanks again

Weird question but does somebody knows how the DIAGRAM at 9:00 was made? The one for the preferred Proxmox setup

@RicardoRibeiro-dorcari

2 жыл бұрын

It's answered on the top comment, I believe that's what you're looking for.

@christianlempa

2 жыл бұрын

asciiflow.com ;)

How do you generate that graph that's shown at 9:13?

@christianlempa

4 ай бұрын

I think I was using asciiflow back then ;)

Out of curiosity why are you using a /16 on the network. I understand growth but a /16 is confusing to me, I’m wondering why not just use a /23 (512 IP’s) or a /22 (1024 IP’s) for your particular setup?

@christianlempa

2 жыл бұрын

The main reason I use a 16 network is that I can organize my IP ranges in a meaningful way that's easy for me to remember. F.e. I have an IP range for my DHCP IP addresses in 10.20.10.1-254, for my static test devices in 10.20.3.1-254, and production in 10.20.0.1-254. With a subnet of 16 it's still the same network, but I can easily remember the purpose and ranges of my devices. Don't know if that's the best way to do it, but for me it's easier ;)

These Demos are always shown with simple applications like Nginx and it's starting page. How does the configuration work with for example MediaWiki? It has it's own storage and Database in each instance. How are they replicated?

@christianlempa

Жыл бұрын

That’s a far more complicated topic, maybe I’ll cover that in a separate video, but it depends heavily on the app whether it’s capable of scaling and HA. Tried that once with WordPress and it was horrible xD

I am surprised that you are not using Terraform!

Was the following diagram done with mermaid? 8:46

@christianlempa

6 ай бұрын

no, I've used asciiflow and some handish work

Or maybe just use cassandra for an external database it solves all the problems :)

Storage is a real issue with Kubernetes. I have deployed several Kubernetes clusters at home and always abandon them. Storage is hard and not worth the effort for a home lab. I have tried high available storage like Longhorn but that consumes so many resources. Mounting a share means you need to taint your pods to run on the one machine with the mount.

@TomOtomanski

2 жыл бұрын

Kubernetes was built to primarily run non-presistent applications. Thing are starting to get better but ideally you'd want to run your storage outside the cluster. If you're running something like TrueNAS/FreeNAS then you should be able to use NFS as your persistent storage. For a home lab environment though why wouldn't local storage not be sufficient? Yes, you may need to configure nodeSelectors but this is just for a home lab setup?

@wstrater

2 жыл бұрын

@@TomOtomanski Kubernetes is designed to run multiple instances of your application across multiple machines so your app has high availability. The video is about running Kubernetes on a high availability cluster. Using node selectors for your app that is run with a high availability scheduler on a high availability cluster just so it has access to a drive mount just defeated all that HA. I am really interested in seeing how others solve this but I run my “production” home lab on a single machine using Docker because so many of my apps maintain state. No HA but no headache.

@travnewmatic

2 жыл бұрын

Democratic-csi with truenas has been my recent go-to!

@christianlempa

2 жыл бұрын

The people here also have great thought on that, I currently experiment with Longhorn, and yep that requires some resources, but seems like a great solution to me.

@camaycama7479

2 жыл бұрын

@@christianlempa please do the following video on Longhorn!

Hi Christian I dont see the files with the instructions in your repo

@christianlempa

24 күн бұрын

Can you tell me what reference you mean?

@user-ix6yc7pw2p

24 күн бұрын

@@christianlempa 11:18 min, you have a directory called infrastructure, but there's no kubernetes folder

@user-ix6yc7pw2p

24 күн бұрын

I don't see the file "install-k3s-ha-embedded-db"

I can't really figure out how you would connect to the cluster through a proxy (sophos in your case, nginx in my case). I have a self-signed certificate that I put on the nginx and that my computer should trust, however when I try to connect to my cluster through the lb-nginx I'm getting either untrusted certificate errors or "unknown error". You do not really explain that part at all as it seems to just work for you.

Installation is easier with k3sup

Caddy > Traefik

K3s HA vs Docker Swarms?????

@nabilelmasri4510

2 жыл бұрын

Docker swarm is practically dead

@km-ln8gp

2 жыл бұрын

@@nabilelmasri4510 I still use both based on use case...

Sophos is probably the worst firewall one can use.

I am not quite sure the complexity and management effort required by a Kubernetes cluster are worth it on a home server. A much simpler Ubuntu 22.04 LTS setup with Docker will amply serve 99.9% of all home server needs, with a much, much lower management load and much simpler installation.

@christianlempa

Жыл бұрын

You need to check out my other video about Docker vs Kubernetes! :)

Cannot watch a serious video with music in the background! Far too distracting! :(

The title is misleading, it is not related to Kubernetes, but with K3S which is not accurate. Downvote.

It looks like an ad, I can see that soon there will be no channel where Rancher and K3s would not be praised.

@christianlempa

2 жыл бұрын

Well, it's just a great solution, that is free and open-source, who wouldn't praise it? :D

@iounios_italia

2 жыл бұрын

Personally I appreciate this content quite a lot. It's free, it's informative, even sponsored products are advertised only as a suggestion and always offer a free plan...

@costenalolek973

2 жыл бұрын

@@iounios_italia ​ @The Digital Life The thing about Sponsored Content is that, it doesn't show any flaws, so it's not worth attention in any way, because it is half the truth.

@iounios_italia

2 жыл бұрын

@@costenalolek973 that is true, but I think that the channel’s target is mainly comprised of people with a sufficient set of skills so that they will try out the sponsored solution and easily get an idea whether it suits their deployment scenario or not. If some sponsored products are what it takes to keep Christian creating content which is useful to me , I will take that :)

you should try kube-vip with k3s, and I used that bind on my zerotier interface