Automate Homelab Deployment With Terraform & Proxmox

Ғылым және технология

Terraform is a powerform 'infrastructure as code' tool to automate deployments. In this video I use it to create VMs in Proxmox ready for a Kubernetes or Docker Swarm cluster.
Examples Used:
github.com/JamesTurland/JimsG...
Terraform:
developer.hashicorp.com/terra...
Telmate/Proxmox:
github.com/Telmate/terraform-...
BPG/Proxmox:
github.com/bpg/terraform-prov...
Recommended Hardware: github.com/JamesTurland/JimsG...
Discord: / discord
Twitter: / jimsgarage_
Reddit: / jims-garage
GitHub: github.com/JamesTurland/JimsG...
00:00 - Overview of Terraform
05:39 - Code Overview
15:15 - Proxmox Configuration
16:32 - Starting Terraform
22:30 - Deploying with Terraform
25:06 - Outro

Пікірлер: 83

  • @AT-lp8iz
    @AT-lp8iz5 ай бұрын

    Thanks Jim, your clear and concise explanations of complex subject matter are some of the best - I’ve learnt so much from your videos !

  • @Jims-Garage

    @Jims-Garage

    5 ай бұрын

    Thanks, appreciate your feedback. Join Discord if you have any problems.

  • @simonbritton2677
    @simonbritton2677Ай бұрын

    Thanks Jim, you were right about things moving. Learned a lot based on some changes they have made, couple of things for anyone trying this as of Jun 14th. can now use the version in providers without having to make anything. terraform { required_providers { proxmox = { source = "Telmate/proxmox" version = "3.0.1-rc3" } } } The cloudinit_cdrom_storage = "local-lvm" seems to have been deprecated and moved to the disks block, my tweaks as per below worked for me. Adjust for your use of course. disks { scsi { scsi0 { disk { storage = "local-lvm" size = 12 } } } ide { ide3 { cloudinit { storage = "local-lvm" } } } } Again, thanks Jim, keep them coming.

  • @helioradial
    @helioradialАй бұрын

    Again, fantastic job Jim! Your channel is a pure gold mine! Thank you thank you and thank you! :)

  • @Jims-Garage

    @Jims-Garage

    Ай бұрын

    Very welcome, appreciate the feedback.

  • @TheRealClutch1010
    @TheRealClutch10104 ай бұрын

    Thank you for this series. I've used terraform on cloud providers and I have found the Telmate provider documentation to be lacking. Using your video and your example repo has helped me get my systems off the ground!

  • @Jims-Garage

    @Jims-Garage

    4 ай бұрын

    Great, thanks for the feedback. Good job

  • @MarcMcMillin
    @MarcMcMillin5 ай бұрын

    This is awesome! I'm looking forward to using it. Thanks Jim!

  • @ivanaevsky
    @ivanaevsky2 күн бұрын

    Perfect!!! Thx a lot!

  • @ninja2807
    @ninja28075 ай бұрын

    As always great tutorial. Thanks 💯

  • @Jims-Garage

    @Jims-Garage

    5 ай бұрын

    Thanks for the feedback

  • @rexeus
    @rexeus5 ай бұрын

    Great video! video suggestion- how to secure VMs/K8s clusters and use of tools like Wazuh etc...

  • @jhanssenfavaro
    @jhanssenfavaro3 ай бұрын

    Thank you so much by the Help! Very good !!!

  • @Jims-Garage

    @Jims-Garage

    3 ай бұрын

    You're welcome 😁

  • @Justin_Jay
    @Justin_Jay5 ай бұрын

    nice, will have a look at the repo

  • @jdturner2813
    @jdturner28135 ай бұрын

    my OCD would not handle the non-sequential VM IDs vs node names...great work as always

  • @moejoe4654
    @moejoe46545 ай бұрын

    Thanks Jim, I appreciate your videos a lot. btw it would be interesting to look into Ansible dynamic inventories for your next videos.

  • @Jims-Garage

    @Jims-Garage

    5 ай бұрын

    Thanks for the feedback and pointer. I'll take a look.

  • @hushandnoise
    @hushandnoise5 ай бұрын

    Not all heroes wear capes. Thank you Jim!

  • @Jims-Garage

    @Jims-Garage

    5 ай бұрын

    Haha, thanks. You're welcome 😁

  • @MrIvaroki
    @MrIvaroki5 ай бұрын

    Great video! Next step to deploy from git+cd ;)

  • @Jims-Garage

    @Jims-Garage

    5 ай бұрын

    That's the plan! All in good time... soon hah!

  • @ISBayHudson
    @ISBayHudson5 ай бұрын

    This is similar to something I've done - but I use proxmox containers nearly exclusively. I found the telmate provider and ran with it but later found bgp and realise now how better it is but switching seems like such a pain... I'll get around to it. Main comment refers back to containers and credentials. FYI connecting via API credentials has it's limitations as you can't created privileged containers (something I required to connect NFS shares from my storage device). With either of these modules you will need to switch to your root user credentials if you wanted to create privileged LXC containers. FYI I use terrafom for provisioning and ansible for config/ setup - keeping secrets in bitwarden and use the bitwarden ansible plugin to retrieve the secret

  • @ijustwanttoeatcookie
    @ijustwanttoeatcookie5 ай бұрын

    Awesome video! I am actually working on the exact same project and I’m using your script as input to provision the cluster with a few tweaks The only thing I added into the process is packer. I preinstall packages for my K3s-ansible controller and I export the public SSH key onto a share during building the cloud init image. I also use the ansible server as my terraform and packer builder server. Just a bit of a chicken and egg problem. For my K3s nodes I inject this public ssh key and create the ansible user in the packer image. Once all done my ansible machine can immediately connect to all K3s nodes and start building the cluster. Still a work in progress but it’s super nice you are doing exactly what I’m working on and your videos are extremely useful

  • @Jims-Garage

    @Jims-Garage

    5 ай бұрын

    Thanks, I'm going to focus on packer later 🙂 good luck

  • @DMBrownlee

    @DMBrownlee

    5 ай бұрын

    I was using Packer to build VM templates in Proxmox that I could then clone with the Telmate Terraform plugin. I started looking into using cloud-init as an alternative just before Proxmox 8.1 broke Telmate for me. One of the cloud-init configuration options is adding your own keys (and admin username, if you like) so there is no need to use packer to build your own image, just to insert keys. I'm now using the stock cloud-init image from Debian for my project and I have stopped using Packer for now. I've seen examples of Terraform managing the generation of key pairs which would be a good prerequisite to add to this video.

  • @ijustwanttoeatcookie

    @ijustwanttoeatcookie

    5 ай бұрын

    @@DMBrownlee you're right, but I'm not only inserting keys! I use an ubuntu server 22.04 cloud init image and I create 2 users. My own admin user and the ansible user. I generate the ansible priv/pub keypair while building the image and it then stores the public key on my nfs share. I do this so that I can inject the public key in the image for my k3s nodes. In addition, I also install some software packages and set-up some repos with packer. for my Ansible master server I install: ansible, packer, terraform and kubectl so that I have a dedicated image that can fully control and configure my k3s cluster. For my K3s nodes I intent to do the same by baking k3sup into the image using packer. Will just use ansible to basically connect the nodes together as a cluster. I also don't use the telmate provider for terraform anymore since the update cycle is way to slow. Instead I opted for thegameprofi's provider for proxmox. It is similar to bpg's provider and it's rock solid so far on proxmox 8.1

  • @DMBrownlee

    @DMBrownlee

    5 ай бұрын

    @@ijustwanttoeatcookie cloud-init also lets you configure the default user as well. If the default user is your configuration management user, you can use ansible post-creation to manage the rest of the system configuration including adding additional admin users if needed. Still not seeing anything that would require building an image with Packer. Not saying Packer is bad. I used it for years and it works great. But it only makes sense if you cannot use the vendor's stock cloud-init.

  • @ijustwanttoeatcookie

    @ijustwanttoeatcookie

    5 ай бұрын

    ​@@DMBrownlee you're right I just realised I can perfectly well use ansible to copy its public ssh key to all my k3s nodes. Just adjusted my code :) Bit offtopic here on the homelab, but wouldn't you say that having a golden image for specific server types (web servers, data base servers etc) can be quite conveniant for enterprise environments? Ansible can provision with the required packages for each server as soon as the newly created vm's / containers come up of course. However, having that stuff baked into the image can save quite some time and bandwith when deploying at scale or am I overlooking something?

  • @imtiazali1628
    @imtiazali16285 ай бұрын

    Thanks James. I tried telmate but it did not worked. I using now bpg provider and test the same. It will provision the required number of machines and create the host file for anisble then ansible will provision the k3s cluster with metallb. testing now longhorn and other apps provisioning. Thanks again for such informative videos

  • @tonychia2227
    @tonychia22274 ай бұрын

    Opentu is the open source version of terraform now.

  • @JohnWeland
    @JohnWeland5 ай бұрын

    I hope that Talmate project gets legs under it and grows. It would be nice in a Proxmox cluster to be able to assign these VMs to various nodes in the deploy. eg three Proxmox nodes, this would then build a master and x workers per Proxmox node. I wonder if there is a provider for iDRAC7, could get real close to metal then. get three new dell servers, use Terraform to update them, provision disks, install Proxmox, setup the cluster, and CEPH; this run this to provision the VMs

  • @dromer1967
    @dromer19675 ай бұрын

    Thanks for this video! I struggled a bit to see the link between the cloud-init tempate I created in proxmox and how to specify that parameters of it in terraform so that the vm is properly created (my vm would never boot because the hdd definitions were wrong or something). That link is now much more clear to me so thanks very much for that! One think I was wondering however, why does your use of the plugin seems so complicated, building it yourself and all? Am I missing something there, perhaps because you are using a liniux(/wsl2) prompt and I am simply using the windows binaries? Because in main.tf I simply have this terraform { required_version = ">=1.7.2" required_providers { proxmox = { source = "telmate/proxmox" version = "3.0.1-rc1" } } } And this works fine without having to compile any binaries or plugins myself 😊

  • @luisenrique2138
    @luisenrique21382 ай бұрын

    Hi Jim's great work thanks! i followed this guide and work fine to me... so how i can add soft packages to vm deployed? i need add some package after vm deployed but no add in image template.

  • @andrewivanof2940
    @andrewivanof2940Ай бұрын

    Thanks Jim. This doesn't work anymore with cloudinit_cdrom_storage :( maybe you can release short update video with repo update.

  • @PCMagikHomeLab
    @PCMagikHomeLab5 ай бұрын

    And maybe the time packer from hasicorp?

  • @Patterner
    @Patterner5 ай бұрын

    to be fair, Terraform and Terrifying sound very similar.

  • @mattiavadala7870

    @mattiavadala7870

    5 ай бұрын

    the same for Ansible and Anxiety xD

  • @motionthings
    @motionthings5 ай бұрын

    Just an fyi. Terraform went closed source. The open source fork is called opentofu

  • @Jims-Garage

    @Jims-Garage

    5 ай бұрын

    Good to know, thanks for sharing

  • @andrewwarner3969
    @andrewwarner39695 ай бұрын

    I'm running the script exactly as you have shown in your video (with key variables changed of course) but some vms are not being created. For example, it will create master 1 and 3 but not 2. The Terraform process also never seems to finish even though the vms it has created are up. ??

  • @Jims-Garage

    @Jims-Garage

    5 ай бұрын

    I sporadically witnessed this issue. Ctrl c and rerun it, should complete

  • @iKn-ft2bc
    @iKn-ft2bc5 ай бұрын

    Jim, i wanted to do a school project with proxmox but im failing since months. I can get API Acess and Generate a testCloudinit but i cant remove it or manage it. i think i have problems getting the API keys. i know what they are for but i cant get it working. i dont really know how to get them theres something i didnt understood well. My school project is Automating proxmox with Terraform. then i can manage vms and containers- with this i want to get basicly an appstore running with little knowlage of linux and such for the end user. basicly a solution for 1 admin at a school to install many fancy things without reading much into them

  • @hyperprotagonist
    @hyperprotagonist5 ай бұрын

    Speaking for terraformation, if you moved out of the garage to, say, a loft studio… would you then be known as Jim’s Loft?

  • @Jims-Garage

    @Jims-Garage

    5 ай бұрын

    Jim's Cellar coming soon 🔜

  • @hyperprotagonist

    @hyperprotagonist

    5 ай бұрын

    @@Jims-Garage😂

  • @jhmc93
    @jhmc93Ай бұрын

    Do u have a link of that video where you created the clone machine (23.04-non-KVM), Good video

  • @Jims-Garage

    @Jims-Garage

    Ай бұрын

    No, but follow the cloud init video and just swap the image

  • @jhmc93

    @jhmc93

    Ай бұрын

    @@Jims-Garage did u make a cloud init vid?

  • @Jims-Garage

    @Jims-Garage

    Ай бұрын

    @@jhmc93 yes, part of the Kubernetes series

  • @MarkConstable
    @MarkConstable5 ай бұрын

    Wake me up when you have an OpenTofu tutorial available for LXC containers.

  • @thomaspetit4312
    @thomaspetit43125 ай бұрын

    There is a `proxmox_cloud_init_disk` resource available. Is there a specific reason you didn't use this for the tutorial?

  • @Jims-Garage

    @Jims-Garage

    5 ай бұрын

    Thanks, I wasn't aware of that from the documentation on the provider. I'll test it

  • @Pasukaru0
    @Pasukaru05 ай бұрын

    I don't know why my comments keep getting deleted, so I'll try once more. We also use terraform at work. Will you also make a video on Pulumi? It's open source and more flexible imho.

  • @Jims-Garage

    @Jims-Garage

    5 ай бұрын

    Thanks, if you're putting links it often filters it. I will take a look, appreciate the feedback.

  • @csd4ni3l
    @csd4ni3l5 ай бұрын

    Didnt you leak your api tokens at 9:00 and a bit before when viewing providers config?

  • @Jims-Garage

    @Jims-Garage

    5 ай бұрын

    I did :) They're also in the example config on GitHub. Takes about 10s to swap them.

  • @niceitguy2444

    @niceitguy2444

    5 ай бұрын

    @@Jims-Garage still you should consider using terraform variables for specifying the value as sensitive to not be display/be ofuscated at runtime. And declare your variables in a separate tfvars file so you can retain the logic and manage multiple environments

  • @rubenkhachaturov3309
    @rubenkhachaturov33092 ай бұрын

    You need to define emulatessd = true in disks to get ssd=1

  • @Jims-Garage

    @Jims-Garage

    2 ай бұрын

    Thanks 👍

  • @rubenkhachaturov3309

    @rubenkhachaturov3309

    2 ай бұрын

    @@Jims-Garage I have posted my configuration example in your Disqord channel

  • @Jims-Garage

    @Jims-Garage

    2 ай бұрын

    @@rubenkhachaturov3309 thanks 👍

  • @ltonchis1245
    @ltonchis12455 ай бұрын

    I was thought this infrastructure as code could redeploy Proxmox OS With my configurations And vm's and containers

  • @chfmrf9605
    @chfmrf96053 ай бұрын

    Great video!

  • @Jims-Garage

    @Jims-Garage

    3 ай бұрын

    Thanks

  • @1988marksie
    @1988marksieАй бұрын

    I find the bpg proxmox provider is much better personally

  • @ernestoditerribile
    @ernestoditerribile5 ай бұрын

    Why haven't you switched over to VSCodium yet?

  • @Jims-Garage

    @Jims-Garage

    5 ай бұрын

    It's a great project. I feel there's little privacy win when I'm still using Windows.

  • @AljoschaKrui
    @AljoschaKrui5 ай бұрын

    You can Just as easy use ansible playbooks to spin up multiple vm's.

  • @javisartdesign
    @javisartdesign5 ай бұрын

    Terraform bad OpenTofu cool!

  • @-rm-rf

    @-rm-rf

    5 ай бұрын

    Tofu +1 😂

  • @manuelschmidt570

    @manuelschmidt570

    5 ай бұрын

    Very simpel view on the matter, but hey we are on the Internet so what do I expect. 😂

  • @ernestoditerribile
    @ernestoditerribile5 ай бұрын

    I wanted to visit your website, but I'm blocked by your cloud flare.

  • @Jims-Garage

    @Jims-Garage

    5 ай бұрын

    Thanks, but I don't have one.

Келесі