Terragrunt Tutorial: Create VPC, EKS from Scratch!
Ғылым және технология
🔴 - To support my channel, I’d like to offer Mentorship/On-the-Job Support/Consulting - me@antonputra.com
▬▬▬▬▬ Experience & Location 💼 ▬▬▬▬▬
► I’m a Senior Software Engineer at Juniper Networks (12+ years of experience)
► Located in San Francisco Bay Area, CA (US citizen)
▬▬▬▬▬▬ Connect with me 👋 ▬▬▬▬▬▬
► LinkedIn: / anton-putra
► Twitter/X: / antonvputra
► GitHub: github.com/antonputra
► Email: me@antonputra.com
▬▬▬▬▬▬ Related videos 👨🏫 ▬▬▬▬▬▬
👉 [Playlist] Kubernetes Tutorials: • Kubernetes Tutorials
👉 [Playlist] Terraform Tutorials: • Terraform Tutorials fo...
👉 [Playlist] Network Tutorials: • Network Tutorials
👉 [Playlist] Apache Kafka Tutorials: • Apache Kafka Tutorials
👉 [Playlist] Performance Benchmarks: • Performance Benchmarks
👉 [Playlist] Database Tutorials: • Database Tutorials
▬▬▬▬▬▬▬ Timestamps ⏰ ▬▬▬▬▬▬▬
0:00 Intro
02:00 Develop Terraform Code to Create VPC
13:15 Convert VPC Terraform Code into Module
23:46 Use Terragrunt to Create VPC
31:15 Create EKS and Kubernetes Addons Modules
49:54 Production Ready Setup
▬▬▬▬▬▬▬ Source Code 📚 ▬▬▬▬▬▬▬
► GitHub: github.com/antonputra/tutoria...
#terraform #devops #cloud
Пікірлер: 124
🔴 - To support my channel, I’d like to offer Mentorship/On-the-Job Support/Consulting - me@antonputra.com
People who aren't DevOps engineers might not fully understand the tremendous value you share in your videos. In this video, you intuitively and comprehensively teach Terragrunt, an amazing tool used by some of the best DevOps teams in the industry, and not only that, you use it as an instrument to easily teach advanced AWS topics. If that wasn't enough, you cover the entire scope of infrastructure environments; development, staging, and production. What more can I say other than it's absolutely perfect?
@AntonPutra
11 ай бұрын
Wow thanks! I appreciate it!
I've been on the software development (CI) side for 30+ years and only recently had the opportunity to contribute to the CD side of a new startup. Thanks for sharing these great tutorials! Helped me a lot while getting started on my DevOps journey.
@AntonPutra
9 ай бұрын
Thank you so much!!
i dont think you understand how much your explanation and combining so many different elements of devops into one video has helped me so much. Like, exactly what i was looking for. thank you so much!
@AntonPutra
10 ай бұрын
Thank you! Glad to hear that!
Fantastic video! I truly appreciated the concise and fast-paced explanation. I've liked and subscribed. Thank you!
@AntonPutra
8 ай бұрын
Thank Sergey!
Anton you are such a "go-to" guy for this stuff. Keep up the amazing work you are doing for the community. We appreciate you so much. 👊
@AntonPutra
Жыл бұрын
Thanks Robert!
@SuperAleksandar96
10 ай бұрын
@@AntonPutra 10/10 videos Thanks
Thank you 🙏 for sharing this i was searching about this yesterday and you uploaded this Anton you are great.
@AntonPutra
Жыл бұрын
My pleasure!
Awesome video for using Terraform Mudule based with Terragrunt and store state in aws S3. I followed all the way and now I am maintaining this infrastructure for my production. Waiting for next video related with this video sir.
@AntonPutra
Жыл бұрын
Thanks you!
Really useful having all this information for a beginner. I was tearing my hair out going from blog post to blog post.
@AntonPutra
5 ай бұрын
thanks Joshua
Thanks a lot for a tutorial that covers Terragrunt from the beginning. Спасибо!
@AntonPutra
11 ай бұрын
pojaluysta =)
Thank you so much Anton! Amazing knowledge!
@AntonPutra
5 ай бұрын
Thank you, Tiago!
Terragrunt rocks! We use it for managing complex multi-client infra. Bunch of includes, dependencies, etc :) terraform without terragrunt usable only for very simple projects.
@AntonPutra
Жыл бұрын
agree
This is a fantastic tutorial. Extremely helpful to me.
@AntonPutra
Жыл бұрын
Thank you so much for the support!
Fantastic video. Thank you so much!
@AntonPutra
8 ай бұрын
Thank you Chris!
Top quality, thanks for sharing 🎉
@AntonPutra
Жыл бұрын
Thank you Andres!
Anton, you are such a great DevOps Engineer ...keep it up, my friend!
@AntonPutra
10 ай бұрын
Thank you!
Nice video and perfect speed. Spasibo
@AntonPutra
Жыл бұрын
Spasibo vam =)
спасибо, кучу времени сэкономил)
@AntonPutra
7 ай бұрын
pojaluysta :)
This channel is very util. Hello from Brazil.
@AntonPutra
Ай бұрын
❤️
thank you bro!
Great Tutorial, make sure to use the same instance size as in the video and NOT t.2 micro. My autoscaler wouldn't launch and I could not figure out why, now I know that it's the instance size. My guess would be that in order for autoscaler to work - instance size should not be less than t3a.xlarge. Anyway, thanks Anton, I learned a lot
@AntonPutra
10 ай бұрын
Cool, thanks for the tip
Excellent
@AntonPutra
Жыл бұрын
Thanks
@sujithsurendran7686
Жыл бұрын
@@AntonPutra can you make some videos on Kubernetes CRD and some automation on that.
@AntonPutra
Жыл бұрын
@@sujithsurendran7686 Operator is coming soon, but what do you mean by automation?
Hey Anton, amazing video. The build-up to the final .hcl file put it all together. Can you explain what the differences were between the live-v1/v2/v3/v4 folders and when doing this for a completely different environments, do we need to go through all those steps or can we go straight to v4
@AntonPutra
10 ай бұрын
Thanks! v1 is just raw terraform code v2 uses terraform modules v2 & v4 both terragrunt and different modules You can go straight to v4 if you use open source modules or build your own and go for v4
AMAZING!!!
@AntonPutra
5 ай бұрын
Thanks, Lukas!
great videos! is there tutorial how to add more kubernetes-addons like aws load balancer controllers, efs csi drivers etc. using this deployment method?
@AntonPutra
Жыл бұрын
Thanks, sure you just need to add all of your addons in this module - github.com/antonputra/tutorials/tree/main/lessons/160/infrastructure-modules/kubernetes-addons and create a specific variable such as enable_alb_controller.
Thanks!
@AntonPutra
6 ай бұрын
Thank you, Stephan! I appreciate your support!
Hi Anton Putra, Thank you for the excellent tutorial on setting up an EKS cluster! It was incredibly helpful and easy to follow. I wanted to ask for your guidance on adding Ubuntu 20.04 nodes to my EKS cluster. Could you provide some instructions or point me towards a resource that explains how to do this? Thanks in advance for your help and for the great content you create!
@AntonPutra
21 күн бұрын
Sure, you need to build your image using this official packer script - github.com/awslabs/amazon-eks-ami They are called self-managed nodes here is a reference - docs.aws.amazon.com/eks/latest/userguide/worker.html
Great tutorial, love it! What do you think of Terraspace?
@AntonPutra
11 ай бұрын
Thanks! I haven't used Terraspace yet.
55:29 It would be amazing to update the project to do all the terraform IAM setup (showcased at the end using the AWS console) using terragrunt as well.
@AntonPutra
Жыл бұрын
Thanks for the feedback
@AntonPutra in production environment, when you need to upgrade the eks version… will you do that via terraform ..? or thats where terraform ends its job. because i guess upgrading a live eks cluster with deployments is risky . Thanks
@AntonPutra
11 ай бұрын
Not really, EKS would first drain nodes and respect the constraints that you place. You need to make sure you have pod disruption budgets for all your apps to avoid any downtime. kubernetes.io/docs/tasks/administer-cluster/safely-drain-node/#configure-poddisruptionbudget
Amazing Video🌟 got a question, how could Github infra accessed private Github module without a token ?
@AntonPutra
Жыл бұрын
go to settings and add "deploy key"
Thanks for the video. very insightful.❤ quick query: How can we share values between staging and production using this architecture?
@AntonPutra
3 күн бұрын
you can use read_terragrunt_config function example: locals { common_vars = read_terragrunt_config(find_in_parent_folders("common.hcl")) } reference - terragrunt.gruntwork.io/docs/reference/built-in-functions/#read_terragrunt_config
Hi Anton great tutorial as usual, thanks a lot, but i wanted to ask, how do we handle eks upgrade with terraform , is it enough just by changing the cluster_version and eks will handle the rest, or do we need to do something to scale the cluster first, and do the drain and cordon etc. fact: Putra in my language means prince.
@AntonPutra
8 ай бұрын
Thank you! Based on my multi-year experience managing Kubernetes clusters in AWS and GCP, sometimes it can be as easy as updating the version and applying Terraform. In that case, Terraform will upgrade the control plane and then node pools one by one (drain, terminate, etc). In some cases, for example, when upgrading to 1.21 or 1.22, there were breaking changes in Kubernetes which forced us to create a brand-new clusters and migrate apps there due to the new and deprecated APIs.
hi Antonio, could you please elaborate why you preferred to use folders named by the env name (dev/staging) rather than the terraform workspaces?
@AntonPutra
2 ай бұрын
There's a significant limitation in how to isolate different environments when using Terraform workspaces. I have a dedicated section somewhere in this video where I explain exactly why you would choose folders rather than workspaces - kzread.info/dash/bejne/eZuFp5pucdSYqpc.html
I keep getting cyclic dependency error when I run "terragrunt run-all plan/apply " on dev folder. I downloaded "Anton Putra's" code and and tried running on it and it still gives same error. Can anyone help me. Also I am using local backend and modules.
hi thanks a lot for the video!! can we migrate existing terraform code running on AWS to Terragrunt is that easy to do?
@AntonPutra
10 ай бұрын
Yes, you can, but it's not easy since you need to import state for all components you already have.
Hi, do you recommend Terragunt for large scale projects? sometime it becomes hard to maintain.
@AntonPutra
Жыл бұрын
I do, especially for big companies with lots of environments, but keep it simple and beginner friendly.
Very Helpful for beginners like me. Appreciate the effort of posting these. I just have question, when I'm adding the Labels and Taints inside the node_groups, terragrunt doesn't seem to detect it. What should I been missing? I just wanted to add multiple taints and labels for each nodegroup. I haven't added sec group as well to the node group, that will might be same instance.
@AntonPutra
6 ай бұрын
Thanks! Well you would need to add taints to the module itself here - github.com/antonputra/tutorials/blob/main/lessons/160/infrastructure-modules/eks/3-nodes.tf#L26 Example - github.com/antonputra/tutorials/blob/2d1421a4c6b91e8ee22017f8252aee170586b711/lessons/152/terraform/7-nodes.tf#L98-L102 But instead of hardcoding use github.com/antonputra/tutorials/blob/main/lessons/160/infrastructure-modules/eks/3-nodes.tf#L2 variable
@haroldosalmazan
6 ай бұрын
@@AntonPutra Thank you, I was able to add in on the module. However, I'm still trying to figure out how to make an object(taint/labels) inside the node groups to not require an input/s. I have set labels and taints inside the module by using "each.value" but when I skip it on a node group, it fails saying "Unsuitable value for var.node_groups set using the TF_VAR_node_groups │environment variable: all map elements must have the same type." Still looking on what should I have to add to make this possible. Here what I add on module tho, "labels = each.value.labels" taint { key = each.value.taint.key value = each.value.taint.value effect = each.value.taint.effect }
Great video, I am managing a few clients but but with only 2 env, so don't see that much gain of Terragrunt, when initial setup and refactoring would be very complex. But information is very valueable. The ideo of using terraform modules like these addons is super, gonna start this from Monday :) Another thing that is still stopping me from using Terrafrunt is using Terraform cloud as backend, since i couldn't find good explaind example. Maybe that could be idea for next video? :)
@AntonPutra
Жыл бұрын
Thanks, well you can always create your own or try to use www.runatlantis.io
we can use terragrunt to wrap up for a bigger infrastructure on a same cloud provider It can be a library for all resources
@AntonPutra
Жыл бұрын
agree it's very helpful when you have lots of environments and components
How do you reference data blocks that we use to fetch from AWS like authentication credentials in terragrunt?
@AntonPutra
4 ай бұрын
I didn't get the question. In Terragrunt, we usually create all infrastructure using Terraform itself and use dependency references between different parts of Terragrunt. Data blocks are usually the exception, not the common practice.
i'm curious, is there a reason why the tf files were prefixed with numbers ? like the below 0-provider, 1-vpc and so on
@AntonPutra
5 ай бұрын
It's just for the tutorial to sort them in the IDE and show the logical order of how you would create infrastructure components.
Valeu!
@AntonPutra
22 күн бұрын
thank you!!
Do you have any git link to have the code please
Hello master can we do manage own security group instead of using the default one ? and if that so how can i add the security group to use with K8s ?
@AntonPutra
Жыл бұрын
It creates one for you. You can get id of the group using "cluster_security_group_id" output variable and add additional rules using the following resource "aws_security_group_rule".
@rohmersicat3429
Жыл бұрын
@@AntonPutra yes what i mean if i want to allow port 8080 how can i add it on vpc ?
@AntonPutra
Жыл бұрын
@@rohmersicat3429 I need to know your use case. For example if you your application (pod) needs access to RDS, you add EKS security groups as source in your configuration. Optionally you can setup Security groups for pods
Amazing!!!
@AntonPutra
Жыл бұрын
Thanks Ian! =)
Could you provide this same example but without using terragrunt? Just terraform?
@AntonPutra
11 ай бұрын
Sure, here for exampe - github.com/antonputra/tutorials/tree/main/lessons/167/terraform Also, I have terraform module version - github.com/antonputra/tutorials/tree/main/lessons/156
Hi, in 58:40 when i run terragrunt run-all plan command, it's not working. showing can't read resource with the name of eks
@AntonPutra
5 ай бұрын
Check if you used the same name for the cluster
thumb up and subscription strictly required!!!
@AntonPutra
Жыл бұрын
Thanks!
Hello, why is my pod not triggering for scale up of the worker nodes, `pod didn't trigger scale-up`.
@AntonPutra
5 ай бұрын
Best way to find out is to check autoscaler controller logs. Kubectl logs …
@AntonPutra
5 ай бұрын
Best way to find out is to check autoscaler controller logs. Kubectl logs …
how to know when should use Terragrunt instead Terraform?
@AntonPutra
Жыл бұрын
when you have more then 2 envs
Why you wrote your own module instead of using official VPC module ? is there a reason or just for teaching purposes
@AntonPutra
5 ай бұрын
In the long run it’s much easier to maintain your own resources. Less abstractions less dependencies. I’m talking about 1+ year. Less technical debt you acquire is better for you and your team.
I think git as a source is difficult to use specially if the organizations are using MFA auth for their git. But over all your video is awesome!
@AntonPutra
8 ай бұрын
I disagree. GitOps is a standard; even in Google SRE books, they specifically mention following the same git approach.
@kayoutube690
8 ай бұрын
You're right! But I'm referring only to jenkins. I never use GitOps but i would love to explore this tool.
@AntonPutra
8 ай бұрын
@@kayoutube690 Due to the recent fork of Terraform to OpenTF, I suggest waiting to see how all those "providers" will be adopted by OpenTF. Terragrunt will no longer be able to use Terraform and will switch to OpenTF because of licensing issues.
Thank you very much for this AMAZING video! If you're creating your cluster in a region other than 'us-east-1', it's necessary to modify the kubernetes-addons/1-cluster-autoscaler.yaml file to add the 'awsRegion' value to the Helm chart, as it defaults to searching for the node group in "us-east-1". Otherwise, you'll get the error "Node ip-xxx.xxxxx.compute.internal should not be processed by the cluster autoscaler (no node group config)"
11:42 that dev/vpc/dev/vpc/terraform.tfstate file doesn't make completely sense
Why you have no ".terraform" dir when init ?
@AntonPutra
Ай бұрын
i believe it's just hidden
Can you share the code
@AntonPutra
9 ай бұрын
Sure, you'll find link the video description
@xavierordonez2484
9 ай бұрын
@@AntonPutra please provide email to keep in touch. Some projects I have
Sorry to say, but you are going very fast and it becomes very hard to understand and have to watch the video repeatedly which is time wasting.But overall your videos are very helpful.
@AntonPutra
Жыл бұрын
Thanks for the feedback, trying to make them short and concise
If you change the region to something other than `us-east-1`, the autoscaler needs `awsRegion` to be set in `infastructure-modules/kubernetes-addons/1-cluster-autoscaler.tf`
@AntonPutra
Жыл бұрын
You can set it using this variable - github.com/antonputra/tutorials/blob/main/lessons/160/autoscaller.yaml#L66
@timorling5197
Жыл бұрын
@@AntonPutra How is that yaml file being consumed? Perhaps I needed to re-init terragrunt?
@timorling5197
Жыл бұрын
I figured it out, put the autoscaler.yaml into the `dev/kubernetes-addons/` directory and add the values like normal with `helm_release` `values = [ file("${path.module}/autoscaler.yaml") ]`