Terraform Remote State Backend using S3 and DynamoDB
Terraform can store state remotely in S3 and lock that state with DynamoDB. In this Terraform tutorial video, I am going to explain how you can configure remote state on S3 backend and enable state locking with Dynamo DB table.
#terraform
#s3bucket
#remotestate #devops
Connect with me on LinkedIn: / nasiullha-chaudhari-5a...
Timestamps -
1. Intro 0:00 - 0:15
2. Benefits of Remote State 0:16 - 0:43
3. Create S3 bucket for storing State file 01:07 - 06:00
4. Create a DynamoDB table for state locking 06:00 - 08:00
5. Apply changes to create s3 and dynamo DB 08:10 - 09:05
6. Add the backend in terraform block 09:05 - 10:45
7. Run the commands and migrate the state 10:46 - 12:40
8. Test the s3 state file 12:41- 14:30
9. Remote state to local state 14:35
Terraform Remote state documentation: developer.hashicorp.com/terra...
S3 resource: registry.terraform.io/provide...
Dynamodb resource: registry.terraform.io/provide...
Terraform stores its state in the file terraform.tfstate in local filesystem. This works well for personal projects, but working with Terraform in a team, use of a local file makes Terraform usage complicated because each user must make sure they always have the latest state data before running Terraform and make sure that nobody else runs Terraform at the same time.
The best way to do this is by running Terraform in a remote environment with shared access to state. Remote state solves those challenges. Remote state is simply storing that state file remotely, rather than on your local filesystem. With a single state file stored remotely, teams can ensure they always have the most up to date state file.
Terraform Remote State Storage Options
Terraform supports storing state in
Amazon S3
Azure Blob Storage
Google Cloud Storage
Terraform Cloud
HashiCorp Consul
With Remote state configuration all the state is stored in backends like s3 which is considered as a good practice.
Benefits of using Terraform Remote State
Shared Storage: Remote state (Backend) allow each of your team members to access same Terraform state files to manage infrastructure.
Locking: With fully-featured remote backends, Terraform can lock the state file while changes are being made. This ensures all changes are captured, even if concurrent changes are being attempted against the same state.
Without locking, if two team members are running Terraform at the same time, you may run into race conditions as multiple Terraform processes make concurrent updates to the state files, leading to conflicts, data loss, and state file corruption.
Versioning: Some backends support versioning. This maintains versions of your Terraform state files allowing you to download an old version if needed.
Encryption: Many backends support encryption of the state file both in transit and at rest.
Security: A local state file save the content in plain text. It is very common to have secrets or sensitive data in the state, so local state files are insecure
Remote operations: Some backends allow to manage operations remotely (Terraform plan and apply execution). You don’t need to use terraform on your system to apply the changes. You could either trigger it from a Web UI, API call or CLI tool.
Less Manual Errors: Using a local state file as a shared storage, manually sync the changes could cause someone forget to sync the state file. Remote state will always sync the state automatically whenever it change.
Also Checkout my other videos
Terraform playlist: • How to create Amazon E...
How to become DevOps engineer without any experience: • Fresher to Remote DevO...
Important Linux Commands : • Linux commands for Clo...
How I became an AWS Community Builder: • AWS Community Builders...
How to become Cloud Engineer: kzread.infoJShyUFJK0-...
@Cloud Champ is a youtube channel I started to record my learning and now is helping a lot of cloud and devops engineer. If you need any help you can always drop a comment or reach out to me.
Tags:
terraform remote state
terraform backend s3
terraform state backend s3 bucket
terraform remote state s3 example
terraform state in s3 bucket
terraform remote state
terraform remote state s3 example
terraform state in s3 bucket
terraform remote state azure
terraform state locking
terraform state file s3
terraform remote state file
terraform backend s3 example
terraform state backend s3
terraform migrate state to s3
terraform state s3 bucket
terraform state s3 dynamodb
terraform tfstate s3
terraform tutorial
techworld with nana terraform
terraform aws
terraform state management s3
state remote s3
terraform state backend s3
terraform migrate state to s3
terraform state s3 bucket
terraform state s3 dynamodb
terraform tfstate s3
Пікірлер: 46
Thank you for this. saved me hours of tutorials
@cloudchamp
Жыл бұрын
You are welcome 🤗 Please share and subscribe 🙏
You are a life saver, really well explained was completing a lab could set up the EC2 instances and use variables fine. Configuring the S3 backend with an updated version of Terraform was too hard to do alone. Thank you 🙏
@cloudchamp
10 ай бұрын
Happy to help ☺️ Also checkout the terraform playlist if that help
You are lifesaver!! Thanks for this project 😊
@cloudchamp
10 ай бұрын
You are welcome 🤗
Thanks CC...much appreciated. Great explanation.
@cloudchamp
5 ай бұрын
You are welcome
Thank you cery much, this video is very helpful
@cloudchamp
Жыл бұрын
Thanks! Happy to help ☺️
Wow this is so accurate!! 👍
@cloudchamp
Жыл бұрын
😊 thank you
Exactly what I wanted
@cloudchamp
Жыл бұрын
👍🏻👍🏻
Thanks for this video
@cloudchamp
Жыл бұрын
Your welcome 🙏
Nice tutorial!
@cloudchamp
Жыл бұрын
Thank you! Cheers!
Perfecd!!
👏👏
👍👍👍👍👍
Awesome
@cloudchamp
Жыл бұрын
Thank you Jhanvi ☺️🤝
Hi. Thanks for your wonderful videos. I currently have multiple projects and I’m using one S3 bucket for my state backend. These multiple project state files are separated by paths in the S3 bucket. Now, I want to implement DynamoDB state file lock. Since I have multiple projects, can I use only one DynamoDB table, like I used one S3 bucket for my state file backend? If the answer is yes, how does DynamoDB separate and isolate the locks for each project in this table? Or do I have to create multiple DynamoDB table for all my projects? Your guidance and thoughts on this would be appreciated.
hi can you explaination about given path. any condition in giving the path
I bro I can't be able to push the code to GitHub coz my file exceed 100 mb .. so what to do I have tried installing lfs also .. in my centos AWS instance
Your champ
I have try to done that but in my side o am getting so many errors like missing lock items, operation error DynamoDB: PutItein and not able to exesting s3 bucket but i have to init the script
Hey Champ, how do I store multiple state files in s3?
Hi, Thanks for the video. I followed the same steps and ran but it doesn't show "Acquiring state lock" when i run "terraform plan" or terraform apply" command. Please help on this.
@cloudchamp
Жыл бұрын
You can run the terraform plan command with force unlock parameter : terraform plan -lock=false
this video is very helpful. I have one doubt "AES256" is it a name or what? if it is a name can we give whatever as name right
@viral_shortsviralshorts
10 ай бұрын
No that’s encryption algorithm I think
Thanks for the tutorial, I was able to follow the steps and the terraform.tfstate file was on s3 bucket, and it the same time i have the terraform.tfstate file locallly on my computer inside the .terraform file, and if I tried deleting locally, it will download it again after running terraform init, is this normal? Can you explain if I am doing anything wrong, thanks
@cloudchamp
11 ай бұрын
Make sure you have backend block with s3 and dynamodb defined and then run terraform init to migrate state from local to s3 👍🏻 Feel free to ask if need help
Hello, How to make use of this state file while working in a team, do you have to pull the remote s3 state file to local ? ie, if any other user in the team wanted to know what resources have already been created, how do you make use of this remote s3.
@cloudchamp
8 ай бұрын
This state file is stored on S3 but can be pulled by anyone in the team, so it’s centrally accessible to all and lock let’s you make change one at a time to avoid state corruption. So everyone can work collaboratively 🤝
how to lock the tf.state in AWS S3 so that other cant write to it?
@cloudchamp
3 ай бұрын
Using dynamo db terraform lock
@nelsondario4762
3 ай бұрын
hello@@cloudchamp- where in your video I can find how to lock the tf.state so others cant write to it? thanks
@Dunkirk_3
2 ай бұрын
@@nelsondario4762the video addressed about pushing state file to s3 and Dynamodb is used just to lock parallel execution. It's one at time.
Hi I am getting error for backend Successfully configured the backend "s3"! Terraform will automatically use this backend unless the backend configuration changes. Error refreshing state: AccessDenied: Access Denied status code: 403, request id: 13YNJNZ963P0G2PC, host id: BesL3R7isQNbpgHpncUwj1Eg9kioyJLxo4Gxw0fHCtKZzqM7vrKlT2Dw+ijf0WNcwjYBJLfOWGE= Any clue
@cloudchamp
Жыл бұрын
Make sure you have the proper access and you have configured aws by inserting the right keys
what is your Age bro?
@cloudchamp
Жыл бұрын
22
@anilbhuvan1116
Жыл бұрын
@Cloud Champ cool bro, I'm 21, love your videos.