Terraform Remote State Backend using S3 and DynamoDB

Terraform can store state remotely in S3 and lock that state with DynamoDB. In this Terraform tutorial video, I am going to explain how you can configure remote state on S3 backend and enable state locking with Dynamo DB table.
#terraform
#s3bucket
#remotestate #devops
Connect with me on LinkedIn: / nasiullha-chaudhari-5a...
Timestamps -
1. Intro 0:00 - 0:15
2. Benefits of Remote State 0:16 - 0:43
3. Create S3 bucket for storing State file 01:07 - 06:00
4. Create a DynamoDB table for state locking 06:00 - 08:00
5. Apply changes to create s3 and dynamo DB 08:10 - 09:05
6. Add the backend in terraform block 09:05 - 10:45
7. Run the commands and migrate the state 10:46 - 12:40
8. Test the s3 state file 12:41- 14:30
9. Remote state to local state 14:35
Terraform Remote state documentation: developer.hashicorp.com/terra...
S3 resource: registry.terraform.io/provide...
Dynamodb resource: registry.terraform.io/provide...
Terraform stores its state in the file terraform.tfstate in local filesystem. This works well for personal projects, but working with Terraform in a team, use of a local file makes Terraform usage complicated because each user must make sure they always have the latest state data before running Terraform and make sure that nobody else runs Terraform at the same time.
The best way to do this is by running Terraform in a remote environment with shared access to state. Remote state solves those challenges. Remote state is simply storing that state file remotely, rather than on your local filesystem. With a single state file stored remotely, teams can ensure they always have the most up to date state file.
Terraform Remote State Storage Options
Terraform supports storing state in
Amazon S3
Azure Blob Storage
Google Cloud Storage
Terraform Cloud
HashiCorp Consul
With Remote state configuration all the state is stored in backends like s3 which is considered as a good practice.
Benefits of using Terraform Remote State
Shared Storage: Remote state (Backend) allow each of your team members to access same Terraform state files to manage infrastructure.
Locking: With fully-featured remote backends, Terraform can lock the state file while changes are being made. This ensures all changes are captured, even if concurrent changes are being attempted against the same state.
Without locking, if two team members are running Terraform at the same time, you may run into race conditions as multiple Terraform processes make concurrent updates to the state files, leading to conflicts, data loss, and state file corruption.
Versioning: Some backends support versioning. This maintains versions of your Terraform state files allowing you to download an old version if needed.
Encryption: Many backends support encryption of the state file both in transit and at rest.
Security: A local state file save the content in plain text. It is very common to have secrets or sensitive data in the state, so local state files are insecure
Remote operations: Some backends allow to manage operations remotely (Terraform plan and apply execution). You don’t need to use terraform on your system to apply the changes. You could either trigger it from a Web UI, API call or CLI tool.
Less Manual Errors: Using a local state file as a shared storage, manually sync the changes could cause someone forget to sync the state file. Remote state will always sync the state automatically whenever it change.
Also Checkout my other videos
Terraform playlist: • How to create Amazon E...
How to become DevOps engineer without any experience: • Fresher to Remote DevO...
Important Linux Commands : • Linux commands for Clo...
How I became an AWS Community Builder: • AWS Community Builders...
How to become Cloud Engineer: kzread.infoJShyUFJK0-...
@Cloud Champ is a youtube channel I started to record my learning and now is helping a lot of cloud and devops engineer. If you need any help you can always drop a comment or reach out to me.
Tags:
terraform remote state
terraform backend s3
terraform state backend s3 bucket
terraform remote state s3 example
terraform state in s3 bucket
terraform remote state
terraform remote state s3 example
terraform state in s3 bucket
terraform remote state azure
terraform state locking
terraform state file s3
terraform remote state file
terraform backend s3 example
terraform state backend s3
terraform migrate state to s3
terraform state s3 bucket
terraform state s3 dynamodb
terraform tfstate s3
terraform tutorial
techworld with nana terraform
terraform aws
terraform state management s3
state remote s3
terraform state backend s3
terraform migrate state to s3
terraform state s3 bucket
terraform state s3 dynamodb
terraform tfstate s3

Пікірлер: 46

  • @kwabenasteve9326
    @kwabenasteve9326 Жыл бұрын

    Thank you for this. saved me hours of tutorials

  • @cloudchamp

    @cloudchamp

    Жыл бұрын

    You are welcome 🤗 Please share and subscribe 🙏

  • @kyronne1
    @kyronne111 ай бұрын

    You are a life saver, really well explained was completing a lab could set up the EC2 instances and use variables fine. Configuring the S3 backend with an updated version of Terraform was too hard to do alone. Thank you 🙏

  • @cloudchamp

    @cloudchamp

    10 ай бұрын

    Happy to help ☺️ Also checkout the terraform playlist if that help

  • @viral_shortsviralshorts
    @viral_shortsviralshorts10 ай бұрын

    You are lifesaver!! Thanks for this project 😊

  • @cloudchamp

    @cloudchamp

    10 ай бұрын

    You are welcome 🤗

  • @practicaldeen5333
    @practicaldeen53335 ай бұрын

    Thanks CC...much appreciated. Great explanation.

  • @cloudchamp

    @cloudchamp

    5 ай бұрын

    You are welcome

  • @RolandTutorials
    @RolandTutorials Жыл бұрын

    Thank you cery much, this video is very helpful

  • @cloudchamp

    @cloudchamp

    Жыл бұрын

    Thanks! Happy to help ☺️

  • @gamingchamp3070
    @gamingchamp3070 Жыл бұрын

    Wow this is so accurate!! 👍

  • @cloudchamp

    @cloudchamp

    Жыл бұрын

    😊 thank you

  • @gamingchamp3070
    @gamingchamp3070 Жыл бұрын

    Exactly what I wanted

  • @cloudchamp

    @cloudchamp

    Жыл бұрын

    👍🏻👍🏻

  • @nabi140
    @nabi140 Жыл бұрын

    Thanks for this video

  • @cloudchamp

    @cloudchamp

    Жыл бұрын

    Your welcome 🙏

  • @furaofeliz3180
    @furaofeliz3180 Жыл бұрын

    Nice tutorial!

  • @cloudchamp

    @cloudchamp

    Жыл бұрын

    Thank you! Cheers!

  • @blazedquote5829
    @blazedquote5829 Жыл бұрын

    Perfecd!!

  • @nasichaudhari3966
    @nasichaudhari3966 Жыл бұрын

    👏👏

  • @kingalex1045
    @kingalex1045 Жыл бұрын

    👍👍👍👍👍

  • @jhanviuday9289
    @jhanviuday9289 Жыл бұрын

    Awesome

  • @cloudchamp

    @cloudchamp

    Жыл бұрын

    Thank you Jhanvi ☺️🤝

  • @adedayoademefun8895
    @adedayoademefun889521 күн бұрын

    Hi. Thanks for your wonderful videos. I currently have multiple projects and I’m using one S3 bucket for my state backend. These multiple project state files are separated by paths in the S3 bucket. Now, I want to implement DynamoDB state file lock. Since I have multiple projects, can I use only one DynamoDB table, like I used one S3 bucket for my state file backend? If the answer is yes, how does DynamoDB separate and isolate the locks for each project in this table? Or do I have to create multiple DynamoDB table for all my projects? Your guidance and thoughts on this would be appreciated.

  • @kavyas5748
    @kavyas57486 ай бұрын

    hi can you explaination about given path. any condition in giving the path

  • @VijethKalavarkar
    @VijethKalavarkar13 күн бұрын

    I bro I can't be able to push the code to GitHub coz my file exceed 100 mb .. so what to do I have tried installing lfs also .. in my centos AWS instance

  • @gudamahendar6992
    @gudamahendar69922 ай бұрын

    Your champ

  • @maheshpansare7508
    @maheshpansare75088 ай бұрын

    I have try to done that but in my side o am getting so many errors like missing lock items, operation error DynamoDB: PutItein and not able to exesting s3 bucket but i have to init the script

  • @martinthong125
    @martinthong1255 ай бұрын

    Hey Champ, how do I store multiple state files in s3?

  • @sribalaje
    @sribalaje Жыл бұрын

    Hi, Thanks for the video. I followed the same steps and ran but it doesn't show "Acquiring state lock" when i run "terraform plan" or terraform apply" command. Please help on this.

  • @cloudchamp

    @cloudchamp

    Жыл бұрын

    You can run the terraform plan command with force unlock parameter : terraform plan -lock=false

  • @rakshirakshitha4198
    @rakshirakshitha419810 ай бұрын

    this video is very helpful. I have one doubt "AES256" is it a name or what? if it is a name can we give whatever as name right

  • @viral_shortsviralshorts

    @viral_shortsviralshorts

    10 ай бұрын

    No that’s encryption algorithm I think

  • @opeyemiojo1154
    @opeyemiojo115411 ай бұрын

    Thanks for the tutorial, I was able to follow the steps and the terraform.tfstate file was on s3 bucket, and it the same time i have the terraform.tfstate file locallly on my computer inside the .terraform file, and if I tried deleting locally, it will download it again after running terraform init, is this normal? Can you explain if I am doing anything wrong, thanks

  • @cloudchamp

    @cloudchamp

    11 ай бұрын

    Make sure you have backend block with s3 and dynamodb defined and then run terraform init to migrate state from local to s3 👍🏻 Feel free to ask if need help

  • @Emil_Benny
    @Emil_Benny8 ай бұрын

    Hello, How to make use of this state file while working in a team, do you have to pull the remote s3 state file to local ? ie, if any other user in the team wanted to know what resources have already been created, how do you make use of this remote s3.

  • @cloudchamp

    @cloudchamp

    8 ай бұрын

    This state file is stored on S3 but can be pulled by anyone in the team, so it’s centrally accessible to all and lock let’s you make change one at a time to avoid state corruption. So everyone can work collaboratively 🤝

  • @nelsondario4762
    @nelsondario47623 ай бұрын

    how to lock the tf.state in AWS S3 so that other cant write to it?

  • @cloudchamp

    @cloudchamp

    3 ай бұрын

    Using dynamo db terraform lock

  • @nelsondario4762

    @nelsondario4762

    3 ай бұрын

    hello@@cloudchamp- where in your video I can find how to lock the tf.state so others cant write to it? thanks

  • @Dunkirk_3

    @Dunkirk_3

    2 ай бұрын

    ​@@nelsondario4762the video addressed about pushing state file to s3 and Dynamodb is used just to lock parallel execution. It's one at time.

  • @pawansolanki20095
    @pawansolanki20095 Жыл бұрын

    Hi I am getting error for backend Successfully configured the backend "s3"! Terraform will automatically use this backend unless the backend configuration changes. Error refreshing state: AccessDenied: Access Denied status code: 403, request id: 13YNJNZ963P0G2PC, host id: BesL3R7isQNbpgHpncUwj1Eg9kioyJLxo4Gxw0fHCtKZzqM7vrKlT2Dw+ijf0WNcwjYBJLfOWGE= Any clue

  • @cloudchamp

    @cloudchamp

    Жыл бұрын

    Make sure you have the proper access and you have configured aws by inserting the right keys

  • @anilbhuvan1116
    @anilbhuvan1116 Жыл бұрын

    what is your Age bro?

  • @cloudchamp

    @cloudchamp

    Жыл бұрын

    22

  • @anilbhuvan1116

    @anilbhuvan1116

    Жыл бұрын

    @Cloud Champ cool bro, I'm 21, love your videos.