Create CloudFormation Stack (EC2 + RDS) & Deploy Docker App - Episode #7

Ғылым және технология

Learn how to create an application stack using AWS CloudFormation. Stack will consist of EC2 instance and RDS for MySQL.
Stack is defined using YAML template that contains 4 resources to be created on AWS,
1. EC2 Instance to run the application on
2. EC2 Security Group that defines the possible inbound ports on the EC2 instance
3. RDS Instance for MySQL database
4. RDS Security Group that defines the possible sources the db can be reached from
devteds.com/episodes/7-create...
Following are the steps to walk through,
1. Create a KeyPair on AWS account to be used for SSH access to EC2
2. Generate API keys and configure the AWS CLI on local machine
3. Create a simple stack using CloudFormation and this stack will consists of an EC2 instance and necessary security group for EC2 instance
4. Update the stack to install and configure docker on the EC2 instance
5. Update the stack to add RDS instance and necessary security group to the stack
6. Create Docker compose file to define the application to be deployed on EC2 instance with it’s database on RDS instance
7. Deploy the application on the EC2 instance (docker host) remotely from local
------------------------------------------------------------------------
Source code:
github.com/devteds/e7-cloudfo...
Docker Image:
hub.docker.com/r/devteds/rail...
------------------------------------------------------------------------
Related Courses & Episodes:
Course "Kubernetes on AWS EKS: Hands-on Get Started": devteds.com/k8s
Docker on Amazon ECS with Fargate using CloudFormation: • Docker on Amazon ECS F...
Multi-Stage Docker Build: How to Build Production-Ready Container Image: • Docker Multi-stage for...
Deploy Dockerized Web API application on Amazon EC2 with RDS:
• Deploy Dockerized Web ...
Episode #3 - APIs on Rails: Create a rails API-only application
• REST JSON APIs on Rail...
------------------------------------------------------------------------
Commands & README:
github.com/devteds/e7-cloudfo...
------------------------------------------------------------------------
Resources:
AWS CloudFormation: aws.amazon.com/cloudformation
AWS EC2: aws.amazon.com/ec2
AWS RDS: aws.amazon.com/rds
Docker Image: hub.docker.com/r/devteds/rail...
Configure Docker Engine: docs.docker.com/engine/admin/
Docker & TLS: docs.docker.com/engine/securi...
------------------------------------------------------------------------
Author:
Chandra Shettigar | Software Engineer & Architect | Cloud & Platform Engineering | Creator & Instructor at Devteds
devteds.com | / shettigarc | / shettigar
------------------------------------------------------------------------

Пікірлер: 92

  • @velunatarajan2649
    @velunatarajan26496 жыл бұрын

    Thanks for quick overview on CloudFormation....this helped me lot !!

  • @ChandraShettigar

    @ChandraShettigar

    6 жыл бұрын

    Thanks for the feedback Velu.

  • @GetOutOfHereStalkan
    @GetOutOfHereStalkan5 жыл бұрын

    This is shockingly well made.

  • @ChandraShettigar

    @ChandraShettigar

    4 жыл бұрын

    Thank you.

  • @christian-peterheimbach3674
    @christian-peterheimbach36745 жыл бұрын

    I really enjoyed that. Given how many painful tutorials are out there. You rock. Thanks.

  • @ChandraShettigar

    @ChandraShettigar

    5 жыл бұрын

    Thank you Christian 🙂

  • @dheerajtripathi6734
    @dheerajtripathi67346 жыл бұрын

    Just 20 minutes of this session and I now find it way more easier, thanks :)

  • @ChandraShettigar

    @ChandraShettigar

    6 жыл бұрын

    Thanks Dheeraj. I'm glad it was helpful.

  • @shivajigandi3040
    @shivajigandi30406 жыл бұрын

    I like the way you demonstrate this video.Good job

  • @ChandraShettigar

    @ChandraShettigar

    6 жыл бұрын

    Thanks Shivaji. I hope this was useful too.

  • @siddheshkerkar1
    @siddheshkerkar15 жыл бұрын

    Awesome!!! Precise and Clear..Thanks

  • @ChandraShettigar

    @ChandraShettigar

    5 жыл бұрын

    Thank you Siddhesh.

  • @santoshlohar604
    @santoshlohar6045 жыл бұрын

    Excellent way of explanation !! thanks for sharing this.

  • @ChandraShettigar

    @ChandraShettigar

    5 жыл бұрын

    Thank you Santosh.

  • @richajoy86
    @richajoy866 жыл бұрын

    Very informative and well explained. Thank you.

  • @ChandraShettigar

    @ChandraShettigar

    6 жыл бұрын

    Thanks Richard Joy. I'm glad it was helpful

  • @RihanPereira
    @RihanPereira5 жыл бұрын

    This is magnificient. I absorbed the know-how in under 25 min sitting. Thank you for your work, chandra!

  • @ChandraShettigar

    @ChandraShettigar

    4 жыл бұрын

    Rihana, Thank you. I am glad you found it helpful.

  • @anutpat
    @anutpat6 жыл бұрын

    Fantastic!!! Very useful information.

  • @ChandraShettigar

    @ChandraShettigar

    6 жыл бұрын

    Thanks for the feedback Ashutosh.

  • @pandianputube
    @pandianputube5 жыл бұрын

    Exquisite! Thanks for sharing.

  • @ChandraShettigar

    @ChandraShettigar

    4 жыл бұрын

    Pandian, Thank you.

  • @cloudshiksha3656
    @cloudshiksha36565 жыл бұрын

    excellent video man, thanks for sharing your knowledge

  • @ChandraShettigar

    @ChandraShettigar

    5 жыл бұрын

    Thank you Pankaj

  • @pulidovpe
    @pulidovpe4 жыл бұрын

    Thanks!! I was an excellent video for beginners.

  • @ChandraShettigar

    @ChandraShettigar

    4 жыл бұрын

    Thank you PulidoVPE

  • @adarshadarshsaxena
    @adarshadarshsaxena4 жыл бұрын

    Amazing work Chandra, generally it is tough to find intermediate material on any technology, appreciate your work please keep on rolling new videos, on devops.

  • @ChandraShettigar

    @ChandraShettigar

    4 жыл бұрын

    Thank you Adarsh 🙂

  • @SureshKumar-pv7gd
    @SureshKumar-pv7gd6 жыл бұрын

    Awesome. Too good explanation.

  • @ChandraShettigar

    @ChandraShettigar

    6 жыл бұрын

    Thank you Suresh.

  • @cybrainx72
    @cybrainx725 жыл бұрын

    Right speed for a typical human brain to consume without buffering

  • @ChandraShettigar

    @ChandraShettigar

    4 жыл бұрын

    Thank you Pluasok. I am still trying to adjust the pace so it is not too fast for many. I am glad you found the pace right.

  • @mikloslorinczi2679
    @mikloslorinczi26793 жыл бұрын

    Thanks Chandra, this tutorial is awesome!

  • @ChandraShettigar

    @ChandraShettigar

    3 жыл бұрын

    Miklos, you're so welcome!

  • @sagarsriva
    @sagarsriva5 жыл бұрын

    great presentation style, keep up the good work

  • @ChandraShettigar

    @ChandraShettigar

    5 жыл бұрын

    Thanks for the kind words, Sagar.

  • @ravic2617
    @ravic26176 жыл бұрын

    I like the pace of your tutorial. This would easily be 2 hours long if some of those other guys do it.

  • @ChandraShettigar

    @ChandraShettigar

    6 жыл бұрын

    Thank you Ravi.

  • @anilkommalapati6248
    @anilkommalapati62483 жыл бұрын

    amazing and brillaint .

  • @venkatk161
    @venkatk1616 жыл бұрын

    Excellent Chandra.

  • @ChandraShettigar

    @ChandraShettigar

    6 жыл бұрын

    Thanks Venkat.

  • @stepheys
    @stepheys7 жыл бұрын

    For Production, I would chose CloudFormation + ECS/Elastic Beanstalk + RDS. With ECS/Elastic Beanstalk, you can directly deploy your container, without having to worry about the docker host, set/change environment variables during deploy, easy to scale up/down

  • @ChandraShettigar

    @ChandraShettigar

    7 жыл бұрын

    Yes I agree - for production or even some non-production environments, I would pick CloudFormation + ECS + RDS. The purpose of this video was to demonstrate CloudFormation with some use-case for which I chose Docker installation. I was expecting a comment and feedback like this :) I am thinking to make a few videos around "CloudFormation + ECS + AutoScale" which I think might be a series of short videos or a course. Thanks!

  • @linux4kakkar
    @linux4kakkar5 жыл бұрын

    really super one!!

  • @ChandraShettigar

    @ChandraShettigar

    5 жыл бұрын

    Thank you.

  • @maheezasharma805
    @maheezasharma8056 жыл бұрын

    Awesome , good job chandra

  • @ChandraShettigar

    @ChandraShettigar

    6 жыл бұрын

    Thank you Maheeza Sharma

  • @sarfarazshaikh
    @sarfarazshaikh5 жыл бұрын

    Excellent, Please create more videos on Cloudformation using cli.

  • @ChandraShettigar

    @ChandraShettigar

    5 жыл бұрын

    Thank you Sarfaraz. Please check Episode #9 (kzread.info/dash/bejne/eaZm27aMo7m5oco.html) which is also a CloudFormation CLI but for ECS stack.

  • @rajdeeprandhawa1
    @rajdeeprandhawa16 жыл бұрын

    Very good presentation Chandra - very simple and elegant

  • @ChandraShettigar

    @ChandraShettigar

    6 жыл бұрын

    Thank you rajdeeprandhawa1

  • @aleskalex
    @aleskalex3 жыл бұрын

    Excelent work Chandra! This was the introduction I needed. Many thanks! I'm interested in the enhancements you mentioned about the scripting section (UserData). What would be the options within CloudFormation to take this part to the next level? Would you recommend any other solutions that might be easy to integrate/learn with CloudFormation? Thanks again for all your work!

  • @halyph
    @halyph6 жыл бұрын

    well done.

  • @ChandraShettigar

    @ChandraShettigar

    6 жыл бұрын

    Thank you Orest Ivasiv.

  • @stepheys
    @stepheys7 жыл бұрын

    I like the way you edit your videos, which makes it crisp, clear and concise. If I may suggest, 3:47-4:00 you should mask your AWS secrets as anyone can use it to hack your AWS account (hope you deleted the IAM user used in this video :) )

  • @ChandraShettigar

    @ChandraShettigar

    7 жыл бұрын

    Thanks for the feedback and suggestions @Bennette Stephen John. I thought of masking the AWS secrets but I chose to clean up all the resources and api keys etc after finishing up the video. In some cases I think masking will still be necessary. Thanks!

  • @ianngacha3594
    @ianngacha35942 жыл бұрын

    Thanks for the tutorial, during the last step connecting with docker, it says the target machine refused the connection

  • @joshuatrampier4355
    @joshuatrampier43556 жыл бұрын

    Thanks for tutorial. How to auto-populate RDS hostname in docker-compose yml file, instead of waiting and pasting?

  • @ChandraShettigar

    @ChandraShettigar

    6 жыл бұрын

    This is not a recommend approach for a real production stack but if you are just trying to automate the process of what is being done in this tutorial, one quick option would be to use CloudFormation Output variable. You can define CloudFormation output, value of which will be "!GetAtt DatabaseInstance.Endpoint.Address". Once the stack is created, you could write a small shell script to describe stack (using AWS CLI - aws cloudformation describe-stack ..), grep the output value and assign it to an env variable. In the compose file app.yml, read the database endpoint from that environment variable. I don't think you can avoid waiting but have the script wait until the RDS instance is ready before starting docker app. As I mentioned in this short tutorial, you could use CloudFormation itself to automate your stack (infra & app). Eg: To define and automate tracking of completion state of resources and then deploy the application. ECS is one good option where you don't deal with docker compose but define all the container services in CloudFormation template.

  • @vr5322
    @vr53226 жыл бұрын

    Great tutorial. Thanks Chanda. Could you please guide me how I can create a docker image and push it to ECR using cloudformation. Thanks in advance

  • @ChandraShettigar

    @ChandraShettigar

    6 жыл бұрын

    Thanks for the feedback Vikas. I am not exactly sure I understood your question. If you're trying to automate the process of creating docker image on every code commit and that you want that image be pushed over to the registry, that would be to create a simple or part of CI (Continuous Integration) pipeline. If that's what you're looking to build using CloudFormation and entirely on AWS, here is a high level of what goes into CloudFormation stack definition, 1. Create code repository resource (AWS::CodeCommit::Repository); You would use this as your source repo. 2. Create container image registry resource (AWS::ECR::Repository); This is where you would want the docker images pushed. 3. Create code build project resource (AWS::CodeBuild::Project), the source for this will be CodeCommit repo. Environment of CodeBuild project is where you would define the instructions to build docker image and push to registry. You will probably need to define buildspec.yml in the root of your code repo with necessary instructions to build & push. You may configure code build project to trigger on every code commit. I haven't used CodeBuild but if I were to create a CI on AWS, this is how I would go about. Now you can use Github code repository instead of CodeCommit in which case the Source definition of CodeBuild project will require some additional inputs. I hope this helps.

  • @shivananduppar1130
    @shivananduppar11304 жыл бұрын

    If you do more videos on devops for beginners it will be more helpful for us.thank you, I hope u ll

  • @ChandraShettigar

    @ChandraShettigar

    4 жыл бұрын

    Yes I will Shivanad. I haven't been able to publish in a while but I will try publishing the next one as soon as I can.

  • @shivananduppar1130

    @shivananduppar1130

    4 жыл бұрын

    @@ChandraShettigar I know Linux commands very well is it okay for devops, but I don't anything abt software development. Help me please

  • @SatishKumar-pk1ih
    @SatishKumar-pk1ih7 жыл бұрын

    awesome

  • @ChandraShettigar

    @ChandraShettigar

    7 жыл бұрын

    Thank you Satish.

  • @PrashantShetage
    @PrashantShetage5 жыл бұрын

    Hi. Your videos are very helpful. I am referring those for my daily tasks. Could you please make a video on CloudFormation for EMR with "Bootstrap Scripts(Postgres/MySql DB installation) & Step Execution(Spark Job)" Thanks

  • @ChandraShettigar

    @ChandraShettigar

    5 жыл бұрын

    Thank you Prashant. I appreciate your feedback and request. When time permits I will look into making a tutorial on EMR but I can't really promise now. If I can't make the video, I will try to post some code examples on github.com/devteds .

  • @masterprogrammingaz8920
    @masterprogrammingaz89206 жыл бұрын

    hi man, thanks for sharing. I have a question that what should I do/ follow/ learn if I want to be an DevOps? --thx a lot

  • @ChandraShettigar

    @ChandraShettigar

    6 жыл бұрын

    Thanks for asking my suggestion but I don't think I can be of much help providing guidance for a devops career path. But I will send you some information and links that might help.

  • @vijayb2357
    @vijayb23576 жыл бұрын

    Hi sir,,, am unable to install Jenkins via cloud formation... Plz help with script for installing Jenkins via cloud formation

  • @shivajigandi3040
    @shivajigandi30406 жыл бұрын

    I had one question in docker.How the developers work culture would be using containerization concept like check in and checkouts and how the developer environment is to be created on demand to test their own code etc

  • @ChandraShettigar

    @ChandraShettigar

    6 жыл бұрын

    In general containerized development environment helps the developers in several ways, 1. Consistent runtime, libraries, database versions - that means developers don't have to worry about installing, upgrading software versions and focus on application development. No tooling blockers and focus on getting the things done. 2. Consistent OS - this is a big plus. Developers can use any OS of their choice in the local environment. The basic OS & libraries that is needed for application is packaged in the container. This helps when you have a mix of developers that have preference for their primary OS. With the application dockerized, setting up environments on demand or the first time one a developer machine is all quick an easy. You probably need to write some scripts to seed data to the environment you spin up. If the application is not run as containers in production you can still containerize in development and CI environments to use the exact same versions of OS, libraries and softwares. That still helps. Sometimes it is hard to introduce containerization or docker in a dev team. For some it is some new tool to learn which takes back seat. But once introduced the team will notice improvement in productivity. I hope that answers your question.

  • @shivajigandi3040

    @shivajigandi3040

    6 жыл бұрын

    Thanks

  • @ddubs123
    @ddubs1236 жыл бұрын

    Using your config, I got an error while the task was creating the EC2 instance. "The specified instance type can only be used in a VPC. A subnet ID or network interface ID is required to carry out the request."

  • @ChandraShettigar

    @ChandraShettigar

    6 жыл бұрын

    Likely there is no default VPC. If there isn't one already, you might want to create a default vpc and try again. FYI - To create default on aws management console - under services, go to VPCs and from 'Actions' dropdown you should be able to create a new default vpc. Make sure to delete the failed stack before you try running 'aws cloudformation create ... " command. Hope this helps.

  • @kashishsharma3527
    @kashishsharma35275 жыл бұрын

    hi your video very nice and helpful, could you please help me to install Java 8 and maven and tomcat on the ec2 instance using cloud formation. thanks in advance please create a video or share any documents.

  • @ChandraShettigar

    @ChandraShettigar

    5 жыл бұрын

    Thank you Kashish. I am glad you found the videos useful. I don't do much Java but I have that in my list of videos to make. I can't be sure of timeline as when I can publish one on Java. In the meantime if I come across any tutorials or blogs on Java 8 + Maven + Cloudformation, I will send them to you.

  • @thimmareddy5089
    @thimmareddy50896 жыл бұрын

    How can add extra root volume to the instance when it is creating

  • @ChandraShettigar

    @ChandraShettigar

    6 жыл бұрын

    Thimma Reddy, Did you mean EBS block device mapping or create some directories to mount to the container? If it's about creating directories to mount to containers, you can add those commands in the UserData. If you're looking to add EBS kinda stuff, you might want to refer the CloudFormation EC2 reference documentation & examples. Following links might help - docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-instance.html docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-instance.html#cfn-ec2-instance-blockdevicemappings docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/quickref-ec2.html

  • @thimmareddy5089

    @thimmareddy5089

    6 жыл бұрын

    Chandra Shettigar I am creating an Beanstalk Env by using cloud formation, when it is launching instance it has to add 20 GB root volume to it

  • @helinafreesgi9135
    @helinafreesgi91356 жыл бұрын

    How do I access the terminal that you are talking about?

  • @rammohan4056

    @rammohan4056

    6 жыл бұрын

    use Putty. then SSH

  • @ChandraShettigar

    @ChandraShettigar

    6 жыл бұрын

    If you are using mac, you may use the "Terminal" app. The one I use on mac is called "iTerm". If you use windows 10, I think there are some good options such as "Bash on Ubuntu on Windows 10" and OpenSSH client. In the older windows, I don't know if there many options besides Putty or Cygwin.

  • @helinafreesgi9135

    @helinafreesgi9135

    6 жыл бұрын

    I keep receiving the below error at cloudformation and the stack gets rolled_back everytime: The specified instance type can only be used in a VPC. A subnet ID or network interface ID is required to carry out the request. Client Request Token:Console-CreateStack-fe29ceb9-82d9-4f4a-9f06-2a3402fd8999 The following resource(s) failed to create: [AppNode]. . Rollback requested by user.

  • @ChandraShettigar

    @ChandraShettigar

    6 жыл бұрын

    Sorry for the delayed reply. I hope you were able to resolve the issue by now. If not, please refer my response to ddubs123's (one of the viewers) question.

  • @JamesTharpe
    @JamesTharpe4 жыл бұрын

    This is a terrible way to handle passwords for your database.

  • @ChandraShettigar

    @ChandraShettigar

    4 жыл бұрын

    James, I agree that using environment variables for database passwords without any encryption is not secure. Scope of this video was very minimal and is a get-started type for cloud-formation & container application. Besides the database password (or in general secret configs - using some encryption or secret management solutions) there are many other topics that the application team need to consider before the application and infrastructure is production-ready.

  • @JamesTharpe

    @JamesTharpe

    4 жыл бұрын

    @@ChandraShettigar Storing passwords in your code might not be the best way to get started. I know password management is difficult, but it's also fundamental to the practice.

  • @ChandraShettigar

    @ChandraShettigar

    4 жыл бұрын

    ​@@JamesTharpe I agree with you on the dev practice. In this case I had to narrow the scope to focus more on CloudFormation basics. May be I should have left a comment about securing database password. Thanks for the feedback.

Келесі