Thank you for this! Just what I needed, solved a question I had been struggling to find an answer to.

Best content as like previous comment I totally agree that this 5 min video taught a lot of quality stuff

Love these DevOps redpills! Actually learnt a lot more practical stuff and good practices in this 5min video than in longer tutorial using tasks and stuff! Subscribed

@JulieNgTech

3 жыл бұрын

Glad to hear! Out of curiosity, what's in the 5 minute videos that's not in the longer tutorials? Sorry for super late response. I just discovered this comments UI for publishers and am finding so many I missed - when I didn't respond immediately to a notification.

@alexandreg3933

3 жыл бұрын

@@JulieNgTech Most videos are introductory or unneccessary long. Yours was straight to the point and practical.

@tezzrexx

Жыл бұрын

@@alexandreg3933 Seconded! Thank you Julie.

The "full" article was very helpfull - thank you!

wow, this video is a masterpiece, all the questions I needed answers to wrapped up in just over five minutes. Thank you, Julie! Heading over to the blog and gonna implement this stuff right away.

@JulieNgTech

2 жыл бұрын

Thanks Matt, glad it was helpful!

Very informative, thanks Julie. I will check out your article 👍

Amazing content, thank you!

@JulieNgTech

2 ай бұрын

Glad you enjoyed it! And I am relieved it is still relevant many years later 😅

This is brilliant, thanks. Heading over to read the article next

@JulieNgTech

2 жыл бұрын

I hope you find the article helpful as well. Let me know if you feel something is still missing.

The article is awesome.. very detailed and with reasoning.. very helpful 👌

Super helpful article! Thank you very much, Julie.

@JulieNgTech

2 жыл бұрын

You welcome Alexander! Thank you for the feedback ❤️

This article was great, really helpful! Exactly what I needed. Thanks!

@JulieNgTech

2 жыл бұрын

Glad it was helpful!

Excellent Video and Article. Thanks !

@JulieNgTech

2 жыл бұрын

Glad you liked it!

Thanks for this Julie! I created the local backend conf file outside of the git and TF working dir all together so I don't even need to worry about having to add my local stuff in gitignore. And wrote a simple bash script wrapper to execute the backend conf file and export local TF_Var's all in one shot. :)

@JulieNgTech

2 жыл бұрын

Automation and scripting FTW :)

Really interesting. I liked the reasoning for using CLI instead of tasks. I would love to see a small video on how to store the state file :') And how to use a service connection with Bash :')

Love it! Thank you for the great info and tips!

@JulieNgTech

2 жыл бұрын

Thanks Lorenzo! Thanks for watching subscribing. Let me know what you want to see more of :)

More of this pls, i just started sys admin role, this stuff is clutch tips.

Very Helpful - Thank you Julie

@JulieNgTech

2 жыл бұрын

Glad it was helpful! Thanks for watching! If it's helpful, please consider subscribing for more :)

I really liked your tips and it's short and sweet

@JulieNgTech

2 жыл бұрын

Glad you liked it!

Great tips!!! Saved me a lot of time . Thanks

@JulieNgTech

2 жыл бұрын

Awesome! Happy coding for Pipelines :)

Thanks Julie. Very helpful

@JulieNgTech

2 жыл бұрын

Glad it was helpful! Let me know what else you'd like to see :)

Your depth of knowledge has earned you a new subscriber! Keep 'em coming :)

@JulieNgTech

Жыл бұрын

Thanks for the feedback. I need these little bits to find motivation to make more videos.

@zimcanit6647

Жыл бұрын

You're welcome. I found tremendous value in your video and article.

Thanks, good tips!

Liked, subscribed and did hit that bell! :) Thanks for sharing.

@JulieNgTech

3 жыл бұрын

Thank you! Let me know if there's other questions I can help answer :)

Great video and write up, thank you Julie.

@JulieNgTech

3 жыл бұрын

Thank you for the feedback ♥️ let me know if there are other topics you're interested in and could use a video :)

@picklednewtons6282

3 жыл бұрын

Hi @@JulieNgTech. I had a look at the devops-governance repo used in some of your examples and I quite like how the drift detection is set up there. A video on how to set that up with a configured response might be cool?

@JulieNgTech

3 жыл бұрын

@@picklednewtons6282 added request to my list :) Be aware, I need to rework the governance repo. Azure DevOps gives least permissions when you have multiple group assignments (unfortunately a footnote in the docs), so the I need to add an additional AAD group per business unit, because ARM uses additive permissions. That is also on my todo list for early March.

How does this video not have more likes?? Seriously!

@JulieNgTech

2 жыл бұрын

Thanks! I'm glad you enjoyed the video that much. I think the likes come over time :)

@anibaldk

2 жыл бұрын

@@JulieNgTech Not sure if you know this, but your post is becoming a DevOps pipeline standard.

Amazing knowledge.Thanks for putting things into perspective. Would be really helpful if you point us to a Repo where you are implemented all this please. Waiting for your series on AzureDevops with Terraform if possible please. Cant thank you enough.You Rock!!

@JulieNgTech

3 жыл бұрын

Thanks Kiran! You can look at some examples here github.com/Azure/devops-governance/tree/main/azure-pipelines Let me know if there's a topic you're interested that's not in the repo - or poorly documented in the repo. Happy deploying!

Because of you , I will pick Azure over AWS

@JulieNgTech

2 жыл бұрын

Really? That made my day 😻 I need more comments like this so I can make videos during work hours.

@marilynlucas5128

2 жыл бұрын

@@JulieNgTech You’re amazing! I love you so much. You’re a true gift to anyone who wants to learn. You’re admirable in every way! I hope I get to meet you in person some day. I always tell people I know who are starting out in enterprise cloud services about you. Enjoy your day!

thanks for the tipps - i used to use the buildin modules too

@JulieNgTech

2 жыл бұрын

Did removing them make accelerate your deployment frequencies?

@pengumind151

2 жыл бұрын

@@JulieNgTech good question, i did not compared the time. But i will. Currently a project where i have to use parameterized custom script extensions with windows powershell. A horror, 6 hours bugfixing. Although windows user .\ as current path in the terraform config for uploaded script you have to use ./ - lmao

Great content, It would be great to have a 2nd edition to show the good practice with the CICD pipeline, Complete deployment and Approvals before deploying

@JulieNgTech

2 жыл бұрын

Hi Roy, yes, I have that on my todo list to walkthrough the pipelines in this repo github.com/azure/devops-governance which does pull requests, deployments, etc. I also recently discovered I don't like how they are done and want to make the git workflow easier and just put a "manual approval" on the service connection. I forget exactly what, but I found a problem that I could not easily get into production without going through all the git checks. In hindsight, I would undo lots of that 😅 and sacrifice some automated approvals for faster deployments and just have 1 manual approval step. Would you want to see this as a video?

@c7roy

2 жыл бұрын

Hello@@JulieNgTech yes I would like to see that on video :)

You're a legend

@JulieNgTech

3 жыл бұрын

Not my goal to be a legend. It's to teach people (and stop repeating myself LOL) But if it happens I won't complain ;-)

Excellent, super useful, thanks so much for sharing. Question on best practices for managing TF state of multiple envs (dev, qa, prod etc). Do you store each env state in a diff storage account, or have a centralised "devops" storage account where you'd have state for each env in the same path, but distinguished by suffix (just like using tf workspaces)? Thanks! Oh, and would love more content, but appreciate super time consuming to put together, love your insights)

@JulieNgTech

Жыл бұрын

Hi Jon-Paul, thanks for the feedback. Re: Terraform state, it depends on your requirement. The biggest thing to keep in mind that the storage account is your security boundary. In a high trust scenario, you can use a single storage account for various state files. But for lower trust scenarios, you want to split them up, not just to prevent read access, but also listing the containers and files in the storage account. See this doc for a more technical detail github.com/julie-ng/cloudkube-aks-clusters/tree/main/backends Hope that helps!

@jpb2085

Жыл бұрын

@@JulieNgTech Thanks so much for the info!

Great video. Any tips on Terraform credentials in AWS? storing them as parameters in Azure DevOps seems convenient but not sure how auditable or safe that would be..

@JulieNgTech

2 жыл бұрын

Not that I know of - because I work for an AWS competitor ;-). But great question! Sorry for the wicked late response. But I put your question in my latest video in the AMA part. For the auditable, etc. definitely go back to your org's security folks and ask them what their requirements are. Then you can see if/how they map to Azure features. We use ADO internally and our credentials are integrated. So it should be possible for you too :)

What an amazing video! Really informative and touching on fundamental points. I have a question related to Terraform State Lock. When, say, a CI build is canceled in the middle of a `terraform apply`, the state gets locked. Could there be a "post run" task that would perform some sort of "unlock" of the state automatically? Or, is it the kind of thing that needs to have a human in the loop to check against possible infrastructure changes before unlocking the state?

@JulieNgTech

2 жыл бұрын

Good question. Instead of asking how, I would ask *why*. Why would a terraform apply need to be cancelled? How often would that occur? The answers to those questions should help you decide whether having that post run task makes sense. The terraform lock feature exists for a reason ;-) In my case, needing to manually unlock it is a rare exception, which is why I would not include it in any of my pipelines. What's your scenario?

@arthurcgusmao

2 жыл бұрын

@@JulieNgTech Great perspective (pointing to the *whys* instead of *hows*). In general, I would like to be able to deliberately cancel a CI job when it is not relevant anymore (e.g.: it is targeting an out-of-date commit), to save time and resources. This scenario usually implies running the CI with the latest pushed commit right after cancelling the ongoing one. Of course, it is not an ideal scenario because, in the first place, one should not be mindlessly pushing commits. Nevertheless, I have observed it happenning in practice with some frequency. Under these circumstances, I would argue that having a post-run job is beneficial. Given that the purpose of the state lock is to prevent multiple, concurrent writers from corrupting the state, and we know for sure that the ongoing apply was cancelled (and will therefore not write to the state anymore), removing the lock seems like a direct logical conclusion. Wdyt? Thanks for leading me towards this reasoning btw :)

Can you please create a detailed video on creating a yaml file from scratch and how to segregate stages, jobs and steps in different yaml

@JulieNgTech

2 жыл бұрын

Why from scratch? Also have you seen this video of mine that does talk about stages, jobs, etc.? kzread.info/dash/bejne/l2SWqJOFnMbSddY.html

I got error when running terraform init -backend-config=azure.conf: Terraform initialized in an empty directory! However I did have a main.tf in the current folder.

@JulieNgTech

3 жыл бұрын

When you say "current folder", did you change it with cd? If I remember correctly that doesn't work for various reasons. Your code is running in a sub shell. Instead use the `workingDirectory` property in your script. See this doc for details: docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema%2Cparameter-schema#script

If you were using different folders for different types of resources, network folder for virtual networks and subnets, application folder for data factories, databricks, etc .. how would you go about then having a file to be able to access the terraform state of those other resources when states are managed by resource type as well? The terraform_remote_state provider needs the config block which asks for the same stuff in your azure.conf?

@JulieNgTech

3 жыл бұрын

Hey Hyn, why would you put networks, databricks, etc. in their own folder? If they need the access to the same Terraform state file, wouldn't they be a single Terraform IaC deployment?

@Klainn

3 жыл бұрын

@@JulieNgTech The reason i've done it is to separate out resources by a type so as to limit the possibility of a random network change causing a databricks rebuild or vice versa. I also wouldn't want to chug through the entire environment with an apply if all I was doing was adding a subnet. I also think I saw it on TF best practices.

@JulieNgTech

3 жыл бұрын

@@Klainn avoiding unintentional builds is one of the most challenging practices to master in DevOps and in my opinion a life-long journey. For infra, I sometimes still flinch when pushing. That being said, I recently gave a talk at DevOps.js that talked about triggers in git repos (one vs many) and that might help you kzread.info/dash/bejne/iHWnt9etf9Sfnbw.html

Hello i have a Question when i run terraform project locally it works fine for me but in azure devops environment does not any idea

@JulieNgTech

3 жыл бұрын

first step of debugging is to compare configuration, e.g. ARM credentials, state files, terraform versions, etc. check your error messages and some googling should help you ;-)

So, when you push a PR to Git, how do you get Terraform to run only "init" and "plan" and not "apply" until the PR has been approved?

@JulieNgTech

Жыл бұрын

Those are 2 separate event triggers. The first you want is for the PR trigger like in this example, which only does `init` and `plan`. github.com/Azure/devops-governance/blob/main/azure-pipelines/pull-request.yaml Theoretically the `apply` would be a different pipeline that triggers on push to the target branch of the PR, e.g. `main`. So commits to you e.g. `feat/*` branches trigger the PR pipeline. Once someone merges it into `main` or whatever you naming convention is, the pipeline would do an `apply` BUT, a BIG HUGE BUTTTTT the Terraform plan that was approved in the PR would be stale. The 2nd pipeline would run a second plan and apply potentially without human intervention. That's why I don't put Terraform apply in the pipelines. Too risky for me.

What about state files from terraform?

@JulieNgTech

3 жыл бұрын

IMO Best Practice is to use 2 Storage Accounts, 1 for production and 1 for non-production - both using SAS tokens to access the state file. This follows cloud governance best practices to separate RBAC and thus credentials for production. Unfortunately you need 2 storage accounts and it is not enough to scope to an Azure Storage Container because Terraform workspaces will query the entire Storage account to find all statefiles it thinks might be a "workspace." Does that soundbyte answer your question? I've been meaning to blog or do a video about it with a demo but I haven't gotten around to it.

Love this. great content. Thank you

@JulieNgTech

2 жыл бұрын