Testing in Production - Talia Nassi - NDC London 2023
Ғылым және технология
How do you know your feature is working perfectly in production? If something breaks in production, how will you know? Will you wait for a user to report it to you? What do you do when your staging test results do not reflect current production behavior? In order to test proactively as opposed to reactively, try testing in production! You will have an increased accuracy of test results, your tests will run faster due to elimination of bad data, and you will have higher confidence before releases. This can be accomplished through feature flagging, canary releases, and data cleanup. You will leave this talk with strategies to mitigate risk, to better your understanding of the steps to get there, and to shift your company’s testing culture, so you can provide the best possible experience to your users. This talk is beneficial because too many people are stuck in the old mindset of testing their features in a staging environment. At the end of the day, we don't care if your features work in staging, we care if they work in production.
Check out our new channel:
NDC Clips:
@ndcclips
Check out more of our featured speakers and talks at
ndcconferences.com/
ndclondon.com/
Пікірлер: 9
The statement "if there's a bug in your 'feature flag' it is not going to impact the non-targeted users" is wrong on a couple of levels: 1. It assumes that ALL new code is protected by the feature flag (FE--> BE/API --> external deps/storage/cache/...). This is extremely unlikely to happen in a complex system as all of your layers and components will need to have access to the feature flags (or you pass the feature flag values in your APIs which means leaking implementation in APIs, among other non-patterns). 2. it assumes that a bug which induces a performance regression doesn't affect old code, which is obviously false. Looking from 10000ft this is a very nice theoretical solution, but once you get into implementation details it gets messy very fast if that stated goal is implemented ad literam.
@archi-mendel
Ай бұрын
There isn't a single practice that will help you deliver 100% confidently. If followed consistently, this will help you avoid lots of bugs in production. When doubled with canary releases, it will help you avoid the most of the wide-scale incidents. And if they don't help - well, it's software development, we have incidents sometimes, so having a robust rollback system is important. They key is that shift right practices shift the mindset from preventing bad stuff from going to production to ensuring that the bad stuff has little chance to impact end users.
Talia: "Keep your hand up if it's an exact replica of production! - Exact!" - (few hands seem to be going up) Talia: "Liars - but that's fine" 🤣🤣🤣😍
Very interesting concept. We recently had to struggle big time with deployment to prod. This is a sensable approach, especially the feature flags
@marcg7843
7 ай бұрын
The use of feature flags that I've encountered is for enabling/disabling a feature/variation. Even with feature flags, you'd need a way to insulate the average user from the change until it is ready either via associated actors with the FF and/or multi tenant environment.
All Applications become multi-tenant. The tenant proper and the TEST tenant.
ProtractorJS in 2023? 🤔🤔🤔🤔🤔 It was deprecated 3 years ago
So you get broken feature on production behid the flag, and your team deploy several fixes to it toghether with other teams deploying their new features, changes etc. How to reach the state when you will be able to confirm that some of the features are ready, do you need to freeze deployemnts and perform full regression, or you jsut beleve that there is no impact from other changes?
@florianfanderl6674
9 ай бұрын
Feature releases and deployments are entirely decoupled in this scenario. So you ship new code, but the feature is still turned off. Usually some people in the company enable the release toggle only for themselves and test it in production. So the feature is visible to internal users (devs, POs, testers) only. If the feature works, they enable the release toggle for everybody and remove it afterwards from the code. The important thing is: deployment does not mean shipping features. It also often implies that you are able to ship very often (like a few times a day). But this goes into the direction and arguments for continuous delivery.