How do you calculate the price of running Kubernetes and other Cloud resources and how to you split the cost per teams or other entities? IMPORTANT: For reasons I do not comprehend (and Google support could not figure out), KZread tends to delete comments that contain links. Please do not use them in your comments.

I have just discovered your channel and I've watched multiple videos already. I've been working as a DevOps engineer for 7 months now and use k8s daily. I will most certainly be watching all your other videos. Keep up the great work!

Fantastic.. I think this would work great to charge back to our internal departments.. Thanks again for the amazing detail

Great demonization. You have convinced me to try this tool for our multi-zone k8 cluster. I will definitely try this out. Thanks, @Viktor

cool, thanks for the introduction

Interesting stuff, I'm sure this kind of product will only grow in the near future.

@DevOpsToolkit

2 жыл бұрын

I'm sure of that. The more companies move to the Cloud and the more they shift-left, the more there will be the need not only to see how much something costs, but also how to optimize based on costs and how to split the costs based on teams or any other group. When we run on-prem, the situation was different. Cost based on usage was not that important since the hardware is fixed. If you buy servers, there's not much incentive to try to use them less. They're already yours. In Cloud, we pay for what we use so there is a clear incentive to use less (without sacrificing performance and stability).

Thank you for the demo and info - Great product to explore. (FYI - It is my understanding that the "m" for CPU is referring to "millicores")

@DevOpsToolkit

2 жыл бұрын

You're right. My bad. Not the first time I misspell something in English :(

Thanks! It wouldd be nice to use admission controllers to limits costs by namespaces or deployment for example. Also it would be nice an option to compare the same current kubernetes infraestructure in AWS with GKE or Azure, since you can extrapolates the costs to other providers somehow

@DevOpsToolkit

2 жыл бұрын

I was thinking along similar lines. It would be great it Kubecost would have easy integration points other projects could use. That could be Gatekeeper or Kyverno for admission controllers, Crossplane or ACK for infra and services, etc. It might already be doing that. I need to dig deeper into it...

They are open core, from their own docs, but they are not being upfront about this, -1 to them. Next people are complaining about this tool not being free or OSS, I mean it looks like a good tool, so requires effort to develop and maintain. I guess if we are concerned about this, then better to move with VMs and CM tools. :D

Thanks for another great video! Sadly Kubecost is only free for a single cluster and this could change at anytime depending on the companies policy. the multi cluster (max 100 nodes) starts at $449 pm. Little disappointed since watching video as I was under the impression it was open source.

@DevOpsToolkit

2 жыл бұрын

It is a bit confusing or misleading... You can use open source version without any limitations (e.g. on any number of clusters). It does not have a Web UI and a few "enterprise" features though, but it is there and completely free. More over, KubeCost team will very soon release another project that goes in that direction but, unfortunately, I am not at liberty to talk about it just yet.

You put out lots of great content and this is a great review of the features of the commercial product Kubecost. But as others have pointed out, presenting this as an OSS and free project without explaining in the video that you use the limited free tier of their commercial product is very misleading. I think it'd be fair to add some clarifying info in the video. As captions or whatever. If you're going to review an OSS project you should only use the OSS features of it IMO. Again, you're generally doing a great job with this channel and I've learned a lot from you.

@DevOpsToolkit

Жыл бұрын

You are absolutely right. I made a mistake by (unintentionally) forgetting to clarify that in that video. I tried to improve it since then.

@DavidBerglund

Жыл бұрын

@@DevOpsToolkit yeah no worries! It is a while back your channel is just getting better all the time. Keep it up!

Hello Viktor, I discovered your channel yesterday out of curiosity to understand how the deprecation of dockershim would affect k8's deployments. Oh my gosh, I have information overload already on so many topics I didn't know I'd be curious to learn about until I started learning from your channel. I have a ton of questions, but I'll ask a few. By the way, I am not a DevOps Engineer, at least not yet 🙂, but I am open to learning as much as possible. 1. You mentioned nerdctl locally, Kaniko in the k8's cluster, and some other tools can build images from dockerfile. What would be the alternative to dockerfile? Is there anything like containerdfile 🧐 or any other runtime file? 2. What will the ideal development setup be using k8's locally and not docker compose? 3. This is not a question. Thanks for taking the time to share this information.

@DevOpsToolkit

2 жыл бұрын

1. There is no commonly accepted alternative to Dockerfile. That's OK since that is only a specification that is used by (almost) any tool that builds container images. It's called Dockerfile only because that's the name Docker promoted but the file can be called anything. So, ignore that files are often called Dockerfile and think of it as a specification that (almost) any tool understands and can use to build container images. 2. Rancher Desktop (you'll find a few videos about it in this channel). The more important question is how to define apps running in Kubernetes. There are quite a few ways (e.g., Helm, Kustomize, Crossplane, OAM, etc.). You'll find the videos about all those on this channel. Once you define your apps, you can use them to run your apps in any k8s cluster, no matter whether it is local (e.g., dev) or a remote cluster (e.g., production). 3. No need to thank me. We're all doing our best to help each other.

@DevOpsToolkit

2 жыл бұрын

One more thing... Feel free to ask any question in comments. As a (better) alternative, you can also join us in our biweekly Ask Me Anything (AMA) sessions. They are every second Thursday, next one being on March 24.

@motdde

2 жыл бұрын

@@DevOpsToolkit Thanks, Victor.

I was wondering whether kubecost is laser focused on monitoring and helping in just kubernetes and associated cloud infra/services or it could even become a single pane of glass to monitor and optimize the cost of resources/services distributed across multiple cloud since you briefly pointed to cloud integration. Just curious , since customers are increasingly getting more interested in cost monitoring and looking for opportunities to optimize resource and its cost utilization across cloud based data centers without playing swivel chair

@DevOpsToolkit

2 жыл бұрын

For now, it is focused on Kubernetes resources as well as those related to it. It does not show the costs of the resources that have no relation with k8s. That makes certain sense since Kubecost itself is a service running in Kubernetes. What would be interesting is a potential combination with Crossplane. It is also running in k8s but it manages any type of resources. If we could connect the two, that would give full coverage both in terms of resource management as well as costs. I'll get in touch with Kubecost folks and see whether they might be interested in such a combination.

@shuc1935

2 жыл бұрын

Exciting , I asked the same question to Kubecost folks during my experimentation with the product but got some convoluted although not intentional response. The marriage between these two products a.ka. integration could be a potential game changer as o11y is becoming dominant in monitoring space. Open Telemetry is the second most popular project in the CNCF landscape. Just kubernetes cost management would not cut a big deal as we will have sprawl of monitoring tools , too difficult to observe a system as a whole , and to be honest sprawl of monitoring tools is deterrent to achieving the state of observability. This is my own opinion based on experience, experiment and observations.

Hi! Do you know if it is possible to see more than 1 cluster if we pay for the full version? Thanks!

@DevOpsToolkit

Жыл бұрын

It is. The "business" (full) version does not limit the number of clusters. The pricing depends on the number of nodes.

Kubecost is not free (well only if you have a single cluster). It is actually pretty expensive, especially for smaller teams with multiple clusters that require SSO support and the unified view (needs Enterprise license, which is around 20k++ per year)

@DevOpsToolkit

2 жыл бұрын

KubeCost itself is free (github.com/kubecost/cost-model). It's open core. Now, there are indeed features that are available only to customers which is the model most open source projects are using. It's free, but engineers working full-time on the project need to get salaries so there is a commercial layer on top of it. Those enterprise features typically (as well as in the case of Kubecost) are targeting bigger enterprises. If I got it right, 20k per year is the price for quite a large operation which, typically, does not run on opensource. In any case, you can use github.com/kubecost/cost-model directly.

@jemag

2 жыл бұрын

@@DevOpsToolkit my understanding is that even the base/free version can only be used for a single cluster. It is pretty rare to only have 1, even for smaller teams

@DevOpsToolkit

2 жыл бұрын

​@@jemag I think that github.com/kubecost/cost-model/tree/master/kubernetes is completely free and can be run on as many clusters as you need. That's the OSS version. You won't be the Web UI, SAML, support, and a few other things, but it should work without any cost. You can see it as the last point in guide.kubecost.com/hc/en-us/articles/4407601821207

@admun

2 жыл бұрын

It needs Prometheus for storing data, and is not that easy to setup. I ran into mismatch congratulations with the Prometheus I have on my cluster.... but it was a while back, I hope it is better now. It also has less features, IIRC none of the cost suggestion feature are there. Quote from cost-model doc: Note: This approach provides less functionality compared to other install options referenced above. Also, Prometheus and kube-state-metrics are external dependencies for this installation path.

Great review. CAST AI has covered this on their blog post as well. Basically using both tools Kubecost and CAST AI together will give you an end-to-end solution that will keep your cloud costs in check and generate some pretty impressive savings - CAST AI Kubecost A Perfect Match for End to End Cloud Cost Optimization (can't add URL)

@DevOpsToolkit

2 жыл бұрын

Oh yeah. I think I mentioned it in the video but I should have made it more prominent.

Can kubecost work on EKS + Fargate?

@DevOpsToolkit

2 жыл бұрын

I'm guessing that, in that case, you'd like to exclude scaler from KubeCost and are wondering whether the rest works. Is that case? If it is, the answer is yes :)

I am snuggle how it measure right or not in some case Let say i have a pod it will have high load in just specify time , kind of like cronjob or ruby sidekiq let say normal usage it takes only 100m cpu but when thag request come it takes 1000m for few hour, may be just 1 or 2 times a month in this case i must give it at least1000m request unless i willing to tune before they exexute , which seems not auto enough but kubecost will based on normal suggest tell me i only need around 100m cpu... and when the job running it will never success..... Whats your suggestion master 😅

@DevOpsToolkit

2 жыл бұрын

That's why we can specify requests and limits. Requested memory and CPU represent "normal" usage while limits are telling k8s what the spikes are. On top of those, we have HPA that scales your apps based on specific criteria (e.g., memory, CPU, etc.) so, instead of provisioning the apps for the worst-case scenario, we scale them when needed.

@vn7057

2 жыл бұрын

@@DevOpsToolkit yes but when gke is based on request to assign pod ans do the scale , i give 100m and 1500m limit it will seat to a node that allow those 100m , it fail run but will still seat back to the orginal node , becox the orginal node does have cap 100 or 300m.It will not provision a new node for this case right? Assume i have a nodepool that containt 2 node and allow to scale up both are use 1500m cpu celling both already usedup 1100m the pod will able to assign in this two node without a problem becox that what the request i told

@DevOpsToolkit

2 жыл бұрын

@@vn7057 You're right only if your nodes are having 100% resource utilization. That is hardly ever the case. There are almost always some unused resources. We should try to have resource utilization high, but that is almost never 100% of the capacity. Keep in mind that I'm not talking about the actual resource utilization, not only what was requested. On top of that, if you do have HPA and, let's say, set it to scale up when when actual utilization reached the requested utilization or, maybe, 120% of requested utilization (I'm assuming that there is likely 20% unused capacity), then the app will scale horizontally instead of scaling vertically. You can also use VPA as well if there are too many oscilations in resource usage of an app. If you combine it with "intelligent" cluster autoscaler that will create nodes based on what is needed (instead of making new nodes based on node groups), you have a pretty robust system. Finally, Kubecost gives you recommendations. If does not modify your system. It's up to you to interpret the recommendations.

How about opencost? Which one to choose between opencost and kubecost?

@DevOpsToolkit

11 ай бұрын

I have open cost on my to-do list...

@rubalfromthehimalayas

11 ай бұрын

@@DevOpsToolkit thanks for your reply

@rubalfromthehimalayas

11 ай бұрын

@@DevOpsToolkit kubecost comes with it's own local grafana and prometheus integration, how to use it in already present grafana dashboard? Any idea

@DevOpsToolkit

11 ай бұрын

You can disable both grafana and Prometheus through helm values.

@rubalfromthehimalayas

11 ай бұрын

@@DevOpsToolkit ok, I installed kubecost via yaml and not via Helm. I will install with helm and try to disable both. Thanks

Kubecost free tier only supports 1 cluster.....

@DevOpsToolkit

2 жыл бұрын

That's true. To be more precise, you can use open source version without any limits, but it does not contain the Web UI, SAML, and a few other "enterprise" features. Then there is the free tier that is limited to one cluster. Finally, the rest is for customers.

@admun

2 жыл бұрын

Yes.... but see my other comment for the caveat...

@DevOpsToolkit

2 жыл бұрын

@@admun I saw it and I agree. Kubecost OSS is not necessarily a good option so it mostly boils down to the "enterprise" version.

@admun

2 жыл бұрын

Indeed, if budget is not an issue, it can be something worth trying. But one thing to consider with such tool in the paid capacity is the cost, vs the saving actually made.

@vytautasklova1616

2 жыл бұрын

​@@admun feel free to play around with CAST AI - savings report is free and actual savings all the way will be way higher then platform fee. On average our users usually saves around 63%

You "think" you can scale down your cluster while non-evictable pods (or daemonset) are laughing at you lol

@DevOpsToolkit

2 жыл бұрын

That is indeed a problem, but something we need to do nevertheless. Otherwise, what's the point of having elasticity, if it always results only in scaling up. That's a recipe for bankruptcy. Also, if we cannot evict pods, we cannot upgrade immutable and I really do not want to go back to doing random changes at runtime.

@fenarRH

2 жыл бұрын

@@DevOpsToolkit there is a thing called affinity/anti-affinity to make apps more redundant/reliable/performant which triggers those cases where application availability can be constrained with cluster downsizing.