Persistent Volumes on Kubernetes for beginners

Ғылым және технология

Subscribe to show your support! goo.gl/1Ty1Q2 .
Patreon 👉🏽 / marceldempers
In today's video, we're going to be talking about persistent volumes and how they help us to run workloads that require persistence. These can include databases, caches, message queues and anything that requires data to be stored.
But remember applications that require persistence can be complex. To help run these complex applications, Kubernetes uses StatefulSets instead of deployments.
Remember to checkout my video on StatefulSets here - • Understanding Stateful...
Also if you want to support the channel further, become a member 😎
marceldempers.dev/join
Checkout "That DevOps Community" too
marceldempers.dev/community
If you are new to Kubernetes, check out my getting started playlist on Kubernetes below :)
Checkout the source code below and follow along :)
Kubernetes Guide for Beginners:
---------------------------------------------------
• Kubernetes development...
Kubernetes Monitoring Guide:
-----------------------------------------------
• Kubernetes Monitoring ...
Kubernetes Secret Management Guide:
--------------------------------------------------------------
• Kubernetes Secret Mana...
Like and Subscribe for more :)
Source Code
--------------------------------------------------------------
github.com/marcel-dempers/doc...
Follow me on socials!
marceldempers.dev
Twitter | / marceldempers
GitHub | github.com/marcel-dempers
Facebook | thatdevopsguy
LinkedIn | / marceldempers
Instagram | / thatdevopsguy
Music:
Track: Fox Beat 2 - Jeff Kalee - Pillow Talk - Royalty Free Vlog Music [BUY=FREE] | is licensed under a Creative Commons Attribution licence (creativecommons.org/licenses/...)
Listen: / jeff-kalee-pillow-talk...
Track: Amine Maxwell - Le Soir | is licensed under a Creative Commons Attribution licence (creativecommons.org/licenses/...)
Listen: / le-soir
Track: Sappheiros - Spirit | is licensed under a Creative Commons Attribution licence (creativecommons.org/licenses/...)
Listen: / spirit
Track: n0tavailable - mourner - sad trap beat | is licensed under a Creative Commons Attribution licence (creativecommons.org/licenses/...)
Listen: / mourner
Track: souKo - souKo - Parallel | is licensed under a Creative Commons Attribution licence (creativecommons.org/licenses/...)
Listen: / parallel

Пікірлер: 95

  • @MarcelDempers
    @MarcelDempers3 жыл бұрын

    Understanding StatefulSets for beginners👉🏽 kzread.info/dash/bejne/rJ5q1MF-dcvcZtY.html

  • @modenie
    @modenie3 жыл бұрын

    You're an excellent teacher Marcel, thank you for making these videos. Your vlog format is groundbreaking in tech. While there are plenty of others on youtube offering fantastic content, you've introduced (at least to me) an element that has made it so much easier to take the information in. Well done man!

  • @shanef1867
    @shanef18673 жыл бұрын

    Possibly the best tutorial on the topic!!

  • @fxylxy
    @fxylxy3 жыл бұрын

    Hi Marcel Thank you so much for this simple explanation with a REAL WORLD example of persistent volumes. I have been wracking my brains trying to tie together pv, pvc, and statefulsets. You explained and demonstrated in 20 minutes what other people couldn't get across in hours of demos. Love your channel, keep it up mate 👌👍🙂

  • @krishnamohanyerrabilli4040
    @krishnamohanyerrabilli4040 Жыл бұрын

    Man this is just awesome, I'm beating myself for not understanding, after watching this I can assure myself I got the idea I remember this quote, You can't connect the dots forward - only backward - Steve Jobs Thank you for the lecture marcel, You made my day.

  • @ourjamie
    @ourjamie3 жыл бұрын

    Marcel, your stateful and persistance videos demos are brilliant, thank you for putting in the work to share your knowledge, your delivery is brilliant

  • @karatefutzi
    @karatefutzi Жыл бұрын

    Bro out here making every DevOps engineers life so much easier, thanks the school system around the globe should take you as an example on how to transfer knowledge....

  • @aufkeinsten7883
    @aufkeinsten78832 жыл бұрын

    You're on another level mate, so concise, so clever, so well spoken, love your videos

  • @wihannel
    @wihannel2 жыл бұрын

    This video is almost as good as you triceps. Respect

  • @yovangrbovich3577
    @yovangrbovich35773 жыл бұрын

    Thanks Marcel! Would really love to see one on Argo Workflows/Events for setting up and triggering pipelines in kubernetes!

  • @FuhQshane
    @FuhQshane Жыл бұрын

    Thank you so much for this. I’ve been in class all week and studying my ass off and just have NOT understood the pv claim until now. Now I can get some sleep. Lol

  • @PeterMehr
    @PeterMehr9 ай бұрын

    Great explaination!! After watching at least 10 other tutorials, yours was the first one that really worked!

  • @manideep6814
    @manideep68142 жыл бұрын

    You are too good at this, your videos have helped me understand and breakdown those heavy jargons on K8s doc pages. Thank you.

  • @rolandvarga3912
    @rolandvarga39123 жыл бұрын

    this devops guy LIFTS!!

  • @MarcelDempers

    @MarcelDempers

    3 жыл бұрын

    💪🏽

  • @karthickkarthi.3342
    @karthickkarthi.33422 жыл бұрын

    Every sec has information in your videos.

  • @marcziel5424
    @marcziel54242 жыл бұрын

    Good video generally, but there's a misconception about containers. A container is not a process, it's a sandboxed runtime environment in which you can run one or more processes. You can stop a container (`docker stop `) which terminate all processes inside of a container and then run it again (`docker start `) and the files will still be there. Kubernetes never bothers to restart a container - when it restarts a pod it recreates containers with the same IP address assigned, etc. so you might think that containers are completely stateless (and they're typically treated as such) but a container keeps its filesystem state after all of its processes terminate which is pretty useful when it comes to retrieving logs and inspecting what caused the termination.

  • @joaosakai
    @joaosakai3 жыл бұрын

    Thanks a lot, Marcel this is the best video about the concept that I've found on KZread, your explanation coming from docker volumes creates a solid ground to easily understand the kubernetes part :smile:

  • @maiezbenromdhane8233
    @maiezbenromdhane82332 ай бұрын

    excellent and well detailed! thank you for the explanation dude!

  • @bnssoftware3292
    @bnssoftware3292 Жыл бұрын

    You sir are amazing! Love your style and your editing and your teaching ability. You're like the Kubernetes god! Keep up the great work!

  • @mrpoeky2527
    @mrpoeky25272 жыл бұрын

    Thanks for this amazing video, I was really struggling with this part in Kubernetes. It helped me a lot :)

  • @vincenzocalia6552
    @vincenzocalia6552 Жыл бұрын

    Marcel you're a good man. You always explain things very clearly and dough you type as fast as the light you repo in GitHub shows everything. So many many thanks. I was stuck trying to bind a local PV to a mongo db StatefulSet , looking at your code(dough for Postgres ) made me find a way to debug my pv.yaml.. it results that was a typo in the nodeAffinity key.. thank you so much again. Vinny

  • @ravindramahajan2685
    @ravindramahajan26853 жыл бұрын

    Excellent video It helped me to clear my concept Thanks.

  • @semibenhsan9551
    @semibenhsan95514 ай бұрын

    Just Amazing ! Thank you very much man ! Keep Going !

  • @nikolai.kolosov
    @nikolai.kolosov3 ай бұрын

    Thanks for the great content Marcel

  • @evgenisakhatsky612
    @evgenisakhatsky612 Жыл бұрын

    Great explaination!

  • @bass9454
    @bass94543 жыл бұрын

    Your channel is a godsend 🙏🙏🙏

  • @NIKHILVERMA-xr7lc
    @NIKHILVERMA-xr7lc3 жыл бұрын

    Thanks for sharing. Gives me better understading than k8s doc.

  • @syedmuhammadzaeemhasankazm7740
    @syedmuhammadzaeemhasankazm77403 жыл бұрын

    Thanks alot Marcel, I finally understood the concepts of PV, PVC and how to use them with Postgres thanks to you. Massive thanks! Also I would like to request if you could teach how to do authentication and authorizations from nginx-ingress using Keycloak, as I am sure many people struggle with it... And there arent any KZreadrs like you, who teach from the basics and build up concepts upon that, rather they usually just assume that we just know stuff. Also, since KeyCloak is really popular Auth as a Service, so I am sure it would be very helpful for others too, so please consider doing a video on it, please. Thanks a lot in advance.

  • @shubhamtomar676
    @shubhamtomar6762 жыл бұрын

    Very informative

  • @marwanghubein7879
    @marwanghubein78793 жыл бұрын

    thanks a lot , very informative and useful

  • @carlosgoncalves56
    @carlosgoncalves562 жыл бұрын

    very good video, tks Marcel.

  • @ghauri001
    @ghauri0013 жыл бұрын

    Thank you for this video, please make a video on the production-ready Postgres database.

  • @martinpetersson6979
    @martinpetersson69792 жыл бұрын

    Best in class 🙌✌️

  • @Toemo1990
    @Toemo19902 жыл бұрын

    Great video!

  • @dapkman9354
    @dapkman93543 жыл бұрын

    You're great. Good job!

  • @bashir26
    @bashir262 жыл бұрын

    great video thank you

  • @ribalaladeeb8310
    @ribalaladeeb83102 жыл бұрын

    Great Video, breaking down the content in ingestible bytes. One improvement I'd suggest is timestamping your video so that people can skip ahead to the exact part they are looking for. Keep doing you man your content is great

  • @evgenisakhatsky612
    @evgenisakhatsky612 Жыл бұрын

    Thank you so much

  • @magnomaximo2471
    @magnomaximo24712 жыл бұрын

    Greate, thanks a lot for this video it cant be better

  • @bAbApersianwale
    @bAbApersianwale2 жыл бұрын

    Thank you

  • @nikhilverma4976
    @nikhilverma49762 жыл бұрын

    You are really good at explaining things. Can you do a tutorial on tricep extensions? Thanks!

  • @kingjameszz

    @kingjameszz

    Жыл бұрын

    super underrated comment lol

  • @maheshkariya
    @maheshkariya3 жыл бұрын

    Thanks brother

  • @hurolinci5986
    @hurolinci59863 жыл бұрын

    Suffering for the past week figuring out how to persist data on a database container has finalldy led me here to find the solution...

  • @stefano8936
    @stefano89362 жыл бұрын

    Thank you, the guide was very clear and concise. Do you have a guide on how to handle a "real world" project in the different phases? For example I imagine one would develop in local on minikube, then deploy on the cloud (AWS, GCP, etc), maybe having TEST/PRODUCTION as different clusters. How do you manage all this? Does it make sense to have in the same git repo different kubernetes folders, one for each environment, containing the appropriate yml? Or would you rather have different branches? A guide on this would be great.

  • @hamidullahmuslih6301
    @hamidullahmuslih63012 жыл бұрын

    awesome !!

  • @janituma9769
    @janituma97693 жыл бұрын

    Thx Great Video 👍👍👍👍

  • @jaysistar2711
    @jaysistar27113 жыл бұрын

    Thanks! I have a single tennet app that will have multiple copies of the same pod running in different namespaces with different persistant volume claims. Many of these resources have the same name, but are just deployed in different namespaces. It would be great to see a video about how a single pod .yaml could be used in that way to create different volumes.

  • @c0nsaw
    @c0nsaw3 жыл бұрын

    Excellent

  • @laxmankamble580
    @laxmankamble580 Жыл бұрын

    Hi Gaurav, is this end of K8s series, please add if few more topics pending. and i must saym your tutorial made my concep clear and handson experianced.

  • @haneefmustafa1600
    @haneefmustafa16002 жыл бұрын

    Perfect thanks! Super .......

  • @xxdivinexx90
    @xxdivinexx903 жыл бұрын

    Hey Marcel, love the videos! I think there is a small mistake with your explanation on PVCs claiming storage from PVs. You cannot have several PVCs bound to one PV, once a PVC is bound to a PV the PV will not be able to bind to another PVC but many pods can use the same PVC. You create PVs to configure a certain way to claim storage and to use it you bind a PVC to it. If you aren't using PVs the storage class dynamically creates and provisions a PV for the requested PVC. Keep up the great content!

  • @AnandKumar-dc2bf
    @AnandKumar-dc2bf3 жыл бұрын

    That Brilliant Guy who is Awesomeeeeee.....

  • @abd.alharbi
    @abd.alharbi Жыл бұрын

    Thank you kindly for this video. One question though. Why would a pvc specifies a storage class? isn't it already defined in a pv?

  • @weitanglau162
    @weitanglau1623 жыл бұрын

    Great video Marcel! However, I do have a question. I am able to persist data through the lifetime of the container / pod, but not able to do so if I restart my cluster. How do I achieve this? I am planning to use local storage class

  • @A_Dudkin
    @A_Dudkin Жыл бұрын

    Hi Marcel and thanks you for the great job you do! Can you give a hint or direction please. I've deployed wp with pvc using nfs-provisioner on bare-metal. nfs share is located on the neighbouring vm host in the same subnet and basically both k8s nodes and nfs node use the same physical disk. What I see is that i can easily store anything on this share manually from any pod but WP + mysql is very slow when using this nfs (e.g. it take up to 15 sec to read a few KB files). perms on the share are 777. Access mode readwritemany if it makes any difference. I've googled for this issues and it turns out quite common but there is no specific solution or reason that I could find. Any advices please?

  • @davidchoy6220
    @davidchoy62203 жыл бұрын

    The local file system path you specified, you would see files in there after creating the table right?

  • @wesleyvandermeer9313
    @wesleyvandermeer9313 Жыл бұрын

    HERO!

  • @syedshamshami5250
    @syedshamshami52503 жыл бұрын

    What is the best way to setup storage server and its configuration for kubernetes cluster persistence volume for production system? NFS or ISCASI any hardware brand specific. Thanks !

  • @tanchwa3740
    @tanchwa37409 ай бұрын

    How does it work, then, if I have multiple apps that need to share the same physical storage. From what it sounds, I still need to create a pv for all of them, but can mount them to the same physical path in the storage class, nfs for example. But then how does the storage amount come into play? How can a pv reliably tell a claim “hey, I have this much storage available, use me.” If there is overlap in the physical storage?

  • @NiteshKumar-do4en
    @NiteshKumar-do4en3 жыл бұрын

    Please a video on storage class for aws

  • @georgelza
    @georgelza2 жыл бұрын

    how would I add nodeselectos and affinity rules for statefulsets?

  • @rajump4276
    @rajump4276 Жыл бұрын

    @13:40, How is the PersistentVolumeClaim referencing the PeristentVolume (example-volume) ? How does the pvc know which PV to use?

  • @linuxistul
    @linuxistul3 жыл бұрын

    Hi Marcel, I've created a PersistentVolume using the hostpath StorageClassName and I mounted it in a mysql pod. The Persistent Volume hostpath is "/var/folders/data", and it's mounted in "/var/lib/mysql" in the Container. What's strange is that when I list the PV hostpath "/var/folders/data" it's empty. But when I create a test database and then delete and recreate the deployment the test database is still there. You know where the container "/var/lib/mysql" files could be found? I was expecting to find them in the PV hostpath "/var/folders/data". I'm using docker for mac.

  • @anthonypham7518
    @anthonypham75183 жыл бұрын

    Hey Marcel, once again thanks for another excellent video. One question though. Would the Postgres data survive a Docker Desktop restart?

  • @MarcelDempers

    @MarcelDempers

    3 жыл бұрын

    Technically, it should since the example used host path persistence so unless the host is destroyed, it should persist yes

  • @griet535

    @griet535

    3 жыл бұрын

    @@MarcelDempers Hey Macel, thanks for the video. unfortunately the data being lost when docker restarts.

  • @ezhisaivallabanr3189

    @ezhisaivallabanr3189

    2 жыл бұрын

    @@MarcelDempers data loss after restarting the docker desktop

  • @braxtonl3319
    @braxtonl33193 жыл бұрын

    Hey Marcel, great videos. After I altered the ConfigMap to use a different POSTGRES_USER and deleted/recreated everything including the PV, the old database user & table existed. Can you explain why that happened even after I deleted the PV? Is there more to "scrub" here? I assumed recreating everything would result in a fresh database with the new user credentials. Thanks!

  • @MarcelDempers

    @MarcelDempers

    3 жыл бұрын

    This highly depends on the storage class you use. In this demo I use host path which means data is persisted on the node at a path (which can be changed). The get a fresh storage, you either have to change the path to get a new folder on the host. Remember your old data would remain. Or SSH to the node, clean it up, Or run another container with a host mount and carefully run "rm" command to delete the content

  • @braxtonl3319

    @braxtonl3319

    3 жыл бұрын

    @@MarcelDempers Thank you for such a quick response! I used hostPath/your tutorial code too. If you have any advice on ssh-ing to the docker-desktop (Mac) node to clean up, I'm all ears. Thanks again for these videos, very helpful.

  • @turalaliyev6288
    @turalaliyev62883 жыл бұрын

    Hi Thanks you for great video.! I did the same and tried to deploy jenkins but when i create pv and pvc . mnt/data is empty. and i used volumemount to var/jenkins_home.

  • @MarcelDempers

    @MarcelDempers

    3 жыл бұрын

    If you need help with Jenkins checkout my CI/CD playlist, there are two Jenkins guides with source code for you 💪🏽👊🏽

  • @jezzadiaz5972
    @jezzadiaz59723 жыл бұрын

    Hello. I have a question. What if you would want to save and mount the logs of a server application? Should you still use deployment or statefulset?

  • @MarcelDempers

    @MarcelDempers

    3 жыл бұрын

    For logging, I have a separate video in the pipeline, You would rather export logs from the app and not store it in a volume. Take a look at the EFK\ELK concepts, where basically every pod in K8s writes logs to stdout. It is best practise not to write logs to file and rather use stdout\stderr. These logs will automatically end up in the host docker logs which you can push out of the system using Fluentd\FluentBit. This allows you to centralise logs for all deployments\statefulsets running in your cluster under something like Elastic Search or 3rd party application.

  • @jezzadiaz5972

    @jezzadiaz5972

    3 жыл бұрын

    @@MarcelDempers oh. interesting. I'll look it up! Thank youuu so much. been watching your videos. so much helpful for me. great videos btw :)))

  • @sushil1200
    @sushil12003 жыл бұрын

    where is /var/lib/postgresql/data located? Is this location inside pod?

  • @user-fi9hn4vd5s
    @user-fi9hn4vd5s2 жыл бұрын

    Hi, i got a qusetion, if my volume is stored application's log file, this volume should be oversized, how could i logrotate in this volume to manage the log file size? do u have any suggestion? thank u so much

  • @MarcelDempers

    @MarcelDempers

    2 жыл бұрын

    I would recommend looking into Fluentd for shipping logs and not store logs in a volume. K8s has pod logs go to a container runtime folder and Fluentd mounts that directory to forward logs out. The container runtime is designed to rotate logs automatically. kzread.info/dash/bejne/aJ-hqtiRlKTgebA.html

  • @user-fi9hn4vd5s

    @user-fi9hn4vd5s

    2 жыл бұрын

    @@MarcelDempers thank u so much! its useful to a kubernetes beginner user.

  • @devopskey6251
    @devopskey62513 жыл бұрын

    but If we have a three node cluster one is master other two are worker first time our Postgres pod is schedule on worker1 now its PV on worker1 and second time after deleting the pod k8s schedule it on Worker2 now pod create its PV on Worker2 but now data is not persistence. because data on worker1 can not allocate to worker2 pod? What's a solution?

  • @MarcelDempers

    @MarcelDempers

    3 жыл бұрын

    This is because you are using a deployment with PVs which is not meant for stateful workloads. You'll need to learn about StatefulSets kzread.info/dash/bejne/rJ5q1MF-dcvcZtY.html

  • @marwanghubein7879
    @marwanghubein78793 жыл бұрын

    how to backup and restore database in this case ?

  • @derencevoo9146
    @derencevoo91463 жыл бұрын

    Hi Marcel, my setup is a bit unique, I have created a K8S cluster on a few VMs in our own private cloud and all the nodes are mounted with a single NFS drive (say /data), can I define a storageclass of hostpath type using /data? Does PV has to be mounted to all the nodes?

  • @MarcelDempers

    @MarcelDempers

    3 жыл бұрын

    That depends on the app you're planning to run and how it uses the storage. If you plan on using a statefulset its recommended that each pod has its own storage. StatefulSet will use a storage class to provision a PV per pod and not share the same storage. Hostpath will work, but if you mount a shared storage to all the nodes, they may override each other's data. So unless you mount unique storage to each node, it may not be ideal. I'm new to NFS, but i would see if its possible to provision NFS dynamically using a storage class and see if the statefulset can issue PV + PVCs per pods using NFS dynamic provisioning. That way each pod will get its own NFS storage provisioned by k8s when you scale up and down

  • @MarcelDempers

    @MarcelDempers

    3 жыл бұрын

    Checkout NFS provisioning using Rook rook.io/docs/rook/v1.4/nfs.html Seems interesting. If you have not worked with Statefulsets I'd highly recommend to kzread.info/dash/bejne/rJ5q1MF-dcvcZtY.html

  • @derencevoo9146

    @derencevoo9146

    3 жыл бұрын

    @@MarcelDempers thanks Marcel, I have deployed Rook Ceph RBD in my cluster and now the applications now works like charm

  • @m1dway
    @m1dway2 жыл бұрын

    The tutorial doesn't work on my end. Keeps getting /var/lib/postgresql/data invalid permission. Im running docker with k8s on windows 11

  • @BR-lx7py
    @BR-lx7py Жыл бұрын

    Why does the table already exist @4:45 if you just created the volume. I'm guessing you edited out the part when you first created it.

  • @varunjain3870
    @varunjain38703 жыл бұрын

    You requested for 50mb you got 1gb :P :D

  • @MarcelDempers

    @MarcelDempers

    3 жыл бұрын

    I believe capacity stated on the PVC is that of the PV and not the requested of the claim :)

  • @bobfunk5055
    @bobfunk5055 Жыл бұрын

    Do people really prefer the background music ? Maybe you should poll your subscribers, just distracting.

  • @vshlmlvliveit
    @vshlmlvliveit Жыл бұрын

    Great video!

Келесі