How do I filter rows of a pandas DataFrame by column value?
Let's say that you only want to display the rows of a DataFrame which have a certain column value. How would you do it? pandas makes it easy, but the notation can be confusing and thus difficult to remember. In this video, I'll work up to the solution step-by-step using regular Python code so that you can truly understand the logic behind pandas filtering notation.
SUBSCRIBE to learn data science with Python:
kzread.info?su...
JOIN the "Data School Insiders" community and receive exclusive rewards:
/ dataschool
== RESOURCES ==
GitHub repository for the series: github.com/justmarkham/pandas...
"loc" documentation: pandas.pydata.org/pandas-docs/...
== LET'S CONNECT! ==
Newsletter: www.dataschool.io/subscribe/
Twitter: / justmarkham
Facebook: / datascienceschool
LinkedIn: / justmarkham
Пікірлер: 753
Most people would have simply demonstrated the short way, but by taking the time to elucidate what's happening behind the scenes, you're performing a far greater service. Great job.
@dataschool
7 жыл бұрын
Thanks so much for your thoughtful comment! I really appreciate your support.
@trs4184
3 жыл бұрын
Him: So here's how you filter out by certain values. Me: *Copies code* Gotacha, thanks. Him: But that's the long way and you'd never actually do that Me: Oh haha of course. *deletes code*
You say just the right words to get the concept and everything behind with lucidity. Thank you. i am a beginner but do not feel like so. :)
@dataschool
7 жыл бұрын
Wow, what a nice thing to say! :)
@samuelgeorge6756
6 жыл бұрын
I watch you at 1.5x :-D but like he said even a year later. Nice way to teach. Thank you.
@alexcipriani6003
5 жыл бұрын
you deserve my tuition money for that
@saimaacademy5537
5 жыл бұрын
kzread.info/dash/bejne/qKCalrKnadGzpco.html
@guidogaggl4020
5 жыл бұрын
Achin Gupta: my words. This guy is awesome
3 years later and this dude is still a legend and still saving my ass on basic stuff for work!
@dataschool
4 жыл бұрын
Ha! That's great to hear 😎
Everything you're doing here for us, it's gonna go back to you in a certain way in the future! Thank You a lot for this master piece!!
@dataschool
3 жыл бұрын
You are too kind, thank you! 🙏
You are the only man I can understand on KZread about Data Science.
@dataschool
5 жыл бұрын
Ha! Thank you :)
Thank you so much. I'm new to pandas as this was very helpful, I couldn't understand why it wasn't obvious how to filter rows. You explained it so clearly.
Awesome explanation! I have referred to some resources on pandas earlier and felt like i could easily skip your content to get through faster and grab only the unknown piece of code in the process . But the way you explain things it shows your hold on pandas and i am highly motivated to go through the full video. Cheers great job.
Your voice is so soothing and makes the understanding of concepts much easier and enjoyable. Thank You
@dataschool
4 жыл бұрын
Thank you!
When great teaching meets simplicity. Thanks so much!
@dataschool
4 жыл бұрын
Thank you so much! 🙏
There is something charming in how you say "Okay" when you finish a thought :) The video is very helpful. Thanks for putting this up here.
@dataschool
6 жыл бұрын
Ha! Glad my verbal tics are charming :)
@ludovicolaci9582
4 жыл бұрын
@@dataschool Kevin.verbal_function.finish_thought is this right?
It's imperative that you know the logic behind these codes. Thank you Mark for this amazing playlist.
@dataschool
3 жыл бұрын
My pleasure!
you are a great tutor, taking the long road and then giving the short answer really teaches the concept very well. Thank you sooo much
@dataschool
5 жыл бұрын
You're very welcome!
I loved how you showcased every possible way, from the heavy technique of using a for loop and then right to using operators. It made things absolutely clear. Thanks!!
@dataschool
2 жыл бұрын
You're very welcome!
You are the best Pandas teacher on KZread. I saw your previous videos as well but this is by far the best. You made the whole concept so simple. Thank you very much
@dataschool
5 жыл бұрын
Wow! Thank you so much for your kind words! :)
I like your in-depth explanation of _how_ it works, rather than a "recipe" of "just do it this way" without explaining _why_ it works. Now I'm off to see what other videos on pandas you have!
@dataschool
7 жыл бұрын
Excellent! I'm glad the depth of the explanation was helpful to you! My complete pandas playlist is here: kzread.info/head/PL5-da3qGB5ICCsgW1MxlZ0Hq8LL5U3u9y
Despite the fact that English is not my native language, these are the best lessons about the Pandas that I saw! Thank you!
@dataschool
6 жыл бұрын
It's great to hear that my videos have been helpful to you! :)
Can't believe you do the entire video with so little edits. Amazing!
@dataschool
5 жыл бұрын
It requires a lot of planning... plus it helps that I love teaching this subject and I know it well!
It's was THAT simple, yeah? I wasted two days trying to figure it out myself. Life saver you are. Thanks for the wonderful vdo.
@dataschool
6 жыл бұрын
You're welcome!
Thanks, great step by step learning resource! I already knew the quick answer, but not the reason why it was working :-) Very informative!
@dataschool
8 жыл бұрын
+Alessandro Sarretta You're welcome! I think you will find that understanding how it works in this case will help you to better understand lots of other pandas functionality!
The explanations are really well conceived and you speak so clearly that even the youtube captioner can capture it, subscribed!
@dataschool
6 жыл бұрын
Awesome! Thanks for subscribing, and thanks for your kind comment!
Love these, helped with something I was stuck on for 2 days. I'm a beginner, loads of love
@dataschool
4 жыл бұрын
Great to hear!
You are an amazing presenter. Knowledgeable, fluent, crystal clear. Much respect. Let me know how i can give something back to you.
@dataschool
5 жыл бұрын
Thank you so much! If you want to support me, I'd love for you to join my community of "Data School Insiders" on Patreon: www.patreon.com/dataschool
Many thanks for revealing the mystery behind the weirdness of the filtering syntax. I've been using it already, but now I realize the reason behind it, which is pretty cool!
@dataschool
7 жыл бұрын
You're very welcome! I find that understanding filtering helps you to better understand how to use pandas as a whole, which is why I decided to explain it. Glad it was helpful to you!
Your videos really are terrifically helpful, explaining things thoroughly from base concepts upwards as you do is just brilliant and your explanations are always extremely clear and precise. Thank you so much for all the time you spend on this - I'm tremendously grateful as I'm sure are many others.
@dataschool
7 жыл бұрын
Wow, thank you so much for your incredibly kind words! I'm very glad to hear that you have gotten a lot out of the series.
Love this video !! Exactly the one I was looking for. THANK YOU !... well explained..
@dataschool
6 жыл бұрын
You're very welcome! :)
he put step by step all the stones in my brain and set the wall! :D Simply great !
@dataschool
7 жыл бұрын
I love the metaphor... thanks for your kind words! :)
@yucelozyazgan
7 жыл бұрын
;)
Thank you. I believe by showing us the theory behind it, you're helping us remember the method for a far longer period
@dataschool
Жыл бұрын
You’re very welcome!
Such a pretty explanation. Now I discovered the 'Why' i use the bracket. Thanks!
@dataschool
2 жыл бұрын
You're welcome!
Very clear explanation and make me understand how the logic works behind the filter. Great thanks.
@dataschool
Жыл бұрын
You're very welcome!
Your explanations are so clear. Thanks
Great videos! I have been struggling with the ideas and concepts behind decision trees and ensembles. I hope these are topics that you will cover in the future and if not, I would really appreciate any resources to gain a deeper understanding of this topic. Thank you!
@dataschool
8 жыл бұрын
Thanks! Regarding decision trees and ensembles, I highly recommend chapter 8 of this book for a conceptual understanding: www-bcf.usc.edu/~gareth/ISL/ Here are videos related to that book: www.dataschool.io/15-hours-of-expert-machine-learning-videos/ For Python code and more resources, see classes 17 and 18 of my data science course: github.com/justmarkham/DAT8 Hope that helps!
Very nice thurrough explanation! 🙏 thanks
@dataschool
3 жыл бұрын
Thank you!
All the concepts very well explained. Your videos on pandas tutorial are just great !!
@dataschool
5 жыл бұрын
Thanks! :)
Thank you. Great content is still great, even 5 years later!
@dataschool
3 жыл бұрын
Thanks very much for your kind words!
thanks , I learnt more from your videos than other online pandas resources combine
@dataschool
6 жыл бұрын
You're very welcome!
Very nice tutorials. Easy to understand and to the point. Please prepare more for Data analysis.
@dataschool
7 жыл бұрын
Thanks for your kind words, and for the suggestion!
I've learnt so much through watching this one video. Thank you. Now, I may need to re-write some my existing codes to make it more efficient and simple to comprehend. :-)
@dataschool
5 жыл бұрын
Great to hear!
a big thanks for putting an end to my last 3 hours' confused state.
@dataschool
6 жыл бұрын
Great to hear!! You are very welcome!
Man. You helped million times more than pandas documentary..
@dataschool
6 жыл бұрын
Great to hear! :)
Explained with brilliance! Congrats.
@dataschool
2 жыл бұрын
Thank you!
Thanks !! I was looking for something very specific and 2 of your videos was just right on point ... just subscribed !!
@dataschool
6 жыл бұрын
Awesome! Thanks for subscribing :)
Bro, you are just great teacher! Please keep going what you are already doing. I'll buy whatever you produce! God bless you! :)
@dataschool
7 жыл бұрын
Thanks so much, I appreciate it! :) Right now, I only sell one course, called Machine Learning with Text in Python: www.dataschool.io/learn/ But if you want to hear about new courses that I release, just subscribe to my newsletter: www.dataschool.io/subscribe/
Sir,your content is the best for pandas. Thanks a Lot
Really great walkthrough, thank you!
@dataschool
3 жыл бұрын
My pleasure!
Awesome !! Your explanation is very clear and easy to follow. Keep up the good work !
@dataschool
7 жыл бұрын
Thanks for your kind words!
Expertly explained. Well done
@dataschool
4 жыл бұрын
Thank you!
Best way to explain, simple and accurate. Thanks
@dataschool
4 жыл бұрын
Thank you!
Many thanks for your tutorials. They are powerful. You make complex things easier. Thank you for your work.
@dataschool
7 жыл бұрын
Thank so much for your kind comment! I'm glad they are helpful to you!
amazing explanation. thanks for your time and patience to educate all.
@dataschool
6 жыл бұрын
Glad it was helpful to you!
You are very good teacher! Congrats!
@dataschool
5 жыл бұрын
Thanks!
The best ever Python site! Many thanks for time and effort !!
@dataschool
6 жыл бұрын
You're very welcome!
Saved me so much time. Thanks for this !
@dataschool
2 жыл бұрын
Great to hear!
Bro... Thanks for your great tutorials. I am beginner. From your videos i could understand the correct concept. Thank you very much.
@dataschool
5 жыл бұрын
Great to hear!
you are really a great teacher. It was so easy for me to understand pandas. Thank you so very much.
@dataschool
2 жыл бұрын
You're very welcome!
You're an EXCELLENT TEACHER. Thanks a heap!
@dataschool
Жыл бұрын
Thank you! 😃
Good afternoon. "Long time listener, first time caller." I really like your videos and your teaching style. As others have noted, you explain things clearly and in digestible and understandable chunks, which I appreciate. I have a question and/or a request for Q&A video. I am comparing dataframes. I am using a merge statement and a "fulll outer join". This works well for identifying records from both dataframes that do not match each other. Going a step further, I'd like to identify the individual attributes (i.e., columns) that do not match. My use case is that I often compare extremely wide datasets (200+ columns) and it is sometimes difficult to find the "offending/differing" column(s). I have researched at various places online, and have yet to find a solution that truly fits my needs.
@dataschool
4 жыл бұрын
Thanks for your kind words! That's an interesting question, I'm not sure if I 100% understand. It would be super helpful if you could code up a simple example (just a few columns) of what you are currently doing and explain exactly what your goal is. Thanks!
Beautiful teaching technique. You purposefully did it the long way first. Wow!. So the condition that I always used in [ ] in pandas is actually just a Boolean Series whose length matches the length of the dataframe.
@dataschool
7 жыл бұрын
Exactly! :)
Great teacher, practically my mentor haha
@dataschool
6 жыл бұрын
Ha! :)
I have learned much from your channel. You're a natural for teaching.
@dataschool
Жыл бұрын
Wow, thank you so much! I really appreciate your kind words and your contribution!! 🙏
A really understandable and good explanation!
@dataschool
3 жыл бұрын
Glad it was helpful!
Like your video. Very detailed and clear step-by-step explanation!
@dataschool
6 жыл бұрын
Thanks!
Great videos and very clear explanations, thank you!
@dataschool
6 жыл бұрын
Thanks!
loc helped a lot.
Awesome video, loved the intuitive explanation!
@dataschool
4 жыл бұрын
Great to hear!
You're an expert!!! Thank you so much!
@dataschool
2 жыл бұрын
Thank you!
movies[movies.genre=='Crime'] suppose I want to see both Crime and Action genre, then how to write ?
@dataschool
5 жыл бұрын
See this video: kzread.info/dash/bejne/i4R91siKaJrXmrA.html
Lots of love from India 🙏🙏 you have done a great job sir thank you.
@dataschool
2 жыл бұрын
You're welcome!
Thanks so much! Excellent step by step explanation of the concept and method! Your clear explanation shows how well you have those concepts and logic embedded in you brain 🙂 Must subscribe!
@dataschool
Жыл бұрын
Thank you so much, Murad! I really appreciate your kind words, and also for joining as a channel member! 🙏
This was such a good explanation
@dataschool
2 жыл бұрын
Thank you!
Excellent explanation!! Thank you
@dataschool
2 жыл бұрын
You're welcome!
such amazing simple explanation!!! thanks you!!!!
@dataschool
2 жыл бұрын
You're welcome!
You have outstanding skills to teach. Thank you.
@dataschool
6 жыл бұрын
You're welcome!
Never knew loc is a powerful filter tool!
Your series is terrific, only one issue, as has been mentioned before in these comments: "boolean" has 3 syllables.
@dataschool
5 жыл бұрын
Glad you like the videos!
Great video!! Helped me a lot!
@dataschool
Жыл бұрын
Glad it helped!
Another fantastic video, Im so used to the 8-9 min vidoes now , 13+mins was a bit intense :)
@dataschool
4 жыл бұрын
Thank you so much!
Great explanation as always
@dataschool
Жыл бұрын
Thanks!
You are an amazing teacher.... So clear and so succinct
@dataschool
2 жыл бұрын
Thank you!
That is so powerful explanation! Thank you!
@dataschool
5 жыл бұрын
You're very welcome!
Thank you very much sir for all your videos. its really a great experience and help to learn many things.
@dataschool
3 жыл бұрын
Thanks!
Boolean is three syllables. Thanks for your videos!
@dataschool
5 жыл бұрын
You are very welcome!
Very good tutorials. Thanks.
@dataschool
6 жыл бұрын
Thanks for your comment!
Awesome breakdown and explanation, thanks!
@dataschool
7 жыл бұрын
You're very welcome!
this is quite in depth explaining~~~awesome
@dataschool
7 жыл бұрын
Thanks!
thanks. was great help 1 points if u can tell: how to use multiple condition in the the data frame and select multiple columns of the same date frame
Lifesaver, i can't thank you enough!!!
@dataschool
2 жыл бұрын
You're very welcome!
Great explanation!!!
@dataschool
5 жыл бұрын
Thanks!
This was a great explanation! Thanks.
@dataschool
5 жыл бұрын
You're welcome!
I felt like my dog could understand this. Thank you
Hi Kevin! Thank you so much for the .loc tip! I was finally able to solve one of the things that annoyed me the most: assigning values to a new column based on values in one or more existing columns :) I sometimes prefer using .ix rather than .loc, since .ix is a flexible wrapper over .loc and can handle both label and numeric row selections. The syntax is strangely reminiscent of the R data,table [i, j, by] format, which I find very handy. Thanks again!
@dataschool
8 жыл бұрын
You're very welcome! I will have a video about loc/iloc/ix in the next few weeks :)
@asneogy
8 жыл бұрын
excellent - looking forward to it!
@dataschool
8 жыл бұрын
Here is my brand new video about loc/iloc/ix: kzread.info/dash/bejne/qqqksKNwkqaek9Y.html Enjoy! :)
@dataschool
8 жыл бұрын
You might also find my latest video to be useful, about the SettingWithCopy warning: kzread.info/dash/bejne/ZoZoudV9epCxhsY.html
your way of teaching is awesome!!!!! Thanks bro
@dataschool
8 ай бұрын
Thank you so much! 🙌
movies[movies.duration>=200]
@mattymallz4207
4 жыл бұрын
Thanks Molly!
@asgaraliyev8740
4 жыл бұрын
how can i do this by removing NaN
what a teacher, thanks.
@dataschool
4 жыл бұрын
Thanks!
This is very helpful! Your accent is so clear and it's very easy to understand you. Thanks!
@dataschool
5 жыл бұрын
Thanks!
glad I found your channel!
@dataschool
6 жыл бұрын
Glad you like it! :)
Very clear! Thank you.
@dataschool
2 жыл бұрын
Glad it was helpful!
pandasDoc < dataSchool = True
@dataschool
6 жыл бұрын
Ha! Thanks :)
Wow. Great video!
Awesome. Thanks for explaining concept in details. Gr8 work. Thanks.
@dataschool
2 жыл бұрын
You're welcome!
Thank you so much, it was a lot of help!
@dataschool
3 жыл бұрын
Glad it helped!