Starting in pandas version 0.18.1, you can create a new datetime column directly from a DataFrame, based solely on the column names! It's a useful trick, which I explain in this video: kzread.info/dash/bejne/X4KWu5l-acuxqM4.html

@cossanfran

6 жыл бұрын

Hello! I have a table with the date column. I want to group the data by month / year how do I do this?

@dikshyantthapa3367

4 жыл бұрын

I love you Bruh.. 😂.. No homo..thanks a lot!!!!

You've saved my job on multiple occasions sir, thank you.

@dataschool

3 жыл бұрын

That's awesome to hear! 🙌

I am an aspiring data scientist. I just found a series of your videos. Thank you for doing this for all of us. Keep doing great work!

@dataschool

4 жыл бұрын

Thanks for your kind words, and good luck to you!

Phenomenal description of working with dates and times in Pandas, very helpful.

2022 and you're still saving us. Thanks for the excelent content

@dataschool

Жыл бұрын

You're welcome!

I was actually going to ask you about this. I started learning Pandas thanks to your video series and I feel like I learnt a lot. Thanks for all the awesome videos.

@dataschool

8 жыл бұрын

Wow, that's really great to hear! You're very welcome!

@goktugerce

8 жыл бұрын

Actually, I still need some help. I want to create a column for "year/month" from datetime column. For example, if datetime is 2016-07-06 15:56:19, I want to map "2016-07" into a column. Of course I can get first seven characters by converting it to a string, but what is the correct way to do this? I have following the following lambda function: get_month = lambda x: '{}-{:02}'.format(x.to_datetime().year, x.to_datetime().month) my_df["year_month"] = my_df["timestamp"].map(get_month) For now, I do it like this but I am sure a better and more efficient way exists for the job. I'd be glad if you can help!

@dataschool

8 жыл бұрын

If your desired end result is a string (such as '2016-07'), then I think using string methods is the way to go! Perhaps something like this: ufo.Time.dt.year.astype(str).str.cat(ufo.Time.dt.month.astype(str), sep='-') However, there is probably an even simpler approach that I'm not thinking of...

@goktugerce

8 жыл бұрын

Awesome. It really is simpler than what I was doing. Thanks!

@archidar1

6 жыл бұрын

Hi, I know its a year late, but in case you (or anyone else is interested) "pd.Series.dt.strftime" is an easy way to output dates as strings in whatever format you like. pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.dt.strftime.html#pandas.Series.dt.strftime

I've only recently stumbled onto your videos. Very clear and concise delivery. Good job!

@dataschool

7 жыл бұрын

Thanks so much! Glad you are enjoying them.

loved the tutorial, cleared my doubts! I like how you explain so patiently.

All your videos are worth watching. I have learned a lot about pandas just from your videos. Thanks a ton :)

@dataschool

5 жыл бұрын

Thanks Aarti!

I understood each and everything in this video and it helped me alot for my project. I just want to thank the instructor.

@dataschool

3 жыл бұрын

Great to hear!

I really enjoyed this video and your teaching style! To the point, but well explained at a nice pace. Thank you for this :)

@dataschool

4 жыл бұрын

Thank you so much!

I've been watching your videos for a while now but never got the chance to comment on it i just want to say keep up the great work! You are just awesome!

@dataschool

5 жыл бұрын

Thanks very much for your kind words! Much appreciated :)

I'm impressed. Simple explanations with examples. subbed and hit that bell. Thanks for the vid!

@dataschool

3 жыл бұрын

Awesome, thank you!

I love the way you teach, easy to follow and to understand. Many Thanks.

@dataschool

4 жыл бұрын

You're welcome!

You are naturally born to impart knowledge. Thanks for your videos br!

@dataschool

4 жыл бұрын

Wow, I really appreciate your kind words! 🙏

Thanks for this! You explain things very clearly and concisely.

@dataschool

4 жыл бұрын

Thanks!

This is EXACTLY was a looking for. I love you.

@dataschool

5 жыл бұрын

Awesome!

So many great videos . Absolutely guidness. thanks from GREECE !

@dataschool

3 жыл бұрын

Thank you!

Thank you so much in taking time to explain so nicely.

@dataschool

3 жыл бұрын

You're very welcome!

@Data School , Mark, you are just amazing. :) You make it appear everything quite simple. My humble request is to kindle make a series on machine learning algorithms too.

@dataschool

5 жыл бұрын

Thanks for your kind words and your suggestion!

5 years video, but the best on KZread!!!

@dataschool

3 жыл бұрын

Thanks very much for your kind words!

Thanks so much. You are one of the best teachers I have ever known. Thanks so much once more you are a darling.

@dataschool

5 жыл бұрын

Wow! Thank you so much for your kind words! :)

That bonus is what I needed. Thank you so much!

@dataschool

3 жыл бұрын

You are so welcome!

Excellent explanation .I suggested these video series to most of my friends.

@dataschool

7 жыл бұрын

Thanks so much! I really appreciate you spreading the word.

Thnks for this explanation buddy! So clear and concise!

@dataschool

2 жыл бұрын

Thanks for your kind words!

These are the first videos I look for when I have pandas questions.

@dataschool

5 жыл бұрын

Thanks!

New to Pandas, new to your channel, and soon New year =) Thanks for the videos!

@dataschool

5 жыл бұрын

You're very welcome!

Data School is the best of all resource available on Pandas. thanks a ton!!

@dataschool

5 жыл бұрын

Thanks!

You have the best tutorials for Python.

@dataschool

4 жыл бұрын

Thank you so much!

You're king of this area man!!!!

@dataschool

2 жыл бұрын

Thank you!

Excellent class! As always! Cheers!

@dataschool

Жыл бұрын

Thank you!

I'm so thankful for your tutorials

@dataschool

5 жыл бұрын

You're welcome!

wow.. you're tutorials are just so awesome !!!!!

@dataschool

3 жыл бұрын

Thank you!

Very nice, just what I was looking for . Thanks!

@dataschool

6 жыл бұрын

You're welcome!

Hi DataSchool tk u for gr8 vid on working with dates and times. I am trying to work out how to group data for days, months and years in the same plot, e.g. bar graphs for months and different colours for the years

Very good video tutorials! Very much thanks, Kevin.

@dataschool

7 жыл бұрын

You're very welcome!

Hi, Very useful basics covered in your videos, thank you very much!

@dataschool

7 жыл бұрын

You're very welcome!

Big thanks, Kevin! Great job!

@dataschool

5 жыл бұрын

You're welcome!

Great job, bro! Thnx dude!

@dataschool

3 жыл бұрын

You're welcome!

thank you for this, I have a lot to learn

@dataschool

3 жыл бұрын

You're welcome!

Hi, I liked your way of presentation. It is very precise and nice. I would like to see any tutorial where you explain about taking statistics (mean, median, etc.) of data by filtering any column (say date-time column).

This is a great video. Thank you so much! Also, I wish the pandas API reference still looked like it does in this older video. It's harder to read now.

@dataschool

2 жыл бұрын

I agree...

Exactly what I wanted!! Thanks.

@dataschool

6 жыл бұрын

You're welcome!

Thank you, it helped a lot

Thanks, needed just this!

@dataschool

Жыл бұрын

You're welcome!

thank you so much for this video, it saved me so much time, thank you. Wow. so good.

Very helpful. Thank you so much bro

@dataschool

7 ай бұрын

You're welcome!

Hello sir, you are awesome teacher. Great videos Thank you very much

@dataschool

5 жыл бұрын

Thanks so much for your kind words!

Thank you very much. This has been quite helpful. :D

@dataschool

8 жыл бұрын

Excellent! You're very welcome :)

you are my new hero!

@dataschool

6 жыл бұрын

Ha! Thanks for your kind comment :)

THANK YOU! Very helpful video!! :)

@dataschool

5 жыл бұрын

You're very welcome! :)

You made my day, thank you

@dataschool

5 жыл бұрын

You're very welcome!

Thank you so much, wonderful explanation

@dataschool

6 жыл бұрын

You're welcome!

very informative, thanks as ever!

@dataschool

8 жыл бұрын

You're welcome!

Bravo man, really really useful. 👌💞🤩💪

@dataschool

2 жыл бұрын

Thank you!

Excellent videos.Please consider giving tutorials on time series forecasting ( with various statistical models ) with Pandas.

@dataschool

7 жыл бұрын

Thanks for the suggestion! I'll consider it for the future.

Thanks for the video, it was very helpful =)

@dataschool

6 жыл бұрын

Great!

Thanks for your great video! It seems pandas.Series.dt.weekday_name is removed in pandas version 0.23.0. and one can use pandas.Series.dt.day_name() instead.

@wilmarperez4962

4 жыл бұрын

Thanks for the tip. Was facing the same issue!

@Vishnu_Datta_1698

2 жыл бұрын

Thanks a lot

@jasper8291

Жыл бұрын

Thanks :)

Thank you for the video!

@dataschool

4 жыл бұрын

You're welcome!

Special thanks for your valuable work. Is the attribute "weekday_name" in panda version '1.1.3' now deprecated? and instead now we have 'day_name()' method?

You are gem bro. Thank you

@dataschool

4 жыл бұрын

Thank you!

thanks for the tutorial. is there any way to change the year column meaning change the year series. In year series - starts with 1930 and then go on till 1933, how to change this to 2013 to 2016 in the csv file ? thanks lot for the time and help ! cheers

Hi - qq - I have an excel sheet that has a column that includes dates, some of the dates have errors like '4/4/4/2020' or '/1/12/2020' - - is there a way python generate a dateframe column that lists all of these errors with their corresponding row information?

thank you this was really helpful !

@dataschool

7 жыл бұрын

You're welcome!

This is helpful, thank you.

@dataschool

6 жыл бұрын

You're welcome!

Always helpful

Thanks for the tutorial. I have a question. How can i find unique items under a given column as some could have been repeated?

I have a column containing times in the format hour, minute, second (e.g. 00:24:43) and are currently an 'object' type. How can I calculate the average time for this column? I have tried converting to a pandas datetime but this throws up several different errors when I try to calculate the mean? Also, I have three separate groups (full match, T1 and T2) how can I use groupby to figure out the mean times for each group? Thanks

Your video is really good, we'll be really helpful, if you make some more videos on Dates and Times. Thank you.

@dataschool

6 жыл бұрын

I cover it a bit more in this series: kzread.info/head/PL5-da3qGB5IBITZj_dYSFqnd_15JgqwA6

@debashissahoo5031

5 жыл бұрын

thank you, Now I am planning to cover this series too.

Great tutorial!

@dataschool

6 жыл бұрын

Thanks! Glad you liked it.

Kevin, could we define a ts_min and ts_max, and select the events during this interval?

Thank you for the input

@dataschool

4 жыл бұрын

You're welcome!

Hi, I have a data-frame with the first column as year(YY) with object datatype. How do I convert the column into YYYY format. Some years are before 1970 too.

@dataschool

4 жыл бұрын

Not sure off-hand, sorry!

Thanks for the awesome videos! Any thoughts on how I can use time series functionality with a data set that contains multiple rows with the same date? I mean I use day-level dates for one-year data, and every day may contain multiple observations (my data set has no unique column). Thanks

@dataschool

6 жыл бұрын

I think what you're looking for is resampling. Thanks for your kind comment, and good luck!

how can we handle ranges ( suppose salary ranges ) in a dataset? it would be great if any of you can tell.

Thanks for the video. Quick question: How can I change the ​data type of several columns at the same time?

@dataschool

6 жыл бұрын

Great question! See part 3 of this video: kzread.info/dash/bejne/X4KWu5l-acuxqM4.html

Great video, save a lot of code for me. Very good :)

@dataschool

5 жыл бұрын

Thanks!

Hi Data School, Just a quick question, I am still new at this and my apologies if this question as been answered already, how would I go about changing multiple columns to datetime? Thank you.

For some reason This through s an error ufo.Time.dt.weekday_name but ufo.Time.dt.weekday this works any idea.... Thank you

Hi Kevin, Just like we have 'weekday_name' to know whether it is 'Sunday', 'Monday' etc... what should we use to find the 'month name' ? I serched for help in pandas doc, but not able to find. Kindly suggest.

@dataschool

7 жыл бұрын

Great question! I'm not sure if there is a built-in way in pandas to do this. I would probably write my own code to do this using the map method, explained here: kzread.info/dash/bejne/gpOlktakidTcg84.html

I want to show the week of months as per the datetime columns. How can i do that? Please advise.

AB['Date']=pd.to_datetime(AB.Date) i am getting error Unknown string format: TOTAL my format is dd/mm/yyyy in column Date of a data frame named AB showing dtype as object .

Easy made. smart tutor

@dataschool

Жыл бұрын

Thanks!

great video. legend:)

@dataschool

4 жыл бұрын

Thank you!

Hi , I have a column in date/mm/yy i want to remove the yy can i do this ?

Hey thanks for the video really cleared so many doubts, I had one question tho, I extracted the microsecond from my date-time format and now if I want to plot my microsecond as x axis and all 3 other columns(that has float values) as y axis how do I do that?

@dataschool

6 жыл бұрын

I'm not sure, sorry!

Hello , I have a dataset with datatime index col. and it is weekly data , do I need to set freq='W' to apply forecasting models such as Holt-Winters, I tried : df.index.freq ='W', and got this error: OverflowError: int too big to convert"" please help me to fix this. Thank you

Love your tutorials! I have a DataFrame with multiple date formats in a certain column. I want to convert the values in the column to a consistent mm/dd/yyyy format. How would I go about this? Thank you!

@dataschool

6 жыл бұрын

Great question! I'm actually not sure the most efficient way to do this, I'm sorry!

For anyone who wants to convert timezones, here's how you do it. First, you need to specify which timezone the date originally belongs to. In my case, it was UTC (Coordinated Universal Time). In my DataFrame (let's call it df), I have a column named "timestamp", with type datetime. I localized the column first: timestamp_utc = df["timestamp"].dt.tz_localize("UTC") Then I overwrite the timestamp column: df["timestamp"] = timestamp_utc.dt.tz_convert("Europe/Istanbul") I hope this method is correct and it helps someone!

@dataschool

8 жыл бұрын

Awesome! Thanks so much for taking the time to share your code.

@TheWhatisupwithyou

5 жыл бұрын

Hi, I just tried your code but it returned all NaT values? my original column was in datetime format and UTC + x timezone. I tried to convert it to US/Eastern time. Update: Apologies, turns out the error lies in trying to convert my 'timestamp' equivalent column into date time from object and inserting errors=coerce produced the NaT values.

I am plotting a chart in that I need to show time only, how to trim time from pandas datetime ?

The videos is with very nice explanation .I am getting the time data only in hour:min:sec format and when i convert it from object to time then it gives also the current date with time stamp. I want to fill the missing seconds values so is there any other function available ?

@dataschool

5 жыл бұрын

I'm not sure off-hand, I'm sorry!

Love this course as usual. Thank you. I have a question. How could I do to "aggregate" rows by month or days ? (each row is a sell of a product over years)

@yop1001

6 жыл бұрын

Note to myself : use resample and learn to read comments (df3 = data.resample('M', on='date').sum())

Thanks for the tutorials. I want to compute the difference between two dates and return the result in integer. Much like the last example you showed. can that number of days be returned as integer? thanks

@dataschool

6 жыл бұрын

Actually, this already returns the result as an integer: (ufo.Time.max() - ufo.Time.min()).days

Hello, I have 2 large datasets and want to compare time differences by seconds for instance. I want to Group-by a certain column first, and then see the time differences or duration for a certain action. Can I do this in Python

@dataschool

4 жыл бұрын

I'm sure you can, but it's hard for me to say how off-hand. Sorry!

Hi Thanks for all your videos - what's the easiest way to get seconds since the epoch from a datetime object - Python 2 ?

@dataschool

6 жыл бұрын

Sorry, I don't know off-hand. Good luck!

awesome videos. been watching quite a few now. So, I'm playing with my gpx running data. And I'm trying to convert the duration of my runs so I can plot them. But I just fail. How would you convert ints like 33:28 and 01:44:42 so it would be understood as 33 minutes and 1 hour 44 minutes and so on?

@dataschool

5 жыл бұрын

Glad you like the videos! As for your question, it seems like extracting the datetime attributes (hours, minutes, seconds) and then doing the math with those attributes would solve your problem. Hope that helps!

If I want specific dates in train data and remaining in test data how can we do tht

It is interesting that while ufo.dtypes tells you the type as “datetime64[ns]”, when you look at each element individually type(ufo.loc[0, ‘Time’]) returns a Timestamp object. Seems a bit inconsistent? No big deal, I’ll just have to get used to it. With timedelta however, both the column and the individual elements are of type timedelta.

@dataschool

6 жыл бұрын

Yes, it takes some getting used to.... datetime is the type of the Series, timestamp is the type of the element, and timedelta is the type of a time range!

YOU'RE THE BEST

@dataschool

3 жыл бұрын

Thank you!

hi, i want a function in python that identify which column have date in them??