I moved my entire stack to Polars from Pandas in the past month. It is more intuitive (I am a relatively new programmer), much faster, has a cleaner API and integrates better with databases. If you are just getting going on data analysis you should start with Polars. And there are plenty of resources out there to help you if you want to migrate.

I have been heavily using pandas, polars, and duckdb and heres my two cents. 1. If you need dataframe to work with, you should go with polars no doubt. For certain cases, you may have to convert df from polars to pandas but that’s quite rare and that should be done at the end anyway if necessary. 2. If you need fast query to analyze, go with duckdb. You can directly use polars and pandas objects in memory to crunch data. Very light, fast and efficient. 3. Know how to use all these stacks. Don’t just stick with one only.

Great timing, I was just looking for a polars video.

Thanks - I rely on videos like yours to keep up with what's new.

I also do enjoy your occasional more exploratory and less explanatory videos. They are a motivation to go out and explore the "wonders of the world" and acquire the more in-depth insight by oneself. This applies to both Polars and Huggin. Thank you for having drawn my attention to both. But I also do appreciate the videos in which you take the viewers by the hand. In particular with polars a hands-on video would definitely interest me. Your channel is amazing, rich, in-depth, no-nonsens. By now your some 800 Videos provide more information than I was presented with in years of studying in the 80s at your same university. You truely enrich the world. Big achievment at your age. We are living in truely exciting times. A bottom up approach based on first principles has for a long time been a driver for human intellectual developement, I'd say ever since Newton. AI allows us to tackle problems which are either not governed by first principles such as Newtons, Maxwells, Einsteins, Feynmann laws, maths and mathemetical and logical proofs or where following the way from first principles to tangible and useful application is beyond our scope and maybe possibly forever. It allows as to deal with chemistry and biology, consciousness. There is a lot more to AI than just writing amazing programs like the ones you show. I wish to live on for quite another while to see this new form of thought evolve. Your videos make and keep me curious. I see things today I'd have considered SciFi until very recently.

I started using it this month, even taking documentation from pandas projects and translating them with cluade and ChatGPT. Still use datareader to pandas to polars.

This is an excellent topic for a vid. I've not switched yet. Pandas2 is pyArrow just to reiterate the point, but it's not distributed. My bet is that pandas2 will catch up at pandas3, it's got a massive following and under active development. I've attended one of original pandas public meeting prior pandas2 and their direction of travel is clear. Agreed on the apply function in pandas and its a great point to make in the video.

Keep it up 💯

Given the good press, I've tried to use Polars on a huge dataset. Unfortunately, I couldn't even read my data in, as no API was available. There isn't a file extension that you won't be able to read with Pandas. At this point, Pandas remains the better choice given its compatibility with the Python data science ecosystem.

Wow! Mindblowing stuff... Thank you so much for this... Perfect tutorial...

Thanks. Are there any Technical Analysis libraries please ?

@tthcan8038

16 күн бұрын

I guess TA-Lib is what you are looking for

@TheEcherriman

16 күн бұрын

@@tthcan8038 Yes, the equivalent of pandas-ta

That polar case statement looked a lot like a SQL statement

I see a transformative package so that code can remain unaffected and all that changes is the import.

Duckdb is nx faster pandas 😅. Create one more video about Duckdb

Polars wants to be Tidyverse when it grows up😮

Pandas has a ton of useful features that polars lacks. Last time I used polars, reading in excel files and dealing with dates, especially dates with inconsistent formatting, was a major hassle. People like to shit on the pandas API but I don't find polars that much better. Good thing there's duckdb that you can use on top of both. Troubleshooting will be much easier and faster with pandas due to its immense popularity. It almost feels like polars is for the rare edge case where you have too much data for pandas to handle efficiently, but not enough data to warrant using distributed computing. Good for pipelines in certain cases but not necessarily EDA imho.

@thyche7866

16 күн бұрын

Which edge that polars had over pandas that do you think its better?

please , data engineering with Polars using rust!

Its statement are more familiar with pyspark one

0:40 and also it is easier to use and more human-readable

Thx_.

Polars uses multiple cores

@dipeshsamrawat7957

16 күн бұрын

Thanks to mention it. 😊

@user-pg8qe1zo4j

15 күн бұрын

Well if we work on low end cpu will it make much difference

Why bother? 😅 Obvious waste of time that you gonna revert right when tou encounter some unsolvable issue

@kmbelly6356rt

14 күн бұрын

exactly❤ not every shiny stone is gold.