Ritchie Vink - Keynote on Polars Plugins

Ғылым және технология

Machines have changed a lot in the last decade and Polars is a query engine that is written from scratch in Rust to benefit from the modern hardware. Effective parallelism, cache efficient data structures and algorithms are ingrained in its design. Thanks to those efforts Polars is among the fastest single node OSS query engines out there. Another goal of polars is rethinking the way DataFrame's should be interacted with. Polars comes with a very declarative and versatile API that enables users to write readable. This talk will focus on how Polars can be used and what you gain from using it idiomatically. Most importantly, this talk introduces Polars Plugins, a novel way to define your UDF's in Rust and register them in the main Python polars' engine. This will grant your specific business logic with - Rust performance (The familiar vectorization in python). - Paralellism (orchestrated by the polars engine) (No GIL Locking!) - Optimizations (The optimizer will use the properties of your UDF to elide work).

Пікірлер: 2

  • @zetendra
    @zetendra2 ай бұрын

    thank you so much for sharing with us!

  • @OierVillaluengaMartinez
    @OierVillaluengaMartinez3 ай бұрын

    I just wanted to post to say thank you for creating this wonderful library. Its popularity is well deserved and keep on going with the good work! It has enabled me to speed up data processing. And as your motto says, I came for the speed but I stayed for the syntax, just with few methods you can do a myriad of operations at speed. It's very refreshing to see libraries like this.

Келесі