👷 Join the FREE Code Diagnosis Workshop to help you review code more effectively using my 3-Factor Diagnosis Framework: www.arjancodes.com/diagnosis

@ScorpusMugnes

3 ай бұрын

Hey, Arjan it's been a while now following your design guide videos. But the real problem is that, is it good to place more than one class in a single file, I have been coding in PHP where every class takes it's own file, the code is simple to refactor in that way. What are thoughts?

The naming of the ICECREAM module is a clever play on the abbreviation "IC", which phonetically mirrors "I See" in English. This choice is fitting for a Python debugging tool, as it elegantly underscores its purpose: to illuminate the inner mechanisms of your code, much like peering through a transparent layer in a typically opaque Python environment.

really nice list, thanks for sharing Arjan! i started using argparse recently in a data engineering context, to set up a main job executor and pass ETL jobs and params as arguments. between that and config libraries like omegaconf and hydra, it's completely changed my application design for the better.

A deep dive video on SQLModel would be great. I like it. On my first pass at learning it, the foreign key constraints were a little confusing and I ended up refactoring some code once I fully understood it. But seeing a video might be a great way to know if there is a better way to define them. I was doing a light project with a SQLite-based database, but ultimately want to get to a project that uses PostgreSQL with geocoding. If you did a video like that I would find it very interesting.

Love to learn about new libraries, you always find something new and exciting. Thank you Arjan!

Your very informative and nicely composed videos helped me get a job in this tough market. Thank you very much, and keep up the good work!

@Absalatreal

7 ай бұрын

woow congrats that's really asweome bro , mind if you mentor me in my journy ?

@ArjanCodes

7 ай бұрын

Wow! Thank you so much! I really appreciate it, and I wish you success with your new job! 🎉

Good to see a recommendation for argparse. Most of these modules are already used by me so I'm glad I'm on a similar track. For the question on command line programs, we create command line programs as the work I do is mainly backend and we try to automate as much of our repeatable and tedious processes as we can. Vendor software provides some functionality and our tools just enhance that to make maintenance seamless and fill the gaps not covered by the vendor.

A CLI app is always a good idea when automating boring tasks at work like building reports or extracting data to manipulate. Its quick to develop and fast to execute.

Great video, thanks! Here is a new entry: Ray is a nice framework with a Python API that facilitates code parallelization, whether it is on a single machine or within a designated cluster. Side note for people who might be interested: while Ray offers a user friendly experience, there is a certain degree of syntax overhead, making it challenging to run code in a serial manner without code modifications. To that end, I have build and published a wrapping library (still a work in progress), “ray-ease”, that allows to switch from parallel to serial without requiring code modifications.

Not a library, but I wanted to mention this: Python Poetry saved me so many headaches at work the past 2 months. I have been been a single dev working on some new infrastructure as code stacks with TerraforCDK and was having a rough time with dependency management. It simplifies virtual envs, dependencies and packaing. Def worth checking it out!

Very useful resource! I would definitely would like a video about Fast API & SQL Model. Thanks for all the great work.

In devops we use command line reporting and operation commands. Click and Rich packages are really awesome complements to what we do

Smashing video. Some of these I have never heard about before and will now take a look. Thanks.

Thanks Arjan ! A full video on SQLModel will be very appreciated..

@tomaskazutoshitaniguchi5180

7 ай бұрын

x2

I love Seaborn! I build CLI for data processing from our data warehouse, primarily using PyODBC.

I just found the abseil library for python and so far I really like it. I haven't completely understand all of the build abilities it provides by using bazel, but for building command line Interfaces I like it way more then argparse. You can define arguments as global flags und use or modify them anywhere without having to pass them down accordingly. Found this in a Google Project code and it instantly blew my mind

We add CLI to almost all of our projects for a couple reasons: 1) it's a quick and easy way to test out what is the proper interface/API to your applications before you invest into a more complex API tooling and 2) our applications are mostly pipelines (orchestrated ML lifecycles), not always-up applications. CLI is just a very robust and controllable entrypoint for that type of applications.

@jace3789

7 ай бұрын

Which PYPI project is this ?

We write software with command line interfaces all the time in scientific computing. I use argparse very often. Was glad to learn about tqdm, it looks very slick

I build command line interfaces all the time for data science / ETL utility scripts & processes, pulling data from APIs, roll-up aggregate jobs, interacting with cloud services, etc... argparse, tqdm, and pydantic are all super handy.

Since I've started to learn rust I found the Result type (railway oriented programming) an incredible programming mindset. The basic idea is that many of the exceptions we throw are not really exceptions, but just different paths that our program can take, so we shouldn't treat them differently. If that idea is well integrated into the language, is very powerful and allows to prevent issues.

@yash1152

7 ай бұрын

8:45 Result

@captainwasabi

7 ай бұрын

exception handling is just a GOTO by another name. Handling errors by result is a much better way.

@yash1152

7 ай бұрын

@@captainwasabi zig language for the win.

@Jackovasaur

7 ай бұрын

Learning Rust ATM and I chuckled when Result was discussed, especially with the pattern matching. Should definitely be a language feature in the std library

@isodoublet

7 ай бұрын

@@captainwasabi So are for loops, so is all of structured programming. Both exceptions and result types have their uses. E.g. operator overloading is utterly unusable without exceptions, might as well not even try. Exceptions are also much more convenient (and performant) when errors happen several nested layers deep. Also, result types are fundamentally unsuited to dynamic languages like python.

I write quite a lot of CLI applications, and personally I find the docopt library really neat.

This is super helpful, especially pydantic and icecream. Thank you so much!

@ArjanCodes

7 ай бұрын

Glad it was helpful!

One of my go-to libraries for adding 12-factor structured logging to my code is structlog. Works equally well as a replacement or enhancement of the standard logging library, and extremely flexible and extensible.

@yash1152

7 ай бұрын

what is 12-factor structured logging?

@MichaelONeillIrish

7 ай бұрын

​​@@yash1152pretty sure they just mean "structured logging" (logging all the data in a format like JSON) in a "12 factor app" (which is an architectural pattern for containerized/cloud native applications).

@yash1152

7 ай бұрын

hey @@MichaelONeillIrish ! ohkay, thanks a lot :)

@Jackovasaur

7 ай бұрын

I've recently started using structlog as a drop in for the std log library, and I can't go back. It has the features of rich + loguru in an easy to use interface!

@yash1152

7 ай бұрын

@@Jackovasaur but wount that mean that its not lightweight to run?

"I scream" (bad) -> "Ice cream" (nice). Debugging becomes nice.

nice selection! My favorite library is more-itertools, using it literally in any module

I had to use polars for one project due to Pandas being too slow. That project was an exception, and i am typically working on datasets where the speed of Pandas is not an issue. However, I still use polars when I can because I find the syntax so pleasant. I think its going to vary a lot between analysts (most’d rather use SQL than either library) but there’s a chance that for you, its best feature is not its speed but its syntax.

@Jackovasaur

7 ай бұрын

Pyspark also has similar syntax to Polars and tbh it's made using pandas a bit frustrating!

Thank you, Arjan for the learning - you are so informative. God bless!

@ArjanCodes

7 ай бұрын

Glad it was helpful!

Great list, I'm really interested in the FastAPI and Model DB combo. I'm a backend developer and I was thinking to learn Flask for that, but this seems better.

I really like click as an alternative for argparse. Much simpler to use and very powerful! Only downside is that it adds a dependency.

@solknar7819

7 ай бұрын

And even better than click: typer (based on click), developed by the same person as FastAPI and SQLModel :)

@ArjanCodes

7 ай бұрын

Thanks for the suggestion!

@TanayPrabhuDesai

7 ай бұрын

@@ArjanCodes Even better than Argparse and Click is Typer give that a try. Of course it's a layer on top of Click. Typer is made by the same person who made FastAPI.

@FransjeFranchise

7 ай бұрын

Click is cool but Typer is also great. Fastapi, SQLModel and Typer are all made by the same author: @tiangolo (Sebastián Ramírez). He's a wizard with leveraging type annotations for a great developer experience!

@bpetrikovics

7 ай бұрын

In my experience click makes the code slightly less readable and more confusing. I feel argparse more pythonic.

Love your vids. So much in-depth valuable info. Wish you would create a course from beginner to mastering python though. Get to work! :)

I use python for command line scripts - -replacing bash scripts. Command line tips are appreciated. Thanks and keep them coming.

I write CLIs all the time. I generally design everything like a library so it would be easy to attach any frontend to it, but CLIs are easy to crank out with libraries like Typer. At work, I write a lot of custom tooling for things like: managing/requesting TLS certificates, DNS management for customer domains, and other oft-repeated day to day tasks. At home, I wrote an application to automate organizing my media collections (with remote metadata retrieval). I'm usually in the terminal,and automating GUIs is a pain, so I almost exclusively write CLIs for my libraries and leave the GUI or web service frontend for others to implement.

@teodorlamort3864

7 ай бұрын

Fire is super convenient for simple CLIs!

@vitorbuxbaumorlandi6442

7 ай бұрын

I love Typer! Another great library from tiangolo

You, sir, are a new tag in my pinboard. So happy to have found your content, and am looking into your course.

@ArjanCodes

6 ай бұрын

Thank you for your kindness, John! Welcome aboard!

Thank you, very useful information for me. I use the terminal almost always, as a Linux user, I create an alias for the software I did and it's enough for me. But I must say I m over 60s and I just study and work with Python for my own fun, also I m very used to 80/90's terminal environment, so it's way better for me than using a laggy GUI.

Just discovered tqdm and used it in a recent project. Super simple!

Yes, you betcha I use the command line! Specifically, I have dozens of users who use my Makefiles to git/smbclient/git-select/xmllint/Bamboo configure/setup/deploy our developer environments. The Makefile figures out the host OS and installs OS dependencies using one of yum/apt/yast. It'll also use the pipenv files and specific Python selection to create bash shell launcher scripts. I'm particularly interested in rich-cli which give me a nice interface to either prompt a user for a branch or supply them documentation.

If you're contemplating creating a follow-up video, I highly recommend including a section on the Typed Argument Parser library. I've recently switched to using it instead of argparse, as it offers a more modern and type-safe alternative to Python's argparse library.

@virolana

6 ай бұрын

I'd be curious about that too. I have also heard click is great for creating command line apps. I have used it to deal with user prompt inputs and it made a complicated while loop much easier to read and user friendly in just a few lines.

@HavardSlettvold

6 ай бұрын

First thing I thought of when I heard argparse was TAP! It's such a simplification over argparse, and also allows scripts meant for CLI to be more easily tested and/or used from code, since you can construct the arguments very easily. I've also seen it used alongside click, but I have less experience with that.

If you do any calculations that requires units I will always recommend 'units' from 'astropy'. You can basically transform any unit to any unit (like nanometers to feets, if you need that for some reason) in an easy way. Pretty useful in equations where you are not sure if you have converted the units properly as well

@michaelcorvin1129

5 ай бұрын

FYI, 'pint' is a neat, standalone alternative to 'units' from the excellent 'astropy'. In a bit of testing I found 'unit' somewhat faster BUT either will add considerable overhead when doing computations using quantities with units. I slightly prefer 'pint' in usage. In any case, I recommend one use quantities for I/O function to validate inputs and to provide quantities with units as output, for doing dimensional analyses and validation through the compute chain during development and testing, but to only pass values from input quantities into production computations and then add units back to the outputs. The cool thing is that the computational chain (if implemented with the right abstractions!) should run either quantities or values without any changes.

Click is a nice alternative to argparse with less up-front configuration. (around 50% of the python code I write is command-line, one way or another, to answer your question - either backend services or actual CLI tools) pydantic's settings classes are really nice for 12factor-style config from environment too, with all the type-checking and validation from Pydantic (e.g. this parameter must be a valid URL or JWT token)

Thanks for the informative video... A video on SQLModel would be nice! What about a video about integrating between a REST API and a SOAP API using Zeep (Getting the data from both APIs, manipulating and comparing the data between the two to check if you should make a post, update or delete request to the REST API)? I think it should be interesting... It can probably just be 2 REST APIs as well...

@whu.9163

7 ай бұрын

For sure, because SQLModel has not finished docs yet and also it is not clear if this tool is production-ready like fastapi or pydantic

Awesome libs - if only this popped up for me a few weeks ago, would have been useful for my current project lol.

5:50 I typically build tools with dynamic menus that can show/hide options depending on circumstances. tqdm is one of libraries to show file transfer. For example I built file manager for my photography work that can automatically batch process newest photos into jpgs, save cropped frames and things like that. As for the logging I use my own function to print and write to file with preset formatting. Sometimes I make CSVs out of this so custom method helps. For me it’s more abaut information gathering than pretty colors in console

I write CLIs mainly to fill gaps in automation. Cronjobs, Deployment or Backup scripts etc. I usually write as a CLI, rather then a simple script. This means more work upfront, but is usually worth it once someone needs to execute it manually for the first time, and someone always will. I usually use click in combination with rich, so thanks for this list of libraries. I will check out argparse for smaller CLIs, since it will reduce the overhead, especially in regards to deploying the app.

Excellent video! I will definitely include some in my stack. I would like you to make a Sqlmodel video, I think it is an interesting option to consider

@ArjanCodes

7 ай бұрын

Thanks for the idea!

Thanks Arjan ,creating a complete video about SQLModel and FastAPI will be help full 💯

Amazing, super helpful !!! thank you for sharing.

@ArjanCodes

7 ай бұрын

You’re welcome!

in almost all of my projects: ic, pendulum, loguru, python-dotenv, rich. But i also learned some new ones! :)

Awesome collection. Please create a complete video about SQL Model + FastAPI . Thanks

@loudasaro4358

7 ай бұрын

💯

@evgenymaksimov

7 ай бұрын

I have a long video on sql model + fastapi on my channel, feel free to check it out

typer and pydantic-argparse are my favorite CLI parameters libraries 👌

Yes, I do regularly create commandline python apps, and I use argparse and rich :) The apps are all used exclusively in house in our company, and they are mostly wrappers for internal APIs.

I scream, you scream, we all scream for ice cream.

@yash1152

7 ай бұрын

2:13 only ur comment makes it obviuos why the name is icecream. its literally a take on "i scream". scream debugging lol.

@yash1152

7 ай бұрын

3:04 if not constrained by python version, then these 2 can be accomplished by new f-string syntax: print(f"{foo(123): }")

@yash1152

7 ай бұрын

but yeah, still, ic(foo(123)) is still shorter, and (hopefully) supports older python versions too.

@timoijas7496

7 ай бұрын

For example in pycharm you can create a live template for shortcutting. For example I made one where if I type log and press tab, it turns into print(f'|=') where the cursor will be at the | symbol. Literally makes the library obsolete reducing dependencies.

@JamesCairney

5 ай бұрын

It only makes sense as a name if you've heard the "I scream" rhyme. If you haven't its just a random dairy product.

Result looks very interesting to me, coming from the functional programming world, I could really use a Result monad!

Python's built ins now handle datetimes with timezones really elegantly with the default Datetime and ZoneInfo modules. Might be worth doing a video on those, a lot of people don't realize ZoneInfo is in the standard library.

I make mostly command line tools. I like to use typer for argument parsing when it doesn't conflict with my requirements otherwise I use argparse. I'm surprised typer wasn't on your list.

Wow, very helpful, thanks. About CLI I like to play with pyTermTk.

First thank you for sharing your knowledge, a video that includes both SQLmodel and FastAPI would be amazing!

Honestly happy to see Loguru on this list. Came across it like 2 years ago and much prefer it's simplicity over Pythons built-in logger WHILE still being able to combine both loggers AND maintain complexity as you need it. Brilliant package IMO. 🎉

I might as well also mention Maturin, which makes working with mixed Python/Rust codebases a breeze. I'm currently working on a general-use project template for such projects using it.

@ArjanCodes - I think that it would be very interesting to learn what you experience in other developers as far as weaknesses go. It is likely that a weakness you repeatedly experience is prevalent in other coders as well. Knowing these weaknesses would help other developers potentially learn from you and avoid the same issues.

Only just discovered your channel. Great content keep it up!

@ArjanCodes

7 ай бұрын

Welcome aboard, Dominik! Glad to have you here.

I have never heard that monadic exception handling is called railway oriented programming in some languages, thank you for enlightening me.

Thank you for your effort... I knew some of them but most of libs are new for me... And I wish you had time for another list of libraries that (groupby("use_area") ... For example financial data analysis or quantum physics etc. Thank you again...

KZread algorithm brought me here, and I found very some useful and interesting libraries unknown to me. Your video is really informative. Thank you!

@ArjanCodes

6 ай бұрын

Thank you for watching! I'm glad it was helpful.

Very useful. Thanks for this. Ditto on liking a separate video on SQModel.😀

- loguru can deal with multi-threading/processing. I love it at the first sight

Hi Arjan. This video was really cool 👍 I'm happy I know quite a lot of those libraries 😀 Great you mentioned SQLmodel. I'm starting new project with FastAPI and I want to use it, so I can't wait for video about SQLmodel. One thing based on my experience I want to share with you. Argparse? Noo 😅 you should definitely try alternatives like click (from flask author) or typer (from fast API author). This is like comparing datetime to pendulum 😅

Thanks, Arjan! This sort of "speed dating" video is usually lame,but you did the extra work of putting in the index and the links, so it's very useful. I would suggest one improvement, though: add a one-line description (just a phrase, like "Does error logging and adds debugging capabilities') to the libraries, because the names often don't give even a tiny clue as to what they're about. You asked about using CLI. I go back to the days of punched cards, and spent a lot of my career doing embedded systems, so CLI is my "native environment". For me, GUIs are what you use to make your CLI results look pretty ;-) When I took my first run at learning Python several years ago, I tried argparse, but found it confusing because it required a fair amount of understanding of Python,especially what you can do with various data types. I'm sure I'd see it differently now, especially if I were writing code for others. But I have decades of experience with getopt, both as a user and a programmer, so I use it in Python, too.

I really like you to dive into SQLModel and combine it with a easy way to present the underlying data in some sort of templating solution. Thank you!

@PhilipJoss

7 ай бұрын

Contrasting it with SQLAlchemy would be great too!

@hipertracker

7 ай бұрын

It's not stable and documentation is not great. What about Tortoise ORM or Piccolo ORM?

@MikeOnlineable

5 ай бұрын

Me too, is this similar tooling that is used by low code platforms to generate the database backend?

Thank you for revealing Pendulum. It helps me in my work.

@ArjanCodes

5 ай бұрын

I'm happy to hear it was helpful!

"Ice cream" refers to SoftICE debugger, one of the most popular DOS/Windows debugging tools of that era.

I use click on a regular basis. For me it's the best argument parsing library when it comes to big CLI tools.

Thank you, very useful information.

Thx for your videos! I would love a video about SQLModel!! Seb.

Very nice selection, many new to me. I actually stopped using argparse a while ago and prefer Fire instead.

Great list, thank you Arjan! It would be great to see comparison of SQL Model and Django ORM though.

@hobe4576

7 ай бұрын

...and sqlalchemy

@jeffgruenbaum

7 ай бұрын

These aren’t really comparable since Django ORM can’t be used outside of Django

@hobe4576

7 ай бұрын

@@jeffgruenbaum Hey, where is the spoiler alert - this is difference one! :)

@davorbokun

7 ай бұрын

@@jeffgruenbaum I'm not sure what you mean by outside of Django? It can be used without using a web server and you can access and manage your db through Django ORM in your own scripts and applications.

Hey man absolutely grate to learn from your videos thanks alot really, can you please suggest a usable stable supportive django library for work flow.

invoke is my go-to for building CLI tools. Hierarchical configs, zero argument parsing (!!), command chaining, self-documenting. Its so simple I build a little CLI for every project as executable documentation for processes, tasks, etc. Can't understand why invoke is not in every Python developer's toolbox. And fabric adds ssh on top so you can run invoke tasks on remote hosts. Maybe if Arjan did a video on invoke and fabric more people could enjoy the awesome.

I'm building CLI for the application initial setup. In addition to that, as You know, FastAPI has no built-in shell like the Django has, so I add my own commands with poetry + FastAPI + click.

Ice cream is likely a pun, or rather two of them: 1. Ice cream sounds like "I scream" which hints that it's a logger, though a bit louder. 2. It's abbreviation, ic is pronounced "i see" as in, ic(foo(bar)) "I see foo bar"

Very great list, this is the best channel to follow if you want to become a better python developer.

@ArjanCodes

7 ай бұрын

Thank you!

Excellent overview of libraries. This video has the batteries included.

@ArjanCodes

6 ай бұрын

Thank you for the kind words, haha! Glad you enjoyed the video :)

Interested in SQLModel video and how it compares to SQLAlchemy.

I am a data science practitioner and Polars, FastAPI is for sure great. Great list!

Polars is really a good tip and the user manual is written very well!

Some really cool packages in here that I've never heard of! Please do a video on SQLModel and how to do normal SQL operations such as groupby and aggregations. I tried using this in my code but I found it a lot slower than using Sqlalchemy. But I may have done it wrong. A video would really be amazing!

@adamcetinkent

7 ай бұрын

How does it compare to SQLAlchemy?

@Epic-Poetry

6 ай бұрын

A comparison between SQLAlchemy and SQLmodel would be great

Thanks for compiling this list of 15 useful Python libraries! It would have been even more beneficial if you could have included some examples of how each library can be used. That would help beginners and even experienced developers grasp their functionality more effectively. Nonetheless, I appreciate the effort you put into the video. Keep up the good work! 👍

Yes, I'd like a video about SQLModel 🙂

I regularly create cli applications for handling batch changes in cloud applications. It is just easier/faster to use the cli than to point and click through multiple pages on a UI to manipulate ymal files and things of such. My cli module of choice these days are Typer.

Hoi Arjan, mooie video, het zou inderdaad leuk zijn om een keer een video te zien over het gebruik van SQLmodel met Fast api. Je hebt het in een andere video over dataclasses gehad, is dat niet een beetje hetzelfde?

I use arg parse or py cli tools to dovetail my py modules / SDK into the build and containerization pipelines.

Sweet - picked up a few from the list for myself.

@ArjanCodes

5 ай бұрын

Glad to hear it was helpful!

Really enjoyed the video. And really enjoy python programming!

@ArjanCodes

7 ай бұрын

Glad you enjoyed it!

Thanks for this informative video, qq what do you think about Dash python Library?

Python command-line programs is great when working with docker containers, and when working with containers that process jobs and need input parameters.

tqdm is great - any iterable object can show its progress. I use it during scraping.

Yes, would be great a video about SQLModel

Thanks ! I suggest factory-boy to generate test cases :)

icecream may be a word play on ICE - In Circuit Emulator, a design tool from the past (nowadays In Circuit Debuggers are the norm, because transparent hardware emulation of a fast chip is hard or impossible) for embedded systems development. ICE's were expensive, a luxury way to develop and debug.

Thank you for your work

@ArjanCodes

7 ай бұрын

You are very welcome :)

Hydra is an amazing alternative to argparse that allows you to separate the arguments to yaml files

@Jackovasaur

7 ай бұрын

I've been using configargparse which kinda does the same, will check out hydra