I don't know why I am watching this for 2nd time. This guy is hilarious.

@ilyaezepov4141

5 жыл бұрын

2nd?! Those are rookie numbers, you gotta bump those numbers up!

I was really at a loss for words when I saw that inline plot at 2:53

good talk, but the conference organizers need to force people asking questions to wait for a microphone.

I have a programming background so restarting my kernel every and running things top to bottom was always common sense to me. I do agree though that it can be misleading and people may not make that assumption. I think one flaw with his programming style at the end that notebooks solve is having the history of how you checked your data throughout the process. This is really important for cleaning data sets. Unless you save everything you did in iPython along with your python code, it would be hard to trace. For writing models, pure python is definitely better, but for exploring data sets and fixing issues with the data itself, I don't know how notebooks can be beat.

@bulbulovo

3 жыл бұрын

"I have a programming background" Oh you do, do you? How unique!

@zgliu8018

3 жыл бұрын

@@bulbulovo there are many people only knew Excel before Python

@jiangli6498

9 ай бұрын

PyLaTeX

I agree with this dude. Unless you are a monk, you will develop bad habits using notebooks. I felt this most when I started out developing code for research and these bad habits wasted a lot of my time.

I was intentionally searching for this one. Not just me in the community who thinks it sucks.!!! This guy did the job....

For people who don't have good speakers. The text editor that the lady (46:27) and other people (49:07) mentioned was repl.it Atom extension which allows inline plots and other features: Hydrogen

@minhajsixbyte

3 жыл бұрын

Thanks 🙏

Joel is giving well thought reasoning to my feeling about notebooks. I always had these feelings but I couldn't point out what or why I did not like them. I totally agree with him

this guy's meme game is on point lol

Upvote me if you come here because of Fastai's post.

Very often the datasets we load are quite huge and it's time consuming to just REPL it. This is one advantage of notebooks, where we can annotate intermediate steps with the same dataset

@elMicoFilos

2 жыл бұрын

You don't need a notebook to reproduce intermediate steps. That's what REPLs are intended for.

Joel rocks, that is for sure. This made me quit notebooks about 6 months ago and have absolutely no regrets. I brought my data science to the next level. Thanks, Joel!

@ceosuji

3 жыл бұрын

What do you use instead?

Why was the speed set at 1.5x before uploading?

@DanHaiduc

3 жыл бұрын

To me it looks like he was panicking. And possibly on too much caffeine.

@RedShipsofSpainAgain

3 жыл бұрын

Caffeine is a helluva drug...

I admit I was fascinated with notebooks when I first encountered them. BUT, I was not developing code that might need to be broken into methods, tested, or placed in production. I was experimenting. Small stuff. Even in that setting, I have found myself migrating to a tool like VS Code. I'm open on what that tool should be except for me, it has to be free. Joel hits a lot of the things that I found bothering me. Extract methods. Intellisense features. Stuff most people who write code for a living will want and benefit from. Nice talk but he does talk fast.

I do not agree with half of his points. But it was insightful.

@NateGeorge-the-third

3 жыл бұрын

Many of his points are wrong - you can do everything he complains about with VSCode and nbdev. Possibly these things were developed since his presentation though, so at the time, it may have been accurate. This presentation explains most of it: kzread.info/dash/bejne/a4Vq1a6bqpadl84.html

FYI: all the editor (code completion) examples are solved when using JupyterLab with a running language server. But good talk, though, got me thinking.

Full agree with this talk. I never understood why a Notebook is go-to tool in Data Science. It is performance limiting tool. Actually, REPLs did exist for a long time in many languages (Scala, Haskell, Node, Java9, Swift) and they are useful. But we should not abandon IDEs for the sake of in-cell programming.

I am still quite new to data science and I've been using notebooks for my personal projects but listening to this I agree with him and shift my focus. However, I am a little confuse what if I am doing EDA? should I use a notebook or an IDE like Pycharm? because with EDA I am trying to understand the data and that requires me to plot tables and charts as well as code to aggregate data to understand it better. It seems faster to do that in a notebook than to create modules for them especially when viewing graphs isn't as nice in an IDE.

@prabhanshurajpoot7419

4 жыл бұрын

I have the same problem, I keep switching between spyder and jupyterlab. Trying to learn neovim. Any input would be valuable.

@TSchmoog

3 жыл бұрын

Hey I don’t know how you got on but for EDA you can use IPython

@OppaMack

2 жыл бұрын

Jupyter is a tool, for data exploration it's awesome!

No one else saw the loss reference? Just me? Ok, it's at 2:55. Can't miss it.

It seems to me that Emacs' org-mode and its noweb capabilities could deal with all the problems mentioned?

Rare to see such a great coder and presenter!

What I took from this talk is: notebooks are an interesting tool and they have their use cases, but quite often, people to beyond those use cases and use notebooks where another tool or workflow would make more sense. Kinda the "if all you have is a hammer" approach. I'm currently writing something that loads quite a lot of geospatial data and then does some things with it. I have never used geospatial data before. For me it's useful to be able to keep the data in memory so I can simply rerun the last step to see what happens, while keeping a record of the stuff I did earlier, while being able to create visualizations inline. But I'm very cognizant of the fact that this is causing me to write worse code than I could, because I feel a lot of friction to go back up and edit stuff or move stuff around. Though that could also be my lack of experience talking.

This guy should just use Pluto and he'd be so happy. I agree with all his gripes and they are all fixed in Pluto.

@PippyPappyPatterson

2 жыл бұрын

Can you point me to any resources that elaborate on Pluto's benefits over Jupyter?

@lukepikaart8323

2 жыл бұрын

@@PippyPappyPatterson well idk about resources as a lot of it is subjective. First of all the look is just way cleaner. But the biggest point with regard to this video is that there is absolutely no hidden state, blocks execute top to bottom so you don't get into some BS where you run the blocks in a different order and it will get you in trouble like in jupyter. Pluto auto detects which blocks have changed and therefore need to be re-run to update variables.

@PippyPappyPatterson

2 жыл бұрын

@@lukepikaart8323 Thanks! Do you know if they will ever remove the need for begin/end block header/trailers?

@lukepikaart8323

2 жыл бұрын

@@PippyPappyPatterson I agree it's kind of weird, but I don't think so especially because I've heard that Pluto is no longer being developed really because of some politics. I think Julia only wants to promote one main ecosystem and they chose VScode which makes sense as Pluto isn't really a development environment. But I don't understand why it can't coexist.

@flynnxd8229

2 жыл бұрын

@@lukepikaart8323 Where did you hear this?

For another perspective from Jeremy Howard(founding researcher at fast.ai, past president of Kaggle) see here: kzread.info/dash/bejne/a4Vq1a6bqpadl84.html Particularly for those new to Data Science, *just use whatever works for you.* You don't have to stick to one thing for everything. You don't have to dislike a tool just because someone convinces you to do so. If it helps you be more efficient, use it. If it disrupts your workflow, then don't bother with it. Particularly for Exploratory Data Analysis(EDA), Notebooks can be great. Also for tutorials, documentation, and demonstrations that describes how things work. But if you are working on more software engineering-focused and code heavy projects that doesn't require visualisations, then you might choose not to use Notebooks. What they usually say is, "Don't use Notebooks in production." And even that advice doesn't hold up all the time. *You just need to be aware of the weaknesses of the tools you are using.* Don't buy into the "real programmers don't ..." hype. And all well-known Cloud ML services are using Notebooks as their main interface to write code. It's not something you can escape. ;)

@elMicoFilos

2 жыл бұрын

"just use whatever works for you" and "[using notebooks] is not something you can escape". Both true, but in conflict with one another. I see why many people may find notebooks appealing, but not everybody finds them useful / fit for their workflows. So let's people use whatever works for them without forcing them to covert to notebookism.

be me, start a guide with jupyter, don't like it use another guide for idle, like it look for more guide for more knowledge about data science pandas numpy... and all that the guide uses notebooks.ai v (not working looking for advice on youtube and finding this video) v the guide actually says in description to use google colab instead cause notebooks.ai isn't working like it used to (and the guide is kinda new) why, a begginer programmer is expected while learning to use so many tools he does not and cannot fully understand when he absolutely do not need them? amazing presentation thank you for making me more convinced about learning using simpler tools

Damn ! This was good !

The 2 things you like about notebooks is the only 2 reasons I use it - I love markdown and I looooove plotting/instant pretty-printing in jupyter.

@Zarbissime

Жыл бұрын

You can plot & pretty print in vscode or pycharm without using jupyter, and use markdown separately for documentation, one good tool for dev & one good for communication, not one bad for boths

@DMSBrian24

Жыл бұрын

@@Zarbissime obviously you can do those things but not nearly as easily, and markdown integrated with code makes examples way easier to follow, ofc i'd never advocate for using it for anything serious but it's great for teaching/demonstrating

I haven't touched Jupyter since three weeks ago when I graduated with my Master's. Now that I'm trying to do basic operations in Jupyter again and it makes me so frustrated I want to scream, I do not understand how these things are so popular.

@transfo47

3 жыл бұрын

Since your comment a year ago, Jupyter has only cemented itself as the de facto standard in Python and Data Science.

For balance, make sure you watch the counter-presentation: I Like Notebooks kzread.info/dash/bejne/a4Vq1a6bqpadl84.html which shows how these things can be done with nbdev. Along with running notebooks in VSCode, almost none of these complaints are an issue any more.

Wow I'm really glad I learned python in Sublime Text and a command prompt

@marcofumagalli8147

4 жыл бұрын

Absolutely. I ve been using also Pycharm for 2 yrs. Can't figure out why people love so much notebook. Thank god I'm not alone.

@minhajsixbyte

3 жыл бұрын

Nice! But will you consider using a text editor that’s not proprietary closed source product please

@sorvex9

3 жыл бұрын

The notebook is very interactive, so thats why. Generally i like to use notebooks for EDA, or experiments and then write it as proper code afterwards.

humor and messages were on point - great job!

wait a second. isn't python extension only exsit for windows vscode?

Google colab also in jupyter format.

I'm no expert but I just feel like, for programmers especially, replicating this whole workflow of working with notebooks instead of the better one that they already have with IDEs just doesn't make sense. It's better for people who are less comfortable with coding I guess.

I think Joel is missing the point: I definitely agree notebooks are encouraging bad habits and are clunky to use but switching between your editor and your interpreter is plain wrong either. Just use your interpreter INSIDE your editor. You don’t need to restart your script every couple of edits, you can have the execution order you want depending on your current needs and you have a personal assistant which helps you with your mental reasoning.

Great motherfuking talk

This guy probably was unaware of using ipython console inside VSCode. Switching and remembering variables names is non sense. Nowadays your file is interactively running inside a kernel

brutal! I don't use them for the same reason. Tangling code snippets from an emacs org-mode notebook and running in the terminal is even better.

I love the memes!

I don’t like the questions part. No mic. No repetition.

I love this *don't like* talk.

The problem with notebooks is that you can only edit them with Jupyter and wow now Jupyter Lab. Files have a gazillion editors to choose from. But some how most people now choose to write code inside yet another complicated software called a browser, and it happened that it's a majority. They outcast us, everything now related to data science is served with F*** notebooks.

@turdsalami

2 жыл бұрын

I edit Jupyter files in vscode

@fojico1234

2 жыл бұрын

@@turdsalami and I edit them in Vim. These are hacks.

@taz3592

Жыл бұрын

@7up I couldn't agree more!

@isodoubIet

Жыл бұрын

@@turdsalami Vscode is also a browser

Thanks for the insight. Next time repeat/summarize your audience members' questions before answering them.

5 years passed and people still use notebooks in research and release buggy, unreadable and unusable code which really kills the point of releasing the code for reproducibility.

Yeah, something about notebooks. LOL

Notebooks are the reason I don't even want to waste my time learning Python or Jupyter. I truly hate that DOS-like command prompt.

@jimbobbillybob

2 жыл бұрын

You can use notebooks through VS Code and other IDEs/editors.

@asaboor

2 жыл бұрын

You would waste plenty of time without notebooks

tf? I just watched the whole presentation on mute and understood everything. that's some visual communication skills right there.

Funniest guy ever. His live coding is really great too (not as in funny, but as in interesting).

Vs code with anaconda Python enviorment!!!!

I hate notebooks and I wish I could press a button to delete them from existence. FFS use bash and vim and write scripts/ipython if you want to test stuff quickly.

@isodoubIet

2 жыл бұрын

Correct

@PolitiquePotontiele

Жыл бұрын

lmao dude just use emacs literate programming(orgmode).It allows ya to transfer data between cells written in different languages(like throw SQL directly into python without dependencies). It also has evil-mode(vim emulation), exporting to pdf and html(you can use custom CSS to make it look like Jupyter. Also it has todolists, links, inline pictures etc

Almost all the points apply to an ide as well. Would be better to stick to things that are true. Eg there’s no reason you cant write modular functions and classes in a notebook, i do it all the time. It’s literally the same as without the notebook There’s no reason you can’t do tdd in a notebook, i do it all the time. It’s literally the same as without the notebook There’s no reason notebooks are not reproducible. Click the restart and run all button, easy. It’s literally the same as without the notebook There’s no reason notebooks are incompatible with requirements.txt. It’s literally the same as without the notebook To your credit, there are reasons not to use notebooks, but throwing all these incorrect points in makes your point weaker, not stronger. I’m going to share a well kept secret: use each tool for what it’s best at. There is no silver bullet. ;)

@avialexander

2 жыл бұрын

"There's no reason that X isn't possible" is not the right way to think about tools. For example: There's no reason you can't use a refrigerator to warm your food up. It is possible. Why do you need a microwave? It's simple. Just pull the refrigerator out, try not to damage the flooring. Then disconnect the plug from the wall, most people don't utilize this hidden gem of a feature. Then bam, your food will start to warm up and you're golden! The problem isn't necessarily even that the feature isn't popular, or that it's misused often, or that it's hard to use. Those are symptoms of the real problem which is that jupyter is not built to do certain things well, or sometimes at all.

@MaxwellMcKinnon

2 жыл бұрын

​@@avialexander Exactly

I was forced to watch this :(

Apparently so far only 34 people like to use Notebooks

There is a difference between w notebook and the environment it is viewed in. Jupytext is a solution to writing notebooks in text editor. Complaining about lack of linting in a browser is just silly. A general solution is to use the right tool for the problem at hand. The prototyping example would've been quicker in the interactive notebook. Once you get your working code you can export to script and do the tests after that. Finally, badly written notebooks are not worse than badly written spaghetti code.

@isodoubIet

2 жыл бұрын

"Complaining about lack of linting in a browser is just silly." No it isn't.

This vid is overmemed. Really distracting. Had to view it twice.

"Not for me" - is the main massage of this talk with manipulative statements and extreme examples. That fellow could use his tallent to show good examples (in contrast to the bad code) rather than making fun of notebooks misuse.

just here for the memes

Why should anyone learn Python in Jupyter Notebooks? That's crazy! It's not made for this at all. Every tutorial which says that: buuuuuuuurn

@angelg3986

3 жыл бұрын

I learned Python using notebooks and find it wonderful and I now think about changing Python with Julia again with help of notebooks. I test everything first in a notebook, then in pure Python module.

@asaboor

2 жыл бұрын

That's should be the way to lean by practicing. It's painful to learn via files and much more painful in terminal

So long story short, dude was trying to (ab)use a tool for something it's not designed for and then complaining about it. Notebooks are first and foremost meant for exploratory work, when you want to have codes and plots side by side, or prefer to see a tabular dataframe formatted nicely, Excel-esquely. Notebooks are not meant to work as text editors, or to replace one. Neither are they meant to teach a programming language or OOP. So if you want any of the above, use another tool to fill in the gap, it's that simple. Though most of the issues he pointed out are valid, the tool itself is not the one to blame here but the users. Why the hell would one run PEP8 in a notebook? And if one's code is already bad in a notebook, how can you expect it to be way better if it had been developed in a text editor? And cell magics are not production ready, but for every cell magic there is always an alternate way of doing it without one, etc. Admittedly the memes are good for a laugh but the comparison is awfully lopsided it's impossible to take this guy seriously.

@bulbulovo

3 жыл бұрын

You don't get it, do you?

@asaboor

2 жыл бұрын

Now you can use notebooks to build and document python modules using nbdev. I wrote my first python module using this technology and it's amazing

@isodoubIet

2 жыл бұрын

"You're holding it wrong!"

The biggest thing I learned here is Joel has no idea how to use Jupyter Notebooks.

Cant watch this video! Notebook is a style. IDEs' functions are tools. It is not style's fault that the tool is not implemented the way(completion etc) you want it. Maybe I am at fault not watching the whole thing, but to me Joel is mixing apples and oranges all over the places. As a computer programmer, one should always keep in mind memory is your state. If you run things twice, don't blame the tools allowed you to screw it up. I.e., this is suggesting one way (your way) of doing things, from beginning to end. Why the world has to exist the way you want it? Would there be a possibility I DO want to run the cell twice, just to test things? If you agree there's other ways of accomplishing tasks, why would you call it bad? Can't keep on watching this nonsense...

I like notebooks. But this video made me think of some issues we have and also what we could do to alleviate these problems. With that in mind, I created a python package that addresses some of the issues raised in the video. Feel free to check it out at databooks.dev/ ! 🚀