120K views · Oct 10, 2018 · 56:13 · 6 HN recommenders · 13 mentions · discussed 2018–2025
Joel Grus's JupyterCon talk explaining why he dislikes Jupyter notebooks. He covers hidden state, trouble with reproducibility and testing, and how notebooks frustrate teaching. He shows his preferred workflow and what Jupyter could change.
Why Hacker News points people here: It is the go-to funny, irreverent critique of notebooks, linked whenever HN debates notebooks versus testable code.
“The parent article links this great talk that goes into more detail than the parent post and is much easier to understand”HN comment, 2024
“they're definitely irreverent, but I find myself strongly agreeing with everything he says.”
Irreverent critique of notebooks the commenter fully agrees with.
Here they are - they're definitely irreverent, but I find myself strongly agreeing with everything he says. link this video
“I highly recommend it.”
Talk critiquing Jupyter notebooks against engineering best practices.
Jupyter notebooks are really abysmal when it comes to best engineering practices. There's an amazing talk called "I don't like notebooks" by Joel Grus at last years JupyterCon [0]. I highly recommend it. [0] this video
“I think both talks are very worth watching”
Critical counterpart to Jeremy Howard's pro-notebook talk; both show strengths and shortcomings.
If you haven't yet: I think both talks are very worth watching: I don't like notebooks ( this video ) by Joel Grus I like notebooks ( link ) by Jeremy Howard notebooks are a very interesting medium and they have both shortcomings as well as things they are great at and neither were totally obvious to me.
“I would recommend the talk”
Argues for moving code from notebooks into testable libraries.
Get extensive code out of notebooks into testable libraries. Confirm it works before running a big computation. We try to keep our notebooks slim and factor anything complex into a Python library. We can experiment with this libraries implementation, write unit tests on smaller data sets to confirm it works, before doing a bigger experiment. We can also deploy the library to prod and ensure users get the exact same thing we experimented with. I would recommend the talk “I don’t like notebooks” this video (There’s also the alternative nbdev that creates a full software engineering and experimentation setup in 100% notebooks. Not something I particularly like, but it might be another option.)
“it's a really funny talk, a lot of the points are IMHO invalid as Joel is misinformed on how things can be configured, but still a great watch.”
Funny talk on notebook flaws, though some points reflect misconfiguration.
First, yes, this is a common question. IPython does not try to deal with that, it's just the executing engine. Notebooks, do not have to be stored in ipynb form, I would suggest to look at link , and notebook UI is inherently not design for multi-file and application developpement. So training humans will always be necessary. Technically Jupyter Notebook does not even care that notebooks are files, you could save then using say postgres ( link ) , and even sync content between notebooks. I'm not too well informed anymore on this particular topic, but there are other folks at link that might be more aware, you can also ask on discourse.jupyter.org, I'm pretty sure you can find threads on those issues. I think on the Jupyter side we could do a better job curating and exposing many tools to help with that, but there are just so many hours in the day... I also recommend I don't like notebook from Joel Grus, this video it's a really funny talk, a lot of the points are IMHO invalid as Joel is misinformed on how things can be configured, but still a great watch.
“The parent article links this great talk that goes into more detail than the parent post and is much easier to understand”
Detailed explanation of why notebooks make state hard to track.
REPL stands for Read–eval–print loop link A REPL, by its name, is a very narrow version of the broader paradigm of interactive computer programming environments. But Notebooks are not REPLs, unless you use REPL to mean "interactive programming environment" and not REPL. Notebooks are much broader than a REPL! In a notebook, you can go back and edit and run individual lines of the notebook without re-running the whole notebook from the start and without re-computing everything that depends on what you just edited. Behavior like this makes it super hard to track the actual state, and super easy to lose track of how things are how they are. That's pretty terrible! The parent article links this great talk that goes into more detail than the parent post and is much easier to understand: this video
“Video of talk”
JupyterCon talk critiquing Jupyter notebooks.
You're not the only one. Here are slides from a talk that Joel Grus gave at JupyterCon 2018 titled "I hate Jupyter Notebooks": link Video of talk: this video
“The primary complaint there is that notebooks have a disconnect between the state of the program and the display of the cells.”
Talk listing gripes about notebooks, such as hidden state.
It completely fixes a huge number of the gripes in the JupyterCon talk I don't like notebooks by Joel Grus. The primary complaint there is that notebooks have a disconnect between the state of the program and the display of the cells. By using a completely reactive mode the state is no longer hidden. It's more akin to a spreadsheet than a notebook. A number of other complaints are completely circumvented by using a file format that's simply a pure Julia file with clever comments. Video: this video Slides: link Previous discussion: link
“I believe this is similar to what Joel Grus does [0], except %s/jupyter/ipython.”
Joel Grus's notebook workflow using ipython instead of jupyter.
I've been using the VSCode Insider's release as well and have been loving it and the new native notebook features for all the reasons you've listed already. That's an interesting solution. I believe this is similar to what Joel Grus does [0], except %s/jupyter/ipython. [0] this video
“I'm wondering what your opinion is of this talk”
Talk on why good software engineering is hard given bad examples.
I'm wondering what your opinion is of this talk: this video It has some comments about why good software engineering leads to good software, how good software engineering is hard right now because there's a lot of bad examples which cause people to make more bad code, and some suggestions on how things could be done.
“I'm also reminded of this video about the deficiencies of notebooks”
Video on deficiencies of notebooks, cited while defending their use for exploration.
I would love to see the tests that people come up for this... Tests can be helpful and informative, or they can be make-work boilerplate that will never reveal any flaws. And in this type of code base, it's hard to imagine the test that's written in code that will be informative. When dealing with data, "testing" is most often done visually, with plotting. The process of exploratory data analysis is not anything at all like writing code to an architected specification, and the same practices are not as effective for different types of work. Insisting that there should be automated tests for something like this reminds me of the cultural phenomenon of extreme table-phobia in the 2000s that happened as a reaction to using tables for layout. I had trouble getting people to believe that is was OK to use tables for tabular data, it was extremely frustrating. Or in the last 5-10 years, with design trends preferring spare layouts with lots of empty space, it was super hard to get information dense designs into production, because they were so averse to the idea that people may need information density for some things. I'm also reminded of this video about the deficiencies of notebooks: this video Yes, there are huge deficiencies to notebooks. But using them is an entirely different task than the type of code that one would write in an IDE, for code that gets put into a library and will be run by others. I have two work modes: 1) IDE (actually Emacs) where I write tests and the code is
“Jupyter notebooks are flawed (there's a great talk at”
Talk on flaws of Jupyter notebooks.
Yes, Jupyter notebooks are flawed (there's a great talk at this video ). But they are also very convenient for scrappy work and exploratory data analysis.
“I know I found it far more amusing than I should have when it was released.”
I'm assuming you've seen this video ? I know I found it far more amusing than I should have when it was released. I will confess that I found Mathematica kind of neat back in the day. I never got as good with it as peers did. I'm curious if that would be different for me today.
