Welcome to my field notes!

Field notes are notes I leave myself as I go through my day to day work. The hope is that other people will also find these notes useful. Note that these notes are unfiltered and unverified.

nbdev

From the website: nbdev is a system for exploratory programming. Simply write notebooks with lightweight markup and get high-quality documentation, tests, continuous integration, and packaging for free!
Author

TJ Palanca

Published

August 9, 2022

Initial thoughts

  • I’ve always been highly skeptical of using notebooks for software development, as at the time data scientists were notorious for pumping out bad code, and we were shoehorning notebooks (especially non-plaintext jupyter notebooks) into any which workflow.
  • This is interesting because it’s been under development for a good 4 years and just underwent a major version overhaul. A substantial number of people must find it useful. Coupled with the claim of 2-3x more productive, that definitely makes it worth revisiting.
  • With the addition of quarto, that paves the way for cross-language support.

Watching the walkthrough

  • It seems that instead of just using this for analyses-turned-packages, you would actually use this for building software; there’s a language extension fastcore, and even an api client ghapi built using this method.
  • It’s actually quite interesting because instead of using notebooks to describe the development of an analysis, you would use it to actually describe the development of a function then use #|export directive to place it into the final “package”.
  • Uses a central settings.ini file to record all project metadata. .
  • They want you to think of nbdev as a dialect instead of just a module, because it does depart from common standards like PEP8, etc. The most glaring example is widespread use of wildcard imports.
  • Github actions that are set up allow the documentation to be rendered to github pages
  • Commands:
    • nbdev_new to initiate
    • nbdev_export to export the code from the ipynb docs to the package
    • nbdev_clean strip any metadata
    • nbdev_docs updates the README.md to contain the same contents as the index notebook.
  • Package:
    • nbdev.showdoc.show_doc prints neat information from the docstrings and type annotations.
    • [email protected] allows you to add methods to classes that have been defined (useful if splitting the data into multiple cells).
    • nbdev.test.* contains some testing utilities. As far as I know {pytest} is not supported yet.
    • %debug magic in jupyter allows us to breakpoint into any part of the python code.
What does that mean for package managers like poetry? Turns out it doesn’t really merge quite well.

Practical Walkthrough

  • I started trying to port over my legislative analysis to nbdev style.
  • I can easily see this being super super useful.
  • Biggest issues:
    • My favorite dependency manager {poetry} isn’t really well supported. There are a bunch of duplications needed between the settings.ini and pyproject.toml. The dependencies needed to be specific in the settings.ini file in a very difficult way and installing into an environment was not that practical.
    • There was a lot of boilerplate which I generally don’t like.
    • While I am a big fan of {quarto}, the default themes didn’t really seem that readable, to be honest.
  • In the end, I decided I’m not going to adopt {nbdev} but I will definitely watch out for any updates.