Welcome to my field notes!

Field notes are notes I leave myself as I go through my day to day work. The hope is that other people will also find these notes useful. Note that these notes are unfiltered and unverified.

Makefiles

From its website, GNU Make is a tool which controls the generation of executables and other non-source files of a program from the program’s source files.

As a data person, Make has been intensely useful in creating [containerized data science workflows] that are reproducible and also lend themselves well to CI/CD tools like GitHub Actions, CircleCI, or Travis.

Custom Flags

Some flags allow you to modify the execution behavior of make.

- to not halt execution when command fails

docker-pull: 
    -docker pull tjpalanca/image

Adding the dash at the start allows you to “try” certain things that may or may not work in a CI context. In this example, we try to pull the latest version of the image to use as a cache, but it’s not an issue if it is unable to do so because we can always build from scratch.

@ to not echo code to be executed

docker-pull: 
    @docker pull tjpalanca/image

Adding the @ at the front inhibits echo-ing of the command that is executed, cleaning up your logs if you need access to it.