This is part of my library of evergreen notes. It is continually updated as I refine my thinking around a particular topic.
Makefiles
From its website, GNU Make is a tool which controls the generation of executables and other non-source files of a program from the program’s source files.
As a data person, Make has been intensely useful in creating containerized data science workflows that are reproducible and also lend themselves well to CI/CD tools like GitHub Actions, CircleCI, or Travis.
Custom Flags
Some flags allow you to modify the execution behavior of make.
-
to not halt execution when command fails
docker-pull:
-docker pull tjpalanca/image
Adding the dash at the start allows you to “try” certain things that may or may not work in a CI context. In this example, we try to pull the latest version of the image to use as a cache, but it’s not an issue if it is unable to do so because we can always build from scratch.
@
to not echo code to be executed
docker-pull:
@docker pull tjpalanca/image
Adding the @
at the front inhibits echo-ing of the command that is executed, cleaning up your logs if you need access to it.