From a programmer's perspective, one of the simplest and most flexible ways to interact with a computer is through a terminal. By only using plain text to get input and provide outputs, the program's interface not only becomes easy to reason about but also simple to reuse by other programs. This last feature is one of the main ideas behind the Unix Philosophy, specifically the rules of Modularity and Composition.
These types of programs are called command-line interface applications or CLI apps. They get started from a Shell process (e.g. Bash) through a Terminal (usually a virtual one like iTerm). In the case of Elixir, there are a couple of ways to kick off a CLI app but every one of them ends up creating a Beam process.
To better understand the default interfaces here's a high-level diagram of the CLI app running:
From the diagram we can see how the process (our program being executed) can interact with the operator (the person executing the CLI app) by using the stdin, stdout and stderr. The first one is used to get data into the process by either keyboard typing or another file stream, and the other two (stdout and stderr) outputs data. Here the programmer decides which stream to use for each case which usually end up having the stderr for general errors and stdout for anything else.
Thanks to the Shell we can pipe (connect) a process's stdout to another process's stdin to create a processing pipeline and through composition complete more complex tasks.
The are other ways to interact with the process that range from simple and common like OS signals (e.g. when the Shell gets a ctrl-c it sends a TERM signal to the foreground job) to more sophisticated IPC mechanisms. With the latter, we can interact with other processes in the same host or even remote ones allowing our program to perform more types of tasks.
CLI apps requirements
Now that we have a mental model of what a CLI program running from a Shell looks like we can start thinking about common CLI requirements to control what the process will do.
Get initial parameters: When a program starts it needs initial parameters to decide how to run (or not). These parameters can be provided by CLI command arguments, system environment variables, configuration files or any other mechanism programmed into the app (e.g. pull configuration from a well-known configuration server).
Run in the foreground: CLI tools are normally run by human operators (or indirectly via another script or program using it) and are normally expected to run in the foreground.
Prompt input interactively: Some CLI applications might need to get input interactively as they progress through their tasks. E.g. ask for root password to do sensitive tasks or ask for further configuration options.
Interact with the File System: Depending on the application, having access to the FS is important because large amounts of data are usually faster and more convenient to handle in files than using the standard streams.
Interact with other processes: With the shell or other running processes within the same host or remote ones.
This is not an exhaustive list and each application can require a different set of features to achieve its goals. In the next section we'll explore what tools exist in the Elixir ecosystem, what each option offers and what are their main downsides.
The main focus of Erlang/Elixir and specifically the Beam is building and running highly concurrent, fault-tolerant distributed systems, not CLI applications. However, that doesn't mean it doesn't offer a good starting point to cover use cases where a CLI app is needed.
In this section we'll go from the default solutions included by Elixir/Erlang to external projects that can be used to build full-featured CLI apps.
Elixir Scripts
The simplest tool to get the job done is Elixir Scripts. These are .exs files that are interpreted by the elixir command. E.g.
elixir my_cli.exs
It doesn't need any project structure (i.e. Mix project) and thanks to the Mix.install/2 addition (since Elixir 1.12) it can install external projects as part of the script execution. For inspiration check out this repo.
The simplicity comes with a cost, that could be acceptable depending on the use case, but in general they impose the following restrictions and limitations:
a. Needs the source code to run
b. It needs elixir to be installed on the host
c. Code organization doesn't scale well
d. Doesn't leverage Mix which is the default build tool for Elixir projects which provides tasks for creating, compiling, and testing Elixir projects, managing its dependencies, and much more.
One good use case for this type of solution is one-off tasks where the person who wrote the code is the same one who would execute it.
Mix Run
To easily enhance exs scripts we can create a Mix project and place the script within the project to let it use the modules defined in the project. From here it is easy to add dependencies, set up supervision trees, organize code in modules, add unit tests and much more.
To run the script from the context we just need to execute:
cd project_name
mix run my_cli.exs
my_cli.exs has access to modules defined under lib/ and all dependencies defined in the project.
This alternative has solved downsides c. and d. from the Elixir scripts but it still requires a. and b. . But don't despair, this is something we can address with some of the alternatives to be described.
Mix Releases
A release is a self-contained artifact that contains compiled code for the current project.
From the docs:
Once a release is assembled, it can be packaged and deployed to a target, as long as the target runs on the same operating system (OS) distribution and version as the machine running the mix release command.
This means they don't even require Erlang or Elixir in the running hosts because it includes the Erlang VM and its runtime by default. They don't even require the source code by default which can be convenient for some cases.
This is great! We've eliminated all downsides from Elixir Scripts but there's one limitation to be aware of: releases are optimized to run Elixir/Beam applications, not CLI ones. This means they work great as daemons but have limited support for foreground CLI apps. There are two workarounds to slightly overcome these limitations:
a. Eval a function:
bin/RELEASE_NAME eval "IO.puts(:hello)"
b. Call a remote function:
bin/RELEASE_NAME rpc "IO.puts(:hello)"
In both cases, we can leverage all the benefits from the Beam and the environment where the process is running but stdin, stdout and stderr as well as the command arguments can become a bit challenging to work with. Also, the eval function doesn't start any application within the program by default, and the rpc function requires the release to be running to be able to execute successfully. For more details please check out the docs.
Escript
Similar limitations already existed for CLI apps even before Mix Releases or even Elixir existed! The solution is called Escript and is originally available in Erlang.
Luckily for us we can build escript from Mix projects to create a single, largely self-contained executable! They can run on any machine that has Erlang/OTP installed, and it doesn't require Elixir to be installed by default as Elixir is embedded as part of it. However, it does require Erlang/OTP to be installed on the host.
Setting up and running escript is as easy as configuring mix.exs, defining an entry module with a main/1 function, and then executing the following command to build the artifact:
mix escript.build
From here, a single file will be available to be used as an executable. E.g. Assuming the app is called example_app we type:
./example_app
This looks perfect but it has one downside: it doesn't support projects or dependencies that need to store or read from the priv directory. A well-known library called tzdata is one of those libraries. However, there are workarounds to overcome this limitation but it does leave the feeling we can probably do better.
Burrito
To overcome most if not all limitations, and be able to produce a single binary artifact, there's a fantastic OSS library called Burrito. It lets you wrap your meaty app so you can delight your CLI app users!
From the README.md:
Burrito is our answer to the problem of distributing Elixir CLI applications across varied environments, where we cannot guarantee that the Erlang runtime is installed, and where we lack the permissions to install it ourselves.
Burrito uses Mix releases so we get all their benefits as well as a self-extracting archive. It creates a native binary for macOS, Linux, and Windows (*).
In the next part of this series we'll explore Burrito in depth to build a complete CLI app.
(*) This can be configured and cross compilation depends on the build host.
Honorable mentions
Even though these are not strictly speaking alternatives to build CLI apps they are mentioned here because they are alternatives to achieve some of the goals a CLI app can do. And in some cases they are better alternatives for niche cases. E.g. Mix tasks.
Mix tasks and archive: Even though these are not tools to create general-purpose scripts they are great alternatives when we need to extend mix tooling to improve our development workflow. Tasks can live within the same mix project and be reused by other programmers within the team and can also leverage archiving to install tasks globally.
Livebook: yeah, you heard me right! This is a fantastic environment where you can code and run Elixir scripts, use Kino to display charts, run machine learning models, and organize the code for a human to understand it step by step just to name a few.
Docker: When we want to have full control of the environment where our app will run without requiring changes to the running environment we can always count on Docker. In recent years it has become ubiquitous so any CLI app that can accept the extra delay of running its app via docker can wrap the CLI with it and distribute it as a docker image.
Coming up Next
In Part 2 we'll explore each tool we have described to implement a well-known CLI app to see for ourselves where the benefits and limitations of each alternative are, with the hope you'll end with enough knowledge to decide what tool fits best for your future use cases 🚀
