Environ variables are not for configuring software

The recent project I’ve finished and deployed gave me a strong understanding why using environment variables for configuring is a bad idea. Briefly, I’m going to not use them anymore for setting up my software and maybe clean them out from my previous projects. There is a pile of reasons for that.

Generally speaking, env vars aren’t bad in general. I’d just like to highlight that they come from the ancient Unix time when people didn’t have suitable development tools. But nowadays we have some. For example, thanks to JSON or YAML, we do not need to parse integers or floats manually. The same about arrays or maps. Some better formats like EDN even provide built-in date support or even custom tags to parse special values. With these great tools, there is no need to roll back for 20 years ago and parse text chunks once again. That’s the routine we’ve successfully passed.

In short words, a config is usually a structured data. And we’ve got stuff to read data from text without any problems. Why use env vars then?

You might be a bit wondered reading this since it conflicts with the third section of 12 Factor App Development Manifest. This section clearly says:

The twelve-factor app stores config in environment variables (often shortened to env vars or env). Env vars are easy to change between deploys without changing any code; unlike config files, there is little chance of them being checked into the code repo accidentally; and unlike custom config files, or other config mechanisms such as Java System Properties, they are a language- and OS-agnostic standard.

Well, I read this paragraph a number of times but still guessing is there any sense out here. Easy to change? Well, you’ll need to edit some file or config anyway. Little chance to commit them into a repository, really? If you have an ENV file with 30 variables, there is no any difference between it and any other file in your repository, say “config.json”, “settings.prop” or “params.conf”. ENV file is just yet another file without any special properties. You still need to control by your own whether it should be ignored or tracked by your repo.

The whole #3 section says nothing about what benefits the env vars bring into a project. From my prospective, since the 12 Factor App was initially written by Ruby programmers, there is nothing else then just a cargo cult.

The idea of getting rid of a regular JSON, YAML, Java .props config file is totally wrong. Of cause, if a program requires one or two arguments, there is no a reason to distribute a separate config file. Especially when those parameters have default values and might be overridden by command line arguments. But the problem is, our modern web apps require up to 30 parameters at once and even more.

Definitely, you’ll keep them in a text file, say “ENV” or “vars”. So what’s the difference here to have a JSON file?

If you take cprop, a Clojure library that turns env vars into a map, you’ll see that it provides a bunch of tricky rules for types coercion and splitting nested values. For example, “true” and “false” strings will be converted to a boolean value. A variable with a double underscore will be read as a nested map:

export FOO__BAR=42

{:foo {:bar 42}}

That tricky logic may let you down sometimes. Imagine you’ve got a secret API key for some service that it a string but consists only of numbers:

export API_KEY=123123123

In Clojure, the result would be a number, not a string:

{:api-key 123123123}

There won’t be any error during parsing, but suddenly, in the middle of running an app, you’ll get an exception that says something like “the Long type cannot be cast to a Char Sequence”. It’s because the code tries to act on that API key like it’s a string, but it’s an integer.

You should have marked that key with double quotes instead:

export API_KEY='"123123123"'

Well, that’s a known issue and it’s even mentioned in the official readme, but still. I can never remember the proper order: single-double or vice versa. I don’t want to fight with those damned quotes again. It has been enough in shell scripting: single quotes, double quotes, back-ticks. Now again in my Clojure? No way.

In terms of cprop, there is nothing weird here because it doesn’t know anything about the variable’s semantics. They are all just text and nothing else.

I’d like to stress that we’ve just invented our own system of rules and wrote plenty of code just for one purpose: to turn env vars into a map. It’s plain to see the code behaves a bit strange and forces us to keep that ugly scenario with quotes in mind.

Even a dull JSON file that supports a limited number of hard-coded types seems to be a salvation after been using env vars for a long time, really. A string will always be a string and won’t turn into a number. There is no need to add a double underscore in the middle of a name to simulate nested data.

But the main reason for getting rid of env vars is they tie us to all that legacy that our systems still need to carry. Let’s say honestly, env vars are just another part of Unix shell and related stuff. I do not have anything against them when they are used in the right way. For system administration, for example, but not for web development.

I mentioned before that a program should never rely on shell commands. Never call mkdir, curl or any other commands. You may think it reduces the code, but in fact, it’s a deal with the devil. Java code that creates a folder will work on any OS or device as expected. But a shell command won’t.

The same I may say about env vars. They are part of a shell system and that’s why are great for OS environment, but not a web application. They should never affect an app running in production. The codebase should rely only on its own resources. From the app’s prospective, a JSON/EDN config file is something that an app may control. But the env vars are not.

I think that’s enough said on the subject. Provide your own config based on any structured file format, no matter if it is a JSON, YAML or EDN file. But leave the env vars alone. They are from Unix legacy and have nothing common with modern development.