Data-Driven Development is a Lie

UPD: there is discussion on Hacker News on that article. Thank you Mike for letting me know.

In the Clojure community, people often discuss such things as data-driven development. It is like you don’t write any code or logic. Instead, you declare data structures, primarily maps, and whoosh: there is a kind of Deus ex Machina that evaluates these maps and does the stuff.

That’s OK when newcomers believe in such things. But I feel nervous when even experienced programmers tell fairy tales about the miracle that DDD brings to the scene. That’s a lie.

I’ve been doing Clojure for nine years, and DDD is useful in rare cases only. Yes, in some circumstances, it saves one’s time indeed. But only sometimes, not always! And it’s unfair: people give talks at conferences about how successful they were with DDD in their project. But they would never give a speech about how they messed up by describing everything with maps.

Let me give you an example. Imagine we’re implementing a restriction system. There is a context, and we must decide whether to allow or prohibit the incoming request. Obviously, every Clojure developer would do that with maps. We declare a vector of maps where each map represents a subset of the context. Should at least one rule match the context, we allow the request.

Something like this:

(def RULES
  [{:some-field 1
    :other-field 2
    :third-field "foobar"}
   {:some-field 3
    :other-field 40
    :third-field "sample"}])

(defn matches? [ctx rule]
  (= rule (select-keys ctx (keys rule))))

(defn allow? [ctx]
  (some (fn [rule]
          (matches? ctx rule))
        RULES))

Above, the RULES var is declared on top of the module, but ideally, it comes from an EDN file.

It looks short and solid. Moreover, This! Is! Data! The next time a business wants us to add a new rule, we will extend the vector with a map. At this point, a programmer who has implemented this gets a ticket to the nearest conference and gives a speech about their success. This talk gets shared across the community channels. This is how the cargo cult grows, in fact.

As I mentioned, none of the YouTube talks tells about what happens after the developer returns from the conference. There are upcoming changes in the logic: a certain field might have multiple allowed values. For example, the :role is either an admin or a manager. The developer scratches his head but quickly finds a solution: let some rule values be a set. If a value is a set, we process it using the contains? function. Otherwise, the values get compared as usual.

(def RULES
  [{:some-field 1
    :other-field 2
    :role #{"admin" "manager"}}])

;;; when matching the fields
(let [...]
  (cond
    (set? value)
    (contains? (get ctx k) value)
    :else
    (= (get ctx k) value)))

Good. But in a month, there is a new requirement: negation. Now we allow the request if the :role field is of any value but not manager. Again, the developer gets puzzled for a while, but then he finds the solution with the [:not ...] clause. If it’s a vector and the first item is :not, we use not= instead of =.

(def RULES
  [{:some-field 1
    :other-field 2
    :role [:not "manager"]}])

(let [...]
  (cond
    (set? value)
    (contains? (get ctx k) value)

    (vector? value)
    (case (first value)
      :not
      (not= (= (get ctx k) (second value))))

    :else
    (= (get ctx k) value)))

In a month, the business asks to add ranges and comparisons. Say, to disallow the request if the :level field is in range of 3 to 9 (inclusive and exclusive, pay attention). Here come the [:in :field [3 9]] and [:< field 9] expressions in an EDN file. The developer gets angry as the house of cards he has built has turned unstable.

One day, something that he’s been afraid of all this time has finally come. He’s asked to introduce complex and/or/not logic. If (foo and bar) or (foo and not this), allow the request. If (this and that) or (foo and bar), disallow it. What is the outcome? Our developer is smart and has brief knowledge about interpreters, so he considers this a challenge. First, he extends the maps using [:or ...], [:and ...] and [:not ...] notation:

[:or
 [:and
  [:= :role "manager"]
  [:in :level [3 9]]]
 [:and
  [:= :name "John Smith"]
  [:matches :email #"some@pattern"]]]

Then he writes a primitive interpreter that walks a tree and processes it correctly. Of course, he reckons himself a genius. Only he knows how this interpreter works. There is no documentation; it’s like a meme: I’m the documentation!

Sarcasm aside, the developer has ended up with a real mess. Although it’s still a Data-Driven approach indeed, it’s awful. The rules are no longer data but a poorly designed DSL. The logic used to scan a vector of maps has grown into a poorly designed interpreter. Both are fragile, buggy, and uneasy to tweak.

Should the programmer really care about the quality of the code, he or she would notice that the initial idea of relying on data didn’t pay off. At some point, they should have stopped and said: we won’t go further with data. Complex checks are easier to implement in pure Clojure than crafting an interpreter. But they’ve committed way too far into this. They’re afraid for their reputation: they think admitting their mistakes is a sign of weaknesses (although it’s a sign of strength instead). They’ve watched too many YouTube videos about the divinity of Data-Driven development, and they’re still looking for a mysterious Graal.

Everyone who is praying for DDD must bear in mind the law of energy conservation. Either express the logic in code or declare some data and a framework that traverses the data and does something. In ordinary Clojure code, the complexity is distributed evenly. The DDD approach shifts that balance. Writing a vector of maps takes 5-10% of the total effort, but crafting a framework that runs that vector is 90-95%. It’s not good for a project when complexity differs dramatically. A new business requirement might easily hit the most complex part of the project. Nothing prevents the business from asking to check regular expressions; or if it’s a leap year; or if the client has a birthday today; or if their last name is Smith, or similar.

Thus, don’t write DDD frameworks. Don’t grow the complexity of the project. Use the standard Clojure facilities like functions and maps but not DSL and DDD.

By the way, do you know that DDD has been with us for years? It’s good old XML! There have been times when developers were obsessed with XML. That markup language was everywhere: to describe forms, settings, dependencies, workflow, schemas and even data transformation and logic! Would you like to maintain a project where every single bit is an XML file?

These days have passed, fortunately. But that weird willingness to describe everything with EDN reminds me of XML. What is the difference, after all? Both are definitions only, which is the top of an iceberg. Deep inside, there is a framework that drives these definitions.

If asked to implement that restriction system, I would start with maps too. But’d change something as soon as the rules get more complex. I’d make a rule not map only but a function as well. Say, if it’s a map, I check if it’s a subset of the context. But if it’s a function, I apply it to the context and decide on the boolean result:

(def RULES
  [;; a simple map case
   {:some-field 1
    :other-field 2
    :third-field "foobar"}

   ;; something more complex
   (fn [ctx]
     (or (and (-> ctx :foo (= :something))
              (-> ctx :bar (= 42)))
         (some-complex-check ctx)))])


(defn matches? [ctx rule]
  (cond

    (map? rule)
    (= rule (select-keys ctx (keys rule)))

    (fn? rule)
    (rule ctx)))

I’d even do it in this way:

(defn kv= [k v]
  (fn [ctx]
    (= (get ctx k) v)))

(def rule
  (every-pred (kv= :foo 1)
              (kv= :role "admin")))

Then I’d add more primitives for regex, patterns, ranges, etc and compose what I need from them. In Clojure, functions get composed great!

It’s no longer the data-driven approach, but I don’t care. A function is much simpler than a map and an interpreter that processes it. With a function, I can express any logic I want. There is no need to implement negation, or, and operators and more. We’re already given a great language — Clojure — so why would you stick with EDN and a poor interpreter?

One may say that DDD is great because it’s about the data. Adding a new rule boils down to extending an EDN file but not writing code. But in fact, there is no difference at all. Even if you edit an EDN file, you start a new Git branch, edit a file, add a new test and create a pull request. The pipeline is the same as it has to be for the ordinary code. When editing a .clj file, you create a branch, add a new function, write a test and open a pull request. Both ways involve the same steps.

Data-driven development lacks debugging capabilities, and that’s really an issue. Remember that vector of maps and functions which I proposed above:

(def RULES
  [;; a simple map case
   {:some-field 1
    :other-field 2
    :third-field "foobar"}

   ;; something more complex
   (fn [ctx]
     (or (and (-> ctx :foo (= :something))
              (-> ctx :bar (= 42)))
         (some-complex-check ctx)))])

Should a function-driven rule behave weirdly, I’d put a debug tag, run a test and easily debug it. I’d just hang in the middle of the execution of that function and see all the local vars. I can even try some expressions in REPL. But how in the world are you going to debug a map? It’s unthinkable because it’s just data. You cannot blame a map! You need to debug your framework, which is much harder than debugging a single function. Frameworks operate on context and lambdas, and debugging them properly requires more effort.

Talking about debugging, there is such a thing as stack trace. It’s extremely important when dealing with logged exceptions. Now, if you have a function-based rule that has failed due to an error, you’re good. You have a message, a file and a line number pointing to the cause of an exception. But the stack trace might be completely different if you have a framework that operates with maps and lambdas. The framework tried to compare to maps, but it failed with NPE. What were these maps? You’ll never know.

TL;DR

Did you watch Kung Fu Panda? There is no secret ingredient. That ingredient is you. I want you to stop looking for a magic trick that would make a job for you. There is no trick or technique; it’s all about diligence and simplicity.

Keep your code simple, dull and clear. The complexity must be distributed evenly across the codebase. Avoid various Somethig-Driven approaches and DSLs at all costs. A series of simple functions is much better in terms of maintainability.

Don’t trust YouTube videos and talks. Every talk is a staged show where the best parts are exposed to the viewer, and the failures are held back. No one gets to the scene to tell us how they’ve messed up.

Trust yourself only. If you are really interested in some idea or technology, test it with your hands. Don’t blindly trust those videos where speakers say it’s amazing. Instead of sharing such a video, say: I’ve checked it, and it’s great. Or, I’ve checked it, and it’s not as great as it’s said. Your own experience is much more valuable than modern trends awareness.

That’s all.