No Mocks
Even in a virulently anti-mock world we’re going to need some mocks. Unless your application is extremely self contained, you’re going to butt your head against a few barriers: databases, 3rd party endpoints, etc, etc… something lurking at the edge is going to need mocking.
Sure, you could just reach out to a full suite of sandboxes, but what is a sandbox if not a mock on steroids, over I/O? Both sandboxes and short lived DB instances could affect the speed and isolation of our unit tests, so we won’t entertain using them for now.
Instead, let’s reduce our anti-mock stance to permit these boundary exceptions and take a look at a couple of examples:
- React examples
- OO examples (coming soon)
How big or small is a ‘unit’?
In a word, massive.
The unit under tests is as big as is required for the application’s public API to come back with whatever response satisfies that same API. In practice, we’ll travel many classes or functions, through many layers, traversing many domains and hitting as many boundary concerns as might be necessary.
All that just to put a green tick next to a single feature that a client would recognise.
“Hold on…
…they’re not just massive, they’re Integration Tests.” Yep, by many definitions we’ve already strayed beyond the bounds of ‘unit tests’. The only problem is: if we stick to the absolute narrowest definition of a unit test, then Nonstop Mocks wins by default – it’s the only game in town.
Given that particular 3/10 solution isn’t providing the greatest of service – at least for now – let’s widen our concept of unit tests to look at the other extreme.
From here on we’ll permit unit tests to cover as many bits of code as needed, up to, but no further than, the edge of the repo at hand. No databases, no network calls, just code calling code within the same codebase – all able to run in parallel, all within a reasonable time.
Ease of Use: 2/10
We’re not off to a great start here, that supersized unit under test might well be our undoing. Comically, trying to not mock anything at all will lead to us needing plenty of mocks for our databases and third party endpoints.
To gain a good level of feature coverage, we’ll also need this menagerie of mocks to reappear in all sorts of permutations.
It gets worse… given we’re probing at our application from its outermost layers, understanding why we need these particular mocks in these particular permutations will once again require an encyclopaedic knowledge of almost the entire codebase.
In all likelihood, TDD is out of the question in the strict no-mock world too. We can write our “act” and “assert” test steps with confidence. However, run those tests and you’ll never be quite sure if your code is failing because you’ve coded in a bug, or if you didn’t get one of your boundary mocks set right in the “arrange” phase.
Trial and error will get you there eventually, but it’s 2 out of 10 here to reflect the sheer pain and length of that journey.
Stability: 3/10
With our unit test net cast so wide, we’re inevitably going to run into some trouble. Amongst all the many boundary mocks that we’ve set up for each feature variant of our unit, there’s going to be more than a few that seem out of place.
We’ve effectively set ourselves up for infinite fan-out: executing down a multiplicity of codepaths, crossing a wide variety of domain and architectural boundaries. Very quickly we’ll be mocking out the I/O concerns of the dependencies of our dependencies.
Not being driven by the same forces that change the core domain of our use case, these other concerns can change from under us at any moment. New ones will come and go, existing ones will change their signature and each time they do, we’ll need to update every single test arrangement that has the slightest connection to it.
Even if our underlying code isn’t spaghetti, our unit tests certainly are now. Prepare for near constant, surprise necessary updates. Each one feeling barely even relevant to your current test and its assertions. Stable, it ain’t – 3 out of 10.
Tests as Documentation: 7/10
Finally, things are looking up for the anti-mock world. Unlike in the pro-mock world where assertions often only protect interactions with mocks, these anti-mock tests are able to assert upon outcomes. They are, by their very nature, state based tests:
expect(yourProcess(someInitalState))
.toEqual(someExpectedState);The wonderful thing about that is, so are features:
expect(businessFeature(somethingFromTheClient))
.toEqual(somethingTheClientWanted);This sequence is also a close match to the testing staple “given, when, then” – a fantastic formula for writing acceptance tests for User Stories…
User Stories a.k.a features, a.k.a “why clients are willing to give us money”.
Our tests are documenting to other developers how we make money – our tests are documenting to other developers why our code even exists!
I’d love to celebrate further, but there’s a big “but” coming and it concerns our signal to noise ratio.
Yes, we are now surfacing some meaningful information to other developers (rather than just summarising implementation details), but it’s painfully obscured by a ridiculous amount of unstable I/O mocks. For this reason, we’ll reign in the score to a more modest 7 out of 10.
Regression Prevention: 9/10
Team this anti-mock methodology up with contract tests, smoke tests and a few token end-to-end tests and regression should be a thing of the past.
The basic premise being that our unit tests now protect features. Having said that, you did write all the required unit tests to cover every single one of our features and their many, many permutations… right?
Overall Score: 3/10
Of course you didn’t, who in their right mind would? Our 2 out of 10 for Ease of Use highlighted just how taxing it is to even implement this kind of unit testing strategy. On top of that, our 3 out of 10 for Stability pointed out that that heavy fee isn’t even a one time only deal – we’ll need to repay it with every other non-trivial code change to the repo.
This is the achilles heel of the anti-mock stance and it’s likely what drives a lot of larger projects to move to mocking everything indiscriminately instead. Yes, it documents our application and, yes, it protects actual features from regression, but going fully anti-mock simply does not scale.
It breaks my little heart to say it, but it’s another low, low overall score of 3 out of 10 here too.
“Hold on…
…I use this anti-mock strategy in my project and it’s not nearly as hard work or as fragile as you’re making out here”. You lucky bastard. I’m going to put my money on your project being either:
- A very self contained library that barely depends on anything beyond its boundaries and certainly nothing backed by a persistence layer. Or…
- An elegantly decomposed microservice or micro-frontend
Why projects of type A can succeed with an anti-mock stance is pretty simple: they never really needed them in the first place. Why projects in category B also squeak through is a little less obvious, so let’s quickly run through it.
To succeed as a microservice (and not just be a fragment of a distributed monolith) a project will have spent a lot of time, energy and scrutiny on its points of coupling in the wider system. The interfaces will be extremely succinct with a heavy emphasis on information hiding between services.
The knock on effect for our tests is that the size and signatures of our boundary mocks are tiny and easy to understand.
Signal to noise within tests here is therefore a hell of a lot better than in my main, non-microservice, hypothetical. Plus, a main cause of instability (other domains) is by definition, already abstracted away.
If your project is a stateless library or a true microservice, then it’s an adjusted 8 out of 10 score for the anti-mockers. By no means perfect, but it is at least tenable.
“So… Which One Won?”
Before we reveal the winner of our showdown, let’s quickly recap our contenders:
- Nonstop Mocks – so decoupled that it scales effortlessly, it’s just a shame that it’s also decoupled from providing any actual value
- No, Stop Mocks – provides actual value in both preventing regressions and documenting our application, but at a cost so high that no one is willing to pay for it
Drum roll…
It’s time for The Final Result