Let me start by saying Shims are evil. But they are evil by design. They let you do things you otherwise couldn’t do, which is very powerful. They let you do things you might not want to do, and might know you shouldn’t do, but because of the real world of software, you have to do.
The Catch-22 of Refactoring to Enable Unit Testing
In the first part of my series on VS11 Fakes, I reviewed Stubs, which are a simple way of creating concrete implementations of interfaces and abstract classes for use in unit tests.
But sometimes it happens that you have to test a method where the dependencies can’t be simply injected via an interface. It might be that your code depends on an external system like SharePoint, or simply that the code news up or uses a concrete object inside the method, where you can’t easily replace it.
The unit testing agilista have always said, “Refactor your code to make it more testable,” but therein lies the rub. I will again refer to the esteemed Martin Fowler for a quote:
Refactoring is a disciplined technique for restructuring an existing body of
code, altering its internal structure without changing its external behavior.
How do you know if you are changing its external behavior? “Simple!” says the Agilista, “You know you didn’t change it as long as your unit tests still pass.” But wait… we don’t have unit tests yet for this code (that’s what we’re trying to fix), so I’m stuck… Catch-22.
I have to quote Joseph Heller’s masterwork just for fun:
There was only one catch and that was Catch-22, which specified that a concern
for one's safety in the face of dangers that were real and immediate was the
process of a rational mind. Orr was crazy and could be grounded. All he had to
do was ask; and as soon as he did, he would no longer be crazy and would have
to fly more missions. Orr would be crazy to fly more missions and sane if he
didn't, but if he were sane he had to fly them. If he flew them he was crazy
and didn't have to; but if he didn't want to he was sane and had to. Yossarian
was moved very deeply by the absolute simplicity of this clause of Catch-22 and
let out a respectful whistle.
He’s crazy if he wants to go fight, but if he says he really didn’t want to fight then he isn’t crazy and so needs to go fight. We have the same problem. We can’t test because it needs refactoring, but we can’t refactor because we don’t have tests.
Shim Your Way Out of the Paradox
Example 1 -
To see what this really looks like, let’s look at some code (continued from the example in Part 1).
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 | |
There is one minor change from our previous example. We are now setting the
LastUpdated property on the Customer object before passing it to the
repository. (I know this might not be the best way to do this, but go with
me…)
How do we test that Save sets the correct value to LastUpdated?
You might start by writing a test like this. This uses the same Stubs techniques
I showed in the last article and tries hard to deal with the variable nature of
the LastUpdated property.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 | |
There are, of course, a few issues with this code, most notably the bit below
the comment explaining the 10ms window. How small can we make the accuracy
variable before the test starts to fail? How close is close enough? What about
when the viewModel.Save() code does extra work eating up some of that time?
A better way to test this is to use a Shim to override System.DateTime, causing
it to always return a predictable value. That test would look something like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 | |
On line 21 we use Shims to override the Now static property getter on the
DateTime object. This replacement is for all calls to DateTime.Now in the
AppDomain, which is why we are required to provide a scoping region for our
replacements with the ShimsContext.Create() block.
But now this test is no longer “flaky”. It has no dependency on your system
clock, on the length of time that Save() or any other code takes to run. We
have controlled the environment and isolated this test from the external
dependency on DateTime.Now.
This example shows one issue that can make code difficult to test, but of course there are more.
Example 2 - The hidden object instance
Here’s a different example showing another problem.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | |
In this example, we have a method that is creating an instance of the
WPF DispatchTimer class. It uses the timer to track when 30 seconds
have elapsed and then it refreshes a property on the view model (presumably
this collection is bound in the XAML to a list on-screen) by calling
RefreshData().
How can we test this code? There are a few things we might like to test, including:
- Does it start the timer with a 1 second interval?
- Does it refresh the list only after 30 ticks have gone by?
Let’s look at how we might use Shims to create that first test.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | |
In this example we use Shims ability to detour all future instances of an
object by using the special AllInstances property on the generated Shim
object. Since these methods are instance methods, our delegate has to include a
parameter for the “this” pointer. I like to use @this for that parameter to
remind me what it is, but I’ve seen other people use that or instance if
you dislike the idea of using a C# keyword in your code.
We override the Interval property setter (which takes a timespan, hence the
name IntervalSetTimeSpan) and the Start method. By stashing away values into C#
closures, we can call sut.StartTimer() and then verify that what we expected
did in fact happen.
Example 3 - Using Shims to wrap an existing instance
One more example will wrap up my introduction to Shims.
We will now take a look at how we might implement the second test in my
list above. We want to confirm that the RefreshData method is called
only after 30 ticks have elapsed.
Let’s take a look at how we would test this using Shims. In this case we
will still need to control the DispatchTimer, but we will also need to
override the implementation of RefreshData(), which is a method on the
class we’re testing.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 | |
While nobody would call this a pretty test, it does what we asked it to do, and has let us test one aspect of this method we otherwise wouldn’t have been able to test.
Resolving the Catch-22
The Catch-22 that I mentioned at the start of this article can be restated like this:
- You can’t test the code because it has internal dependencies that make it untestable
- You need to refactor it to fix the dependency
- Since you don’t have any tests for this code, you aren’t safe to refactor
- See #1
Hopefully you can see that Shims help you get out of this dilemma. With Shims we can rework that list to something like this:
- You can’t test the code using traditional techniques because it has dependencies that make it untestable.
- Before you can refactor it, you need to create a test that asserts its current behavior.
- You can use Shims to create this characterization test, enabling you to refactor safely.
- Once you have it under test, you can safely refactor and make the dependency explicit and replaceable using traditional (e.g. interfaces and stubs) methods.
- The characterization test should continue to pass after performing the refactoring.
- Once the refactoring is complete, you can create new tests that do not use Shims (but may use Stubs) to test the code.
- Then you can remove the Shims test.
It is a few more steps, but it lets you have a clear, intentional way to refactor code that otherwise would have required “refactoring in the dark”. The Catch-22 is resolved and you can continue to improve the design and testability of your code.
I do recommend people go all the way with this approach and refactor the code until the Shim-based test is no longer required. Every time you need to use something like Shims to test something, it is telling you that you have a design problem. You probably have high-coupling and low cohesion in that method.
Refactoring is the act of intentional design, and you should always take the opportunity to make your design better. Shims can be used to get out of this impasse, but if you don’t think about the problem you will end up with a Big Ball of Mud for your design.
Some final words about Shims
As the famous literary quote goes (Voltaire, Socrates and even Spider Man’s Uncle Ben):
With great power comes great responsibility.
The examples I’ve shown in this post are all examples of design flaws around coupling and cohesion. Knowing that, we should always feel a bit “dirty” about having to use Shims to test something. Every time we do it, we should remember to go the extra mile afterwards and refactor it away if we can (my steps 4-7 above).
The Catch-22 of untestable code is real. Getting out of it is hard. Shims are designed to help you get out of this trap.
Shims may be evil from a purist design and TDD sense, but in the real world we are often faced with code we a) don’t control or b) didn’t write and which doesn’t have any tests. Use Shims to get out of that, but always continue on and fix your design issues.
Conclusion
Visual Studio 11 includes the new Fakes library for creating isolated unit tests. It includes two kinds of Fakes:
- Stubs for creating lightweight, fast running concrete classes for interfaces and abstract classes your system uses. I reviewed Stubs in my last article.
- Shims for intercepting and detouring almost any .NET call. Shims are particularly useful for removing internal dependencies of methods, and for getting you out of the “testability Catch-22”.
Hopefully you have found these two articles useful. For more information on Fakes please take some time looking through the MSDN Documentation.