I’m in the process of teaching another TDD master class in Oslo this week, and in the past two days we went over the major Isolation frameworks in .NET (I refuse to say Mocking frameworks anymore. the “Mock” work is too overloaded as it is). We covered NMock, Rhino Mocks, Moq and Typemock , and alos as a start we used hand written mocks and stubs, and NUnit.Mocks for entry level learning.
Several things can be gleaned from how students reacted to using these frameworks (in random order of thought)
- The verification error messages that all three big frameworks (rhino, typemock and mock) throw could be much clearer (expected X but only got Y)
- The AAA (Arrange act assert) model for isolaation framework, whilke much shorter, is still not as easy to grasp as you might think for newcomers. In fact, I’m not sure people found it easier than record-replay.
- One of the hardest points was remebering the APIs in rhino and Moq, becuase there is more than one entry point to using the API (Constraints alone in rhino have 4 different “root” classes)
- The word “Expect” confuses because it makes you think about “expectations” as in interaction testing, when in fact it can be used for stubbing out things.
- Once people realized that “Strict” mocks are causing lots of problems, it was one of the first questions they had about any new framework they learned. NMock2 doesn’t have a way to turn off Strict mocks. weird.
- Verification Exceptions can be too easily “swallowed” by inner code so the test seems to pass (false positive). i think Isolator is the only one that will re throw swallowed exceptions).
- in AAA, People kept doing Expectations or Stubs method return values when what they meant was they want to veify the method was called
- Given an interface IFoo with a boolean property,if people want to make the property return a false value they instinctively write “myFake.Foo=false”. All frameworks should support default “property like behavior” to make this possible. Rhino mocks has this ability.
- 2 VB.NET students in the class kept hitting walls with everything that had lambdas in it,(Vb.NET has partial support for anonymous delegates) which means right now VB.NET has to use record-replay.
- The fact that Moq needs lambda expression trees and not regular lambdas prevents resharper from generating code signatures using smart intellisense. Moq is not resharper-friendly..
- the Moq syntax for constraints is just too complicated (It.IsAny<string> is OK, but It.Is<string>(s=>s.contains”a”) is just too much code noise and complexity
- The rhino AAA way of constraints is also hard to remember, but a bit more friendly (but not by much) : Arg<string>.Matches(Text.Contains(“A”)
- Mixing inheritance and frameworks fake objects in the same test is very confusing for beginners.Either use an isolation framework or override virtual methods, but not both.
- TestDriven.NET wins hands down against resharper unit test runner
- People love RowTest (which seems to have been gone with the new version nunit 2.5 and replaced with [TestCase(values…)] )
- the book “Test Driven Development with MS.NET” is hideously outdated by now (given out to students)
- People want the ability to define a call-by-call behavior of a stub return value. Feels very much like over-specification
- With hand written stubs newcomers will easily make the mistake of adding real functionality into the hand written class instead of into the production code.
- Test names really help to find misunderstood requirements
- 3 frameworks in 1 day could prove to be too much information. 2 could be just enough.
- When doing TDD people will actually forget that they can debug the test to see why it’s not behaving like it should
- People really likes the fluent API design of NMock2, but disliked its strictness
- If you use moth Moq and rhino in the same test class, you will see all the extension methods mixed up when you open intellisense on any object. *shiver*
- Isolator was the only one where people didn’t ask me a single question about what API to use. they did put calls in the wrong place in the test though (AAA is hard to get used to)
- AssertWasNotCalled feels like over-specification.
- lambdas were easier to get used to than I expected.