Here’s an email message the dev lead at our company sent out tonight:
“Hi,
I analyzed the nightly build failure - it broke because of an examples test that tests recursive fakes, and incidentally uses WhenCalled() on a property getter on the 2nd level of the recursive fake. This does not work because of a hidden bug - WhenCalled() should reset the recursive fake property setter, and when it is called true properties should be off. This bug has been there for a long time, and only the bug fix for recursive fakes with true properties unveiled it.
I will take two actions to resolve this:
1. add a test for this scenario to our normal unit tests
2. implement the fix above - 1/2 hour of work
Please pay close attention to the build status until then - the nightly breaks because of this examples test. Any other test failure should raise a flag.”
I was thrilled to read this. Our automated nightly builds also run all the “example” tests that ship with Typemock Isolator. A couple months back we found out that this was not being done, and added it to the build process. The bug found here is the type that can remain hidden and even if you do TDD there are always chances of not thinking about tests that kind of corner cases.
In this case, not only did the nightly build uncover a bug that would have been embarrassing to find at the customer site (“silly typmock, their own example tests don’t even pass on a newly installed machine!”) but uncovered a buried problem, by chance.
lesson: automate and test as much as you can. even examples that you produce with your product.