Ayende @ Rahien

Refunds available at head office

Rhino Mocks 3.5 Design Decisions: Getting closer to conclusion

There has been an invaluable amount of discussion regarding the question I pose about the API design of Rhino Mocks 3.5. Jeremy Gray in particular has been helpful in making good suggestions and making the argument to simplify the API for developers who are not experts in TDD terminology and usage.

My current thinking is to combine GenerateMock() and GenerateStub() into a single method, GenerateTestDouble(). This method will return a dynamic mock.

If you want a true stub, you will be able to call myTestDouble.StubUnexpectedInteractions(). Note, since stubs have a special behavior for properties, I am also considering: myTestDouble.StubPropertiesAndUnexpectedInteractions(); .

If you want a strict mock, you will be able to call myTestDouble.FailOnUnexpectedInteractions();

Partial mock will not be supported by the static GenerateTestDouble method, and will require using the usual instance methods.

Calling VerifyAllExpectations() will be supported on all mocks.

The only remaining open question is whatever calling Expect() on an object that you previously called StubUnexpectedInteractions() should fail or not.

Thoughts?

Comments

Andre Loker
06/30/2008 06:02 PM by
Andre Loker

I kind of like the idea of being able to "configure" the behaviour of the test doubles after creation. Maybe a consistent naming of these methods would be helpful, like this:

var stub = GenerateTestDouble(); // default is stub behaviour

var dynamicMock = GenerateTestDouble().WithExpectactions();

var strictMock = GenerateTestDouble().WithStrictExpectations();

var propertyStub = GenerateTestDouble().WithStubbedProperties();

Expect() should fail on stubs IMHO, it's an invalid operation on a stub. BTW: The same goes for VerifyAllExpectations IMHO, but you already seem to have chosen otherwise. Today I played a lot with RM 3.5 and came across a situation a few times, where I unententionally called VerifyAllExpectations on the wrong object (a stub instead of a mock).

Regards,

Andre

Jeremy Gray
06/30/2008 06:37 PM by
Jeremy Gray

re: StubPropertiesAndUnexpectedInteractions - unless I'm mistaken, the point of this call would be to stub backing stores for the properties ala PropertyBehavior as well as doing what StubUnexpectedInteractions would do. Would it do anything else that is not covered in its method name?

(runs off to refresh his memory on exactly what behaviour would result from dynamic mock by default)

re: StubUnexpectedInteractions - just back from checking the wiki on dynamic mock behaviour, it would seem that the generated test double would start off returning null or default for members. Were the default test double a strict mock, StubUnexpectedInteractions would have an obvious semantic, but given the behaviour of a dynamic mock, together with the fact that StubPropertiesAndUnexpectedInteractions add in PropertyBehavior, what is StubUnexpectedInteractions left changing about the behaviour of the dynamic mock?

re: PartialMock - Just out of curiosity, and as much as I'm trying to avoid mocking non-interfaces as much as possible, is there a particular reason that partial mocks can't be created in the AAA syntax?

re: VerifyAllExpectations being supported on all test doubles, would I be correct to assume that you are still going to support AssertWasCalled, for those who want to keep using it? Or is it gone?

re: "calling Expect() on an object that you previously called StubUnexpectedInteractions() should fail or not" - hmm, this is a toughie. I can see pros and cons for both options. I am tempted to suggest allowing Expect to be called after so that you can first centrally call one of the two Stub*UnexpectedInteractions methods right after creating the test double, or right after setting up common stubbed behaviour, and then later add in expectations on a per-test basis.

Ayende Rahien
06/30/2008 07:02 PM by
Ayende Rahien

The call the StubXyz() will mean that no expectations are recorded, only stubbed behavior.

AssertWasCalled and friends are there, and will work for both mocks & stubs.

Partial Mock is far rarer in use, and I want to keep the API minimal. The is no technical reason.

The problem with Expect() is that it carries an assumption with it.

For example:

var foo = MockRepository.GenerateTestDouble().StubUnexpectedInteractions();

foo.Expect( x=>x.Bar() );

foo.VerifyAllExpectations(); // will work, stubs have no expectations.

var foo = MockRepository.GenerateTestDouble();

foo.Expect( x=>x.Bar() );

foo.VerifyAllExpectations(); // will fail, expected call was not called

Jeremy Gray
06/30/2008 07:15 PM by
Jeremy Gray

Hmm, perhaps I wasn't clear enough with my questions. Allow me to take a moment to re-phrase one of the questions, based on your example, and then we can go from there.

Ignoring property behaviour for a moment, a dynamic mock has automatically-stubbed interactions by default, in that it will accept all calls and will return null or default. What then does StubUnexpectedInteractions change about that behaviour?

Ayende Rahien
06/30/2008 07:36 PM by
Ayende Rahien

It means that it will not verify expectations.

Jeremy Gray
06/30/2008 08:07 PM by
Jeremy Gray

But one can already do that by calling .Stub instead of .Expect.

My assumptions previously centered on the default behaviour being more strict-mock-like, which drove the suggestion for StubUnexpectedInteractions(). Assuming the centering of default behaviour on what would previously have been known as a dynamic mock allows me to throw StubUnexpectedInteractions away and ask this:

Setting aside for the moment whether or not one should be able to set expectations on "stubs", don't we just need to be able to control the following (in addition to calling .Stub and/or .Expect):

  1. Strict versus loose (default is loose, strict via something along the lines of FailOnUnexpectedInteraction())

  2. Property and event backing store implementation versus not (default is presumably not)

?

In parallel with this, what in the end really makes a stub a stub:

a) property behavior?

b) lack of expectations?

c) inability to set expectations?

Some combination?

Jeremy Gray
06/30/2008 08:10 PM by
Jeremy Gray

(where #1 in the above essentially allows one to push the test double towards the classic CreateMock behaviour and #2 in the above essentially allows one to push the test double towards the current AAA GenerateStub behaviour)

pb
06/30/2008 08:14 PM by
pb

Er, backwards compatibility would be nice too...

Steven Harman
06/30/2008 09:22 PM by
Steven Harman

So, just to be clear... are Partial mocks no longer supported via the AAA Syntax?

I have a case where I've created a partial Mock via an explicit repository, then as part of a shared test context I've stubbed out a public virtual method on that mocked object (via myPartialMock.Stub(x => x.foo())...).

Arrange is now done and so I switch the partial to replay mode via mocks.Replay(myPartialMock).

I then Act on a different method on the partial.

myPartialMock.bar();

Next I want to myPartialMock.AssertWasCalled(x => foo()), but I get a Rhino.Mocks exception saying that "System.ArgumentException: Constructor arguments should not be supplied when mocking an interface."

I'm guessing that some changes happened between 3.5 beta and RC to cause this to no longer be a valid scenario?

Jeremy Gray
06/30/2008 09:49 PM by
Jeremy Gray

@pb - oh, definitely. The traditional syntax will be sticking around, from what I would guess at the moment. As for the AAA syntax, on the other hand, it has never been officially released so what we're trying to do at the moment is nail that syntax down so that it can last a long while upon release.

Ayende Rahien
07/01/2008 12:16 AM by
Ayende Rahien

Steve,

Partial mocks are supported, they just won't have a shortcut static method.

Those mocks will be able to take part in AAA syntax.

The error you see is probably a bug, can you create a test case?

Ayende Rahien
07/01/2008 12:18 AM by
Ayende Rahien

Jeremy,

A stub is something that will not fail the test.

Ayende Rahien
07/01/2008 12:23 AM by
Ayende Rahien

pb,

Backward compatibility with 3.4 will be maintained.

There is no concept of backward compatibility with the beta or RC.

Graham Nash
07/01/2008 12:45 AM by
Graham Nash

I'd prefer to keep it as GenerateMock, GenerateStub, GeneratePartialMock, and GenerateStrictMock. First, it is more intention revealing. I know exactly what kind of test double I am working with.

Second, there won't be any odd cases where methods either do nothing or should cause a failure because they won't be on the class. This is more object oriented also because all of the methods on the class will be supported.

I don't think having to learn a couple of TDD terms is a large barrier for entry.

Ayende Rahien
07/01/2008 12:49 AM by
Ayende Rahien

Graham,

The methods are never on the class, they are extension methods

jdn
07/01/2008 02:00 AM by
jdn

This may be a stupid question, but if you are still deciding on how the API will work, how is Rhino.Mocks 3.5 a release candidate?

Aaron Jensen
07/01/2008 02:11 AM by
Aaron Jensen

I would strongly recommend against StubUnexpectedInteractions. That should be the default behavior.

I'd also recommend getting rid of Expect and VerifyAllExpectations, but I know my view is extreme.

I'm yet to see a compelling argument for not doing AAA in all tests. If you want to ensure something wasn't called, you should assert it:

mock.AssertWasNotCalled( x=>x.Foo());

If you want to make sure only what you expect to have been called was called, you should assert it:

mock.AssertOnlyTheseWereCalled( x => {

x.Foo();

x.Bar(); });

If you want to assert things were called in a particular order:

mock.AssertWasCalledInOrder( x=>{

x.Foo();

x.Bar(); });

Obviously some of these things can be moved to the options, but you get the idea.

Expect is a crutch and it obfuscates the true desire of the test.

Ayende Rahien
07/01/2008 02:12 AM by
Ayende Rahien

The functionality is there, tested and is working.

What we have left is the windows dressing.

Ayende Rahien
07/01/2008 02:13 AM by
Ayende Rahien

Aaron,

I really want to get AssertOnlyTheseWereCalled, fancy committing that?

For AssertWasCalledInOrder, it wouldn't work.

Ordered is for cross mocks things.

Aaron Jensen
07/01/2008 02:18 AM by
Aaron Jensen

Ayende,

Work and my desire to get MSpec into a more usable form preclude me from doing much else at the moment. Also, I'm a little frightened of the AssertWasCalled implementation, so I'm really scared of AssertOnlyTheseWereCalled :).

Oh, and if I patched that my patch would include removal of Expect and VerifyAllExpecattions ;)

Ayende Rahien
07/01/2008 02:19 AM by
Ayende Rahien

No need to be scared. It is just interception again.

It is rhino mocks using rhino mocks to do its work, I consider this beautiful

jdn
07/01/2008 02:21 AM by
jdn

How the API works is window dressing? Feh.

Ayende Rahien
07/01/2008 02:22 AM by
Ayende Rahien

No need to be scared. It is just interception again.

It is rhino mocks using rhino mocks to do its work, I consider this beautiful

Ayende Rahien
07/01/2008 02:23 AM by
Ayende Rahien

I mean, we know how it works, and what the semantics are.

What we are arguing about is what should be done to make this happen.

In other words, we are discussing the facade that you will see as you work with Rhino Mocks

Scott Bellware
07/01/2008 08:46 AM by
Scott Bellware

I think "Test Double" is yet more obscure, self-indulgent pattern language that keeps mocking - and more importantly, the designs that it supports - from being more approachable by the masses.

Ayende Rahien
07/01/2008 01:46 PM by
Ayende Rahien

Scott,

A naming suggestion would be welcome.

Scott Bellware
07/02/2008 06:47 AM by
Scott Bellware

Have to give it more thought, but off the top of my head, maybe something as simple as:

  • Fake.Create()

  • Fake.Of()

Ayende Rahien
07/02/2008 06:49 AM by
Ayende Rahien

Scott,

Fake has a meaning of its own in TDD terminology

Aaron Jensen
07/02/2008 07:18 AM by
Aaron Jensen

According to MF's bliki ( http://www.martinfowler.com/bliki/TestDouble.html )

The only generic term is Test Double. That said, I don't particularly care for that term, and I think assigning terms to the objects themselves isn't as useful as using verbs to describe what you're doing to the methods.

Since "Mocking a method" has less intrinsic meaning than "Expecting a method to be called" or "Specifying that a method should be called", I would just say you should use Mock. Take the term back and continue using it for the greater good.

You create a mock. On that mock, you can stub methods, you can specify that methods should be called, you can forward methods to the real implementation, you can do whatever you want.

Or just call it TestDouble.

Jeremy Gray
07/02/2008 03:24 PM by
Jeremy Gray

@Aaron - Expect is not a crutch, it is a way to set an expectation and provide a return value without doubling lines of code. Unless you consider ways of avoiding doubling lines of code to be a crutch. ;)

@Bellware - The whole reason this discussion got started is because it is exactly the current terminology and, to a lesser degree, behaviour that is keeping this technology from being approachable by the masses, at least in terms of the masses I've run into. If by the end of this discussion we don't have something that clearly improves things well then I am all for leaving them as they are. After all, one needs to question their own assumptions from time to time. I recommend you give it a try.

Jeremy Gray
07/02/2008 03:25 PM by
Jeremy Gray

@Ayende - I really regret having been away from this thread through the last day, because it has become quite clear that you and I still have wires crossed re: the intended behaviour of a call like StubUnexpectedinteractions.

In this proposed syntax change under discussion, you have based the default behaviour on that of a dynamic mock. As such, we don't even need a StubUnexpectedInteractions call, because it is that way by default. We need just two calls: one to go strict and throw on all unexpected interactions, another to add property behaviour. These two calls would push the mock in opposite directions from the default middle ground of stubbing unexpected interactions.

While these two could technically be applied in combination, I wouldn't be opposed by restricting doing so, which would put the three possible states on a continuous line that ranges from very loose, very stub-like behaviour on the one end, through the default dynamic mock -style behaviour in the middle, to the very strict, throwing on all unexpected interactions behaviour on the other end.

Given the above, in my view ANY generated test double, regardless of whether or not a call was made to make it go strict or to give it property behaviour, could have expectations set on it and later verified, and through any of the available syntactical elements for doing so.

In this model, test doubles are just test doubles. For some you may just stub some behaviour, making them colloquial stubs. For some you may just set and validate some expectations, making them colloquial mocks. For others, you may do both simultaneously (and I often need to do so if I want to avoid over-specified tests.)

Admittedly, the fact that I'm having to clarify this repeatedly is a possible sign that this exploration may not end up anywhere useful, but it may also just be a sign of a simple wire-crossing and we might end up somewhere much better. Either way, we'll have validation of whatever syntax results, and I'm all for validating things from time to time.

Aaron Jensen
07/02/2008 03:51 PM by
Aaron Jensen

Jeremy,

Moving assertions into the middle of the flow of a test just to save one line is asinine. Often it doesn't even save that line because you're calling VerifyAll any way. Regardless, I'd say the situation where you could use expect is rare. I think the following conditions need to be met:

  • You are verifying interaction

  • AND The return value from that interaction is "important"

  • AND The interaction can't be tested by some other state change/whatever that is a result of the return value

Even IF you meet all of these conditions, having two lines of code is more readable. Even in pseudo code it's more readable:

foo.Bar will return 5

DoSomething

foo.Bar should have been called.

vs.

foo.Bar should be called soon, and it will return 5

DoSomething

Verify that everything I said should be called was actually called

Also, I can practically guarantee you that many of Rhino.Mocks users misuse Expect all the time. Heck, I used it completely wrong a year ago. I didn't even know about SetupResult. The guidance wasn't there. As a result my tests were highly coupled to my implementation and I was testing far more than I intended to test.

The guidance should be in the framework. Unfortunately, Ayende stated that he feels it is "more natural" to expect before the interaction and I have no way of arguing against his feelings, so I don't expect Expect/Verify to go away from RM any time soon.

Ayende Rahien
07/02/2008 06:08 PM by
Ayende Rahien

Aaron,

Regardless of my feeling (or yours), I am not going to going to break published API if I can avoid it in any way

Aaron Jensen
07/02/2008 10:11 PM by
Aaron Jensen

I agree when it comes to the old pre-3.5 way of doing things. But with the new syntax, you're clearly willing to change some things.

It's moot though, as we don't agree. Some day, if I ever finish/really get started on Machine.Mocks I'll have an alternative with a simple, opinionated api and no remnants of record/replay exposed to the user.

I'm still quite happy using the many portions of Rhino.Mocks I love though :)

Jeremy Gray
07/05/2008 06:33 PM by
Jeremy Gray

@Aaron - when it increases clarity, it gets used. When it doesn't, it doesn't. To do otherwise is the only thing that could be considered asinine.

There are people that like Expect and VerifyAllExpectations, there are people that like using Stub and AssertWasCalled instead. There are also people who mix and match, choosing the option that best fits easch situation. I haven't been calling out for anything that would limit those options, I've just been calling out for simplifications that might (emphasis on might, as this is still an exploration, after all) help the whole model be more understandable to a wider audience and require less mental gymnastics even for those already somewhat familiar with the technology. I dare say you've hyper-focused on a non-issue.

Comments have been closed on this topic.