Ayende @ Rahien

Refunds available at head office

Rhino ETL Video

Paul Barriere has a video up of a presentation about Rhino ETL:

ETL stands for Extract, Transform, Load. For example, you receive files or other data from vendors or other third parties which you need to manipulate in some way and then insert into your own database. Rhino ETL is an open source C# package that I have used for dozens of production processes quite successfully. By using C# for your ETL tasks you can create testable, reusable components more easily than with tools like SSIS and DTS.

It is good to see more information available on Rhino ETL.

Comments

P
03/20/2010 05:16 PM by
P

At around 66min into the video, the presenter used rhino mocks to stub out a service call and then asserted that the service was called in the ETL process.

From a perspective that is 100% dogmatic regarding mocks vs stubs, the presenter should have generated a mock, set an expectation that the service would be invoked in the ETL process, and then verified the expectations on the mock. Is this correct?

Let the war begin! (... and yes my next comment will be about dynamic sql vs stored procs) =D

pb
03/22/2010 09:56 PM by
pb

Below is the code from the test - what is your recommendation?

    [TestMethod]

    public void CanWriteToService()

    {

        var service = MockRepository.GenerateStub

<iuserservice();

        var userRecords = TestOperation(

            new GenerateTestData(),

            new UserWriteToService(service)

        );


        service.AssertWasCalled(x => x.MarkUsersGenerated(null), o => o.IgnoreArguments());

    }
P
03/23/2010 06:24 PM by
P

Without compiling...

[TestMethod]

public void CanWriteToService()

{

var mockService = MockRepository.GenerateMock();

mockService.Expect(x => x.MarkUsersGenerated(null), o => o.IgnoreArguments());

var userRecords = TestOperation(new GenerateTestData(), new UserWriteToService(service));

mockService.VerifyAllExpectations();

}

The test given in the demo works, but I think its a product of rhino mocks blurring the lines between a mock and a stub. From a purist point of view, you should never test (aka have expectations) for a stubbed object. Stubbed objects enable you to isolate aspects of the system under test, but they arent (ignoring rhino mocks) testable.

pb
03/23/2010 11:50 PM by
pb

I like AAA... If you don't, feel free to submit a patch removing it and see how it goes. :)

P
03/24/2010 03:17 AM by
P

Why don't you think what I posted follows AAA?

Arrange -> Create the mock and setup an expectation

Act -> Invoke the system under test

Assert -> Verify the expectations on the mock

I love rhino mocks, and the context/specification style harness we've developed that enforces AAA. When we bring a new developer on board who has never done unit testing, we have to explain the fundamental difference between a mock and a stub. The example given in that presentation violates the true definition of a stubbed component and creates confusion for a developer trying to embrace the difference (spoken from experience when I first used rhino 3.5).

The test case we are talking about is testing the behavior of a system, not state. If you are testing behavior, you should be validating that the system under test interacted with a mocked object as expected.

Mocks for behavior.

Stubs to ignore all the crap that you don't care about in context of the test case, and for very handy when doing state based tests.

As an educator, its important to show that just because you can use a flat head screw driver for a phillips head screw, that doesnt mean you should.

P
03/24/2010 03:23 AM by
P

... and as a comment poster, it is important to be coherent/grammatically correct...

"Why do you think that doesnt follow AAA?" :)

pb
03/24/2010 12:21 PM by
pb

Sorry, I didn't read your example carefully enough - I started stubbing pretty much everything because generally I don't care if the thing I'm stubbing is being called for another method, what order, etc. I just want to verify that it calls the method. I used to do it closer to your way (checking the order even) but had to fix tests more often when things not related to the things being tested were changed.

P
03/24/2010 08:05 PM by
P

I lived through the pain of strict mocks, ordered mocks, and record/replay :)

" I started stubbing pretty much everything because generally I don't care if the thing I'm stubbing is being called for another method, what order, etc. I just want to verify that it calls the method."

^^ Using a mock instead of a stub doesnt make any difference here. You can even stub out aspects of a mock just like a stub (mockService.Stub(x => x.ServiceId).Return(Guid.New())).

In the end you are testing that some object got called, which is a test on behavior, which should be an expectation on a mocked object.

Hasan
04/20/2010 09:39 AM by
Hasan

Dear Ayende, Currently I'm facing problem that probably need Rhino ETL as part of the solution. But, I need to clarify some information since my source is Oracle Database with NHibernate. Is it possible to use Rhino ETL with Oracle as source of ETL data? If yes, How is the way? One of my choice is using Web Service.. Is there any other way beside web service that connect to Oracle data?

Ayende Rahien
04/20/2010 09:48 AM by
Ayende Rahien

Yes, it is.

Take a look at the tests showing the use of C# data sources

Hasan
04/22/2010 06:49 AM by
Hasan

Dear Ayende, I'm currently trying to implement Rhino ETL that will process source data from NHibernate. But i'm still stuck.. session Nhibernate failed to load,

My question are:

Is Rhino ETL using thread by default, since NHibernate is not a thread safe?

Is the performance OK if I use Nhibernate rather than non ORM/NHibernte way for Rhino ETL data source/input?

Ayende Rahien
04/22/2010 06:53 AM by
Ayende Rahien

Hasan,

By default Rhino ETL is multi threaded, yes.

You can change that by replacing the PipelineExecuter in the process ctor.

And yes, using NHibernate is probably fast enough

Hasan
04/22/2010 07:20 AM by
Hasan

Is that mean that we can't use pipeline in Rhino ETL along with NHibernate? I think pipeline is useful to speed up process....

Ayende Rahien
04/22/2010 07:30 AM by
Ayende Rahien

Hasan,

You can use it.

The pipeline is used to execute different parts of the pipeline on different threads.

There isn't any thread hopping inside an operation.

Comments have been closed on this topic.