Ayende @ Rahien

My name is Oren Eini
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by phone or email:


+972 52-548-6969

, @ Q c

Posts: 6,125 | Comments: 45,493

filter by tags archive

Rhino ETL & FileHelpers Integration

time to read 9 min | 1662 words

Well, this image really excites me. It excites me because I got this after integration FileHelpers into Rhino ETL. This image is the result of an ETL script that joined a file against a table, did some additional processing on it and push it to a table in a third database.

I must say that FileHelpers has made this ridiculously easy to do.  All I have left to do is figure out how to test such a thing effectively.


For reference, here is the complete script:

class Customers:
      CustomerID as string
      CompanyName as string
      ContactName as string
      ContactTitle as string
      Address as string
      City as string
      Country as string


      ConnectionType: SqlConnection,
      ConnectionString: "Data Source=localhost;Initial Catalog=ETL_Test; Integrated Security=SSPI;"

      ConnectionType: SqlConnection,
      ConnectionString: "Data Source=localhost;Initial Catalog=Northwind; Integrated Security=SSPI;"


source CustomersFile:
            for customer in Read(typeof(Customers)).From("""Files\CustomersTab.txt"""):
                  SendRow( Row
                        CustomerId: customer.CustomerID
                        CompanyName: customer.CompanyName,
                        ContactName: customer.ContactName


source OrdersFromDatabase, Connection="Northwind":
      Command: "SELECT * FROM Orders"


join CustomersAndOrders:
      if Left.CustomerID == Right.CustomerID:
            Row.OrderID = Right.OrderID
            Row.CompanyName = Left.CompanyName
            Row.ContactName = Left.ContactName


 destination Final, Connection="DestinationDB":
      Command: "INSERT INTO OrdersWareHousing VALUES(@OrderID, @CompanyName,@ContactName)"


pipeline OrdersWareHouse:
      CustomersFile >> CustomersAndOrders.Left
      OrdersFromDatabase >> CustomersAndOrders.Right
      CustomersAndOrders >> Final


 target default:


Markus Zywitza

That's marvellous!

Is there any documentation on Rhino ETL other than your blog entries? If not, is there a good starting point in the source code to look at?


Ayende Rahien

Markus, no documentation yet.

The best place to start look is in the integration tests, I'll upload soon the changes required to support FileHelpers


Wow nice to see the seamless integration !! =)

Thanks a lot for your words Ayende, them are a great motivation to continue developing and releasing the FileHelpers

There are large hours enhacing the API make it more and more usable, mostly thanks to the CLR core team with his:

Designing .NET Class Libraries


And with the Excellent BOOK: Framework design guidelines.

In the company that I work for Im the responsable for our base framework to generate code and so. So I love Framework and LIbrary design, also to design winforms control.

I wish to found time soon to help with a Rhino ETL GUI or some examples

You are doing a OUTSTAND WORK DUDE !!!

About the code you need to add the [FieldQuoted(OptionalForBoth)] because without quotes some values can contains a tab and so fail to load.

We are working in a class level attribute to set all the fields as quoted to make it easy.

Best Regards

Talk to you soon

Jeff Brown

Just curious. Why is the syntax for declaring connections different from the rest of the pipeline? Is it side-effecting?

Ayende Rahien

No particular reason, actually, and a good point for improvement.

Comment preview

Comments have been closed on this topic.


  1. RavenDB 3.5 whirl wind tour: I'll have the 3+1 goodies to go, please - 3 days from now
  2. The design of RavenDB 4.0: Voron has a one track mind - 4 days from now
  3. RavenDB 3.5 whirl wind tour: Digging deep into the internals - 5 days from now
  4. The design of RavenDB 4.0: Separation of indexes and documents - 6 days from now
  5. RavenDB 3.5 whirl wind tour: Deeper insights to indexing - 7 days from now

And 10 more posts are pending...

There are posts all the way to May 30, 2016


  1. The design of RavenDB 4.0 (14):
    05 May 2016 - Physically segregating collections
  2. RavenDB 3.5 whirl wind tour (14):
    04 May 2016 - I’ll find who is taking my I/O bandwidth and they SHALL pay
  3. Tasks for the new comer (2):
    15 Apr 2016 - Quartz.NET with RavenDB
  4. Code through the looking glass (5):
    18 Mar 2016 - And a linear search to rule them
  5. Find the bug (8):
    29 Feb 2016 - When you can't rely on your own identity
View all series


Main feed Feed Stats
Comments feed   Comments Feed Stats