Ayende @ Rahien

My name is Oren Eini
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by phone or email:


+972 52-548-6969

, @ Q c

Posts: 6,026 | Comments: 44,843

filter by tags archive

Rhino ETL & FileHelpers Integration

time to read 9 min | 1662 words

Well, this image really excites me. It excites me because I got this after integration FileHelpers into Rhino ETL. This image is the result of an ETL script that joined a file against a table, did some additional processing on it and push it to a table in a third database.

I must say that FileHelpers has made this ridiculously easy to do.  All I have left to do is figure out how to test such a thing effectively.


For reference, here is the complete script:

class Customers:
      CustomerID as string
      CompanyName as string
      ContactName as string
      ContactTitle as string
      Address as string
      City as string
      Country as string


      ConnectionType: SqlConnection,
      ConnectionString: "Data Source=localhost;Initial Catalog=ETL_Test; Integrated Security=SSPI;"

      ConnectionType: SqlConnection,
      ConnectionString: "Data Source=localhost;Initial Catalog=Northwind; Integrated Security=SSPI;"


source CustomersFile:
            for customer in Read(typeof(Customers)).From("""Files\CustomersTab.txt"""):
                  SendRow( Row
                        CustomerId: customer.CustomerID
                        CompanyName: customer.CompanyName,
                        ContactName: customer.ContactName


source OrdersFromDatabase, Connection="Northwind":
      Command: "SELECT * FROM Orders"


join CustomersAndOrders:
      if Left.CustomerID == Right.CustomerID:
            Row.OrderID = Right.OrderID
            Row.CompanyName = Left.CompanyName
            Row.ContactName = Left.ContactName


 destination Final, Connection="DestinationDB":
      Command: "INSERT INTO OrdersWareHousing VALUES(@OrderID, @CompanyName,@ContactName)"


pipeline OrdersWareHouse:
      CustomersFile >> CustomersAndOrders.Left
      OrdersFromDatabase >> CustomersAndOrders.Right
      CustomersAndOrders >> Final


 target default:


Markus Zywitza

That's marvellous!

Is there any documentation on Rhino ETL other than your blog entries? If not, is there a good starting point in the source code to look at?


Ayende Rahien

Markus, no documentation yet.

The best place to start look is in the integration tests, I'll upload soon the changes required to support FileHelpers


Wow nice to see the seamless integration !! =)

Thanks a lot for your words Ayende, them are a great motivation to continue developing and releasing the FileHelpers

There are large hours enhacing the API make it more and more usable, mostly thanks to the CLR core team with his:

Designing .NET Class Libraries


And with the Excellent BOOK: Framework design guidelines.

In the company that I work for Im the responsable for our base framework to generate code and so. So I love Framework and LIbrary design, also to design winforms control.

I wish to found time soon to help with a Rhino ETL GUI or some examples

You are doing a OUTSTAND WORK DUDE !!!

About the code you need to add the [FieldQuoted(OptionalForBoth)] because without quotes some values can contains a tab and so fail to load.

We are working in a class level attribute to set all the fields as quoted to make it easy.

Best Regards

Talk to you soon

Jeff Brown

Just curious. Why is the syntax for declaring connections different from the rest of the pipeline? Is it side-effecting?

Ayende Rahien

No particular reason, actually, and a good point for improvement.

Comment preview

Comments have been closed on this topic.


No future posts left, oh my!


  1. Technical observations from my wife (3):
    13 Nov 2015 - Production issues
  2. Production postmortem (13):
    13 Nov 2015 - The case of the “it is slow on that machine (only)”
  3. Speaking (5):
    09 Nov 2015 - Community talk in Kiev, Ukraine–What does it take to be a good developer
  4. Find the bug (5):
    11 Sep 2015 - The concurrent memory buster
  5. Buffer allocation strategies (3):
    09 Sep 2015 - Bad usage patterns
View all series


Main feed Feed Stats
Comments feed   Comments Feed Stats