Ayende @ Rahien

Hi!
My name is Oren Eini
Founder of Hibernating Rhinos LTD and RavenDB.
You can reach me by phone or email:

ayende@ayende.com

+972 52-548-6969

, @ Q c

Posts: 5,968 | Comments: 44,489

filter by tags archive

Bisecting RavenDB


One of the results of expanding the RavenDB’s team is that we have to deal with a lot more pull requests. We have individual developers working on their features, and submitting pull requests on a daily basis. That usually means that I have to merge anything between ten to thirty pull requests per day.

Given our test suite runtime, it isn’t really reasonable to run the full test suite after each pull request, so we merge them all, review them, and then run the tests. The problem then is that if there was a test failure… which pull request, and which commit cause it?

That is why git has the wonderful git bisect command. And we have taken advantage of that to enable the following development time feature:

./bisect.ps1 HEAD HEAD~10 General_Initialized_WithoutErrors

It will output something like:

631caa60719b1a65891eec37c4ccb4673eb28d02 is the first bad commit
commit 631caa60719b1a65891eec37c4ccb4673eb28d02
Author: Pawel Pekrol
Date: Fri May 9 08:57:55 2014 +0200
another temp commit

Pin pointing exactly where the issue is.

And yes, this is here as a post primarily because I might forget about it.


Comments

Charles Bryant

I know this post is for your memory purposes, but this is very interesting to me. Can you reveal a little of the secret magic in that PowerShell script? What is your general algorithm to find the culprit? What did you do to identify that commit?

Thanks

dorony

May I ask why isn't everyone working on the same branch? Why deal with constant merges?

Jeremy Wiebe

@Charles Bryant: git bisect supports a script mode where you can pass it a script that returns 0 for success and non-zero for failure. Git bisect will then do the bisect work, running the script at each commit it arrives at and deciding if the commit is good or bad based on the result of running the script. I would guess Ayende has hooked up the automated tests to the script so that if a test fails it returns non-zero (thus telling git bisect that the commit is broken).

There's a good discussion of 'git bisect' here: http://git-scm.com/book/en/Git-Tools-Debugging-with-Git (there's a small blurb about the scripting support at the end of this page).

Ayende Rahien

Charles, We are just shelling out to git bisect, with a script of our own that knows how to build and run our project.

Ayende Rahien

Dorony, We have up to 10 people working on RavenDB concurrently. Even with everyone starting from the same branch in the morning, by evening, they all made their own changes.

Damian Hickey

Given our test suite runtime, it isn’t really reasonable to run the full test suite after each pull request

Erm.. Teamcity? http://blog.jetbrains.com/teamcity/2013/02/automatically-building-pull-requests-from-github-with-teamcity/ . This is also great to give a contributor feedback that their PR didn't break anything before you look at it.

If the build queue grows, just get more agent licences and use EC2 integration for automagic scaling. http://www.jetbrains.com/teamcity/features/amazon_ec2.html

(I've been harping on to you guys about a proper CI setup for a while now, I won't mention it again).

Ayende Rahien

Damian, We have roughly 10 PR per day, often more than that. Our full build process take too long to run this 10 times a day, and having 10 build servers is a bit much.

Damian Hickey

Oren, I'm not buying that. It's a highly parallel problem - it's just a single AMI that teamcity will automatically spin up/down instances of. Given the number of people and PRs 10 build agents running in parallel is really not much at all. That'll be the C in CI ;) (Also the TC->Github PR notification integration is useful.)

Comment preview

Comments have been closed on this topic.

FUTURE POSTS

No future posts left, oh my!

RECENT SERIES

  1. Career planning (6):
    24 Jul 2015 - The immortal choices aren't
  2. Production postmortem (4):
    23 Jul 2015 - The case of the native memory leak
  3. API Design (7):
    20 Jul 2015 - We’ll let the users sort it out
  4. What is new in RavenDB 3.5 (3):
    15 Jul 2015 - Exploring data in the dark
  5. The RavenDB Comic Strip (3):
    28 May 2015 - Part III – High availability & sleeping soundly
View all series

RECENT COMMENTS

Syndication

Main feed Feed Stats
Comments feed   Comments Feed Stats