Integration Tests in ASP.Net
As you probably noticed, I am doing a fair amount of integration testing with ASP.Net currently. I am using Selenium, and after a brief struggle to get myself aligned with it, I think that I like it :-)
It is simple, it is extensible, and it works. Here is a simple example, I needed to check that a drop down is disabled, Selenium has no support for this out of the box, so I could just extend the DefaultSelenium class (in Selenium RC for .NET) and add this:
public bool IsDisabled(string partialElementId)
{
string result = GetElementAttribute(partialElementId, "disabled");
return bool.Parse(result);
}
public string GetElementAttribute(string partialElementId, string attribute)
{
string evalScript = string.Format(@"
this.page().findElement('//select[contains(@id, \'{0}\')]').{1};
", partialElementId.Replace("'", @"\'"), attribute);
string result = GetEval(evalScript);
return result;
}
Note the funky syntax that I have to use in order to get around the ASP.Net element naming. Another thing that I run into was the compilation model for ASP.Net. That one was a biggie.
The problem was that I wanted to specify a default value for page load times (all pages must load in under 0.5 seconds, for instnace). The problem was the just from the command line build, it started giving errors about timeouts when loading a page. Accessing the page manually or running the tests from Visual Stuido always succeeded.
After a bit of head scratching I realized that the problem was that the build recompiled the site, so every time that I hit a new page, ASP.Net had to compile it for the first time. I ended up keeping a list of pages that I already visited and increasing the wait time for pages that I access for the first time.
The last problem that I have is getting the correct information from the pages, let us take a page that threw an error, in the Selenium test, I usually just get an error about an expected element not found. I added an assert that check that "error" and "exception" do not appear on the page, but I didn't figure out yet how to take just the exception text from the page and put it in the error message for the test.
Comments
We gave selenium a try yesterday, and switched to Watin no long after. It's too slow, the firefox support is broken for the latest firefox version, and the API is very limited (not to say it's ugly). With Watin we don't have to add lots of waits to the code too.
Watin is simpler, and has a decent and small codebase.
I'll give WATIN a try, I have tried using WATIR in the past, and it was a PITA, IIRC. The main issues where with handling of new windows, alerts, etc. I have no idea if they are an issue with WATIN, though.
It turns out that Slenium has a very good IDE that can generate really nice C# code when I need, and I can make the tests do intersting stuff.
How do you handle transisions in WATIN, btw?
what you mean by transitions?
Watin supports alerts and those kind of dialogs. Just add a proper dialog handler. The docs might be obscure on that point, though.
Transisions between pages, was what I meant, and as long as we talk about it, what about an ajax call that modify the page that you want to verify.
(think about password stength as an example, you type something, and half a sec later a text appears)
There's no issue with transitions. A click on a link or form submission will block 'til the page loads.
For ajax calls you have to sleep for a couple of seconds, that's all. We've tested much more complex cases using prototype-window lib with no problem.
The only issue I have with Watin is about some method names on their API, but that's more like a pet-peeve.
I like and use Selenium RC too, I've automatized it a bit by creating a template project which automatically starts the server and a web server, if needed. I like the API over the Watin API since I think it's more flexible and allows you to work directly on the DOM.
@Hammet: the official release is broken for FF 2.0.0.1 but the nightly releases work ok, I tested it personally.
Furthermore, the one and biggest advantage of Selenium over Watin is that it allows you to do cross-browser tests, particularly useful when testing web controls and so on.
Simone, I didn't like their API as everything is inputted as a string. Event "wait" expects a string. You also have to manager the waits yourself, I dont have to have tests failing 'cos the build server is overloaded.
I agree that cross-browser is great. When it works...
Well, maybe the scenario in which you are employing the framework is pretty different from mine, I can't get into server overloading issues since I do not have a build server! I appreciate Watin too, but it's not as flexible as Selenium, even if you actually end up paying this flexibility with less reliability.
Hi,
We have been using Selenium (Core/RC/IDE) since mid 2005. We were very happy with it (we do have a few thousend tests). After writing the article about how we do it on my blog, at hammett's suggestion we tried Watin. We made some comparisons, and we ended up with the conclusion that Watin needs less code written (no need for waits), it runs a little faster in some cases (when handling confimations it is a lot slower) and it also has upload support (pretty useless since we usually use flash contols for uploading multiple files at the same time), and last but not least, it is easier to deploy (no java server required).
The unit tests gave us the best documentation, and we started using it in the latest project we are building and so far we are happy with our decision. We did have to stuggle a little with AJAX calls, as Thread.Sleep is not a good solution, until we fould WaitForElementPresent. I wish they also had a WaitForElementNotPresent for ajax calls that make an element dissapear (Selenium had).
Maybe integration/functional testing should have a section of its own in Castle project (documentation/forums). I'd be happy to help, as much as I can.
Thanks,
Dan
Cool Dan. I think you should join watin development list and make this suggestions. We - at Stronghold - are already patching it adding things we need.
You mentioned Flash controls to upload multiple files, what's that?
Hi all,
Hammett did point me to this great discussion (though the watin-development list). First of all use WatiN if you like it. If you don't, be happy with your choice. That's the great thing about choice!
AJAX support should be OK with WatiN. A short explanation:
Each element class (TextField, Button etc) exposes a WaitUntilExists method. If you call this it will wait 30 seconds (default) before throwing a ElementNotFoundException. To make thing easier WatiN does call this method for you when you try to find/acces an element for the first time. For example you have an OK button on your page which appears 5 seconds (java dom injected) after you've done some magic on your webpage. The following code will work since WatiN wait 30 seconds before it throws the exeption.
ie.Button('btnOKid').Click();
You could say it's almost the same as doing:
Button btnOK = ie.Button('btnOKid');
btnOK.WaitUntilExists;
btnOK.Click();
If you need more then 30 seconds, you need to call the overload of WaitUntilExists(int seconds) before accessing a property or method of the button. Say it might take 40 seconds max before the button appears, then you need to write:
Button btnOK = ie.Button('btnOKid');
btnOK.WaitUntilExists(40);
btnOK.Click();
Dan, about WaitUntilNoLongerExists. This can be done too with WatiN. A bit dirty example to show you how:
SimpleTimer timer = new SimpleTimer(30);
do
{
if (ie.Button('btnOKid').Exists)
{
}
else
{
}
} while (!timer.Elapsed)
if (ie.Button('btnOKid').Exists)
{
throw new LetsThrowSomeMudException(); // You'll have to implement that exception yourself :-)
}
Thanks for the idea however, I will put it on the TODO list.
BTW since I'm here, I like to give Ayende a bigs thumbs up for RhinoMocks. Started using it some weeks ago and I'm hooked!
Hi,
I'll be more then happy to help with WatiN. I have noticed a problem yesterday:
for
window.open(...
it tells me there is no HtmlDialog (ie.HtmlDialogs.Count), but for
window.showModelessWindow it works. Unfortunately Mozilla only works with the second. Is there something I am missing?
About WaitUntilExists, we were already using it and it is a great option.
About flash uploader, with the newer Flash versions you have an upload control that allows you to upload more files at once http://www.codeproject.com/aspnet/FlashUpload.asp
Thanks,
Dan
Dan,
window.open does open a new Internet explorer window (instead of a htmldialog/pop up). So use IE.AttachToIE(findBy) instead of ie.HTMLDialogs.
Jeroen
Comment preview