Lazycoder

5Oct/065

Anyone with test framework experience?

Anyone out there with .NET test framework experience want to elighten me?

Which one actually WORKS?

MBUnit, the current favorite amongst the people I read, can’t even load it’s own tests in it’s GUI test runner (at least on my machineS). Let alone run my tests. The documentation consists of a confusing wiki and blog posts professing undying love for the framework. I’d love to pitch in, in the spirit of Open Source, and try to fill in the documentation holes, but who wants to spend a lot of time reflecting over something when there are alternatives out there that are better documented? Heck, I’m just getting my feet wet in this TDD world.

Visual Studio Team System unit testing – works about as well as the rest of Visual Studio. Which means it ALMOST works, but will occasionally just barf up something neat like “Cannot find .Net framework 2.0. blah blah” (which is apparently due to adding references to projects rather than assemblies AND having ClickOnce security enabled. ) Or it won’t let me RUN the tests. Seriously, the little “play” button is disabled in the “test results” window and I can figure out how to enable it. I’ll say this for MBUnit, it’s still documented better than the unit testing is VSTM.

NUnit – comes the closest, except it sometimes won’t load my test projects for some reason. I get a “System.IO.FileNotFound” exception complaining about how it can’t load the project or one of it’s dependencies. *I* can find them all, why can’t NUnit? Then it failed my one and only test, a constructor test. The whole test was two lines amounting to “thing target = new thing(); Assert.IsNotNull(target);”. I got a threading exception. (It looks like it had to do with my overriding the EditingControlShowing event in the DataGridView to enable Autocomplete in some combo boxes in the DataGridView. The exception was a threading exception, telling me to make sure I had [STAThread] in my Main function, which I did. Oddly enough, my only other constructor test, which tested a control in the same namespace also containing a DataGridView with combo boxes that I was manually setting the autocomplete on, would not throw an exception).

Of course, Visual Studio doesn’t recognize any of the other testing frameworks. The NUnit GUI would recognize the VSTM tests, but wouldn’t show any of it’s own tests if the assembly contained VSTM tests. Nice.

Maybe I’m going about this all wrong, maybe you aren’t supposed to test constructors and properties and what not? Maybe you aren’t supposed to use GUI test runners. How can I trust that my unit tests are testing correctly if the testing frameworks aren’t even stable enough to run?

Share and Enjoy:
  • del.icio.us
  • DotNetKicks
  • DZone
  • Reddit
  • Digg
  • StumbleUpon
  • LinkedIn
  • Facebook
  • FriendFeed
  • HackerNews
  • Netvibes
  • Posterous
  • Tumblr
  • Twitter
Comments (5) Trackbacks (0)
  1. Which version of MbUnit are you running? I assume you made sure your test project references the MbUnit.Framework.dll and that you marked up your test classes with the [TestFixture] attribute and your test methods with the [Test] attribute.

    [TestFixture]
    public class MyTests
    {
    [Test]
    public void SomeTest()
    {
    Assert.Fail(“It failed”);
    }
    }

    Put that class in a class library, compile it, then point MbUnit to that assembly.

  2. Well, of course that works. ;) Actually, my code looked like this.

    namespace MBUnitTest
    {
    [TestFixture]
    public class MBUnitTest
    {
    [Test]
    public void FailTest()
    {
    Assert.Fail(“It Failed”);
    }
    }
    }

    I load that in the MBUnit GUI, the test shows up, and it fails as planned. However, the GUI doesn’t show ANY tests in the following code.

    namespace FuzzyDateTextBoxTests
    {
    [TestFixture]
    class FuzzyDateTextBoxTests
    {
    [Test]
    public void TestConstructor()
    {
    FuzzyDateTextBox target = new FuzzyDateTextBox();

    Assert.IsNotNull(target, “Constructor failed”);
    }

    [Test]
    public void TestDateParse()
    {
    FuzzyDateTextBox _fdtb = new FuzzyDateTextBox();
    _fdtb.Text = “07/15/2006″;
    }

    }
    }

    Which confuses me to no end. However, if I add the MBUnitTest code to the FuzzyDateTextBox code and load the assembly in theMBUnit gui, it shows the MBUnitTests, bt not the FuzzyDateTextBox test.

    Any ideas? What am I doing wrong. I swear the code I posted above in the code in it’s entirety.

  3. Em, not that this really solves your problem, but your constructor test will never fail. Calling “new ClassName()” will never return null.

    Also, I noticed that your FuzzyDateTextBoxTests is not declared public. That might be causing your problem where it doesn’t find any tests.

  4. Oh, don’t tell me that I forgot that classes and members are private by default. That fixed the problem in this case with MBUnit. I still had other problems with MBUnit that I haven’t been able to fully explain.

    Will the constructor never fail Kevin? Right now, my constructor is nothing more than the default constructor, in the future I may refactor the class to be derived from something else and call the base constructor in my constructor, and my supers constructor may be more complicated. Winform constructors created by the visual designer ALWAYS call “InitializeComponent” in the constructor, which in turn creates many, many other objects, some of whom have more complicated constructors. In fact, during my NUnit attempts, I had a constructor test fail due to a threading issue, The constructor has never failed during runtime, but my test failed. I suspect due to a bug in the testing framework. What do I do about my unit tests if they are dependent upon bugs in the framework? Do I not write unit tests for things that fail for reasons external to my code? It’s one of my gripes about unit testing, how much is too much. What tests the unit tests I write? How do I verify they are testing the correct thing in the correct maner?

  5. I didn’t mean that the constructor would never fail. But it will never return null. In other words:
    FuzzyDateTextBox target = new FuzzyDateTextBox();

    will never result in “target” being null. If the constructor blows up, it’ll throw an exception, and the Assert will never be run.


Leave a comment


No trackbacks yet.