XPCShell test output changes

13 June, 2013 § Leave a comment

Those of you using XPCShell tests may notice some slightly improved output from the test macros that landed recently on mozilla-central.

Previously, if you called do_check_null(null), you would get an output that was similar to:

[TEST-PASS] null == null

However, if you tried do_check_null("null"), you would get an output that was similar to:

[TEST-FAIL] null == null

This isn’t very helpful, and I ran into this exact issue while working on some code that uses observers. Since the data is passed as JSON, when JSON.stringify(null) is called the result is “null”. The test runner showing null == null as a failure isn’t too helpful :-P

With this recent change, all string arguments will be adorned with double-quotes.

This means that calling do_check_null("null") will now output:

[TEST-FAIL] "null" == null

The same change was made for all of the other do_check_* and do_print macros.

Google Test only executing one test?

30 March, 2011 § 1 Comment

Earlier today while pair-programming, Angie and I ran into an issue where Google Test was only executing one unit test of the 150 or so in our test suite. We weren’t sure what the cause could be. We switched machines and all the unit tests ran fine on my machine.

We first checked the WinMain entry point to see if some configuration flag of GTest had been accidentally flipped. A quick diff showed us that nothing had changed.

So what could be differentiating Angie’s computer from mine? Well it turns out that Angie really enjoys using keyboard shortcuts and macros to help her do her work. A couple weeks ago I showed her some of the Visual Studio macros that are used in developing for Chromium. Angie saw the list of macros and quickly installed the “Alternate between source and header file”, as well as the “Run the currently selected google test”.

It  turns out that the “Run the currently selected google test” works by adjusting the command line arguments that are passed to the GTest executable. This argument sets a filter on the test runner to only run the tests that match the given filter, which in this case happens to be the name of the currently selected test.

As the Jing states, simply clearing out the command argument fixed our problem. If you’re using the macro mentioned earlier, you can also place your cursor outside of a specific test case, run the macro, and the filter will be cleared.

I hope this post comes in handy for anyone else that got stuck in our position.

Using Sikuli IDE to test a Google Chrome Extension

3 January, 2011 § 1 Comment

On Christmas Eve I decided to start working on my first publicly available Google Chrome extension. I blogged previously about it, but today I wanted to cover how I test that the extension is working.

The extension allows a user to take a YouTube video and bring it to the full window size of the browser. This creates a full screen that will not exit when another window is clicked on.

To use the extension, there is a “page action” that is located in Google Chrome’s address bar. When clicked, the video will either expand or contract. After the initial release, I made public the manual test cases for the extension. When I found time later, I automated them with Sikuli.

Here is a video of the test cases being run:

I have uploaded the Sikuli test case files to the project website as an example on unit testing with Sikuli.

Previously, I’ve used Selenium IDE for web-based testing and I know of some coworkers that use AutoHotkey. Have you ever used an app like these for unit testing?

Unit testing code behinds in ASP.Net Web Forms

13 June, 2010 § 7 Comments

Have you ever inherited an ASP.Net website with waaaaaaay too much logic in the code behinds?

A while back I came up with a solution to solving this problem. I don’t have a name for it, but if you do, please leave a comment at the end of this post.

The problem that I was trying to solve is that I wanted to make a new page and implement it using Test Driven Development. The hard part was that there was a lot of code that basically dealt with the visual aspect and interaction of the page but not much with the data model. I needed to find a way to write tests around this code, as the data model code already had a framework for writing tests.

Here is an example of the before-case:

public class CodeBehind : System.Web.UI.Page {
   protected void btnFindUser_Click(Object obj,  EventArgs e) {
      if (Page.IsValid) {
         var dbConnection = new DatabaseConnection();
         var users = dbConnection.GetUsers( txtQuery.Text );
         if ( !users.empty() ) {
            dataTable.Bind(  users );
         }
      }
   }
}

What I came up with was an introduction of a “controller” class that lived outside of the ASP.Net Web Forms project. Here is an example of the after-case:

// CodeBehind.aspx.cs
public class CodeBehind : System.Web.UI.Page {
   protected void btnFindUser_Click(Object obj,  EventArgs e) {
      controller.FindUser( Page.IsValid, txtQuery.Text, dataTable );
   }
}

// CodeBehindController.cs
public class CodeBehindController {
   public void FindUser( bool pageIsValid, string query, DataTable dataTable ) {
      if ( pageIsValid ) {
         var dbConnection = new DatabaseConnection();
         var users = dbConnection.GetUsers( query );
         if ( !users.empty() ) {
            dataTable.Bind( users );
         }
      }
   }
}

// CodeBehindControllerTests.cs
public class CodeBehindControllerTests {
   public void FindUser_WithNoUsersFound_DataTableIsEmpty() {
      DataTable table = new DataTable();
      CodeBehindController controller = new CodeBehindController();

      controller.FindUser( true, "ImpossibleUserToFind", table );

      Assert.IsTrue( table.empty() );
   }
}

With that, I can now mock out the DatabaseConnection and the DataTable and I’m good to go. By passing in Page.IsValid instead of the whole Page object, I’m able to provide types that are easier to instantiate and I also now have code with less dependencies.

Getting this to work is actually very simple. Just introduce another class and put all of your logic in that class. There shouldn’t be a single conditional in your *.aspx.cs page. As long as you follow this pattern, it should be pretty easy to TDD your next ASP.Net Web Forms page.

Any questions? Leave a comment and I’d be glad to help.

Finished Reading Clean Code

20 December, 2009 § 2 Comments

Today I finished reading Clean Code by Robert Martin aka Uncle Bob. I found out about the book from some coworkers during our weekly programming-discussion meetings. The book claims to leave its readers understanding:

  • How to tell the difference between good and bad code
  • How to write good code and how to transform bad code into good code
  • How to create good names, good functions, good objects, and good classes
  • How to format code for maximum readability
  • How to implement complete error handling without obscuring code logic
  • How to unit test and practice test-driven development

Part 1: Principles, Practices, and Patterns

I really liked part one of the book. The principles, practices, and patterns are well thought out and provide a style of programming that is, unfortunately, rarely seen. Since reading this part, I’ve been writing smaller functions (about 4-5 lines at max) and have seen a payoff in the amount of reusability and readability. Chapters 1-7 read quickly, are informative, and are really great pieces to read.

Part 2: Case Studies

The pace of the book really drops off after chapter 7, and each subsequent chapter I lost more of that urge to pick up the book and continue reading. The final nail in the coffin that actually put me to sleep once was the Case Studies part (chapters 14-16). The example of the Args module was too large and my internal-brain compiler just couldn’t handle JITting three pages of code at once. I pushed through and read this whole part, but I wouldn’t recommend you doing the same.

Part 3: Code Smells

Part three is one chapter long and focuses on Code Smells. Code Smells are patterns in code that when seen give a bad taste in your mouth. They are implementations that shout that they could have been done better. The longer the code smells, the worse it gets, until finally the abomination that once started out as a faint odor now is the muskiness in the room that scares off developers from maintaining the module. This chapter is important to get familiar with. It might not be too useful to memorize the different smells, but knowing which ones exist is good enough, because you can always come back to this chapter to see how the code can be refactored.

Unit Testing

The last item I wanted to cover was unit testing. While the book mentions unit testing a lot, and devotes a chapter towards it, I don’t feel like developers will walk away from this book with an understanding of unit tests and test-driven development. Any developer looking for further understanding on unit tests should pick up a book that is devoted to the practice, since there is just far too much to cover in one chapter. I would recommend reading Test Driven Development: By Example.

Moving Away from Exceptional Code

14 June, 2009 § Leave a comment

I once took a training course taught by Ron Jeffries and Chet Hendrickson on Test Driven Development. One of the participants in the course asked both Ron and Chet what are their recommendations on testing exceptional code paths. At first I didn’t think much of the question, since all the unit testing frameworks I’ve used have built-in ways to test exceptional code paths.

  • VSTS Unit Testing (.NET) has an attribute that you can put on a test to say that it is expecting an exception:
    [TestMethod, ExpectedException( typeof ( ApplicationException ) ) ]
    public void CreationOfFooWithNullParameterShouldThrowApplicationException()
    {
       var foo = new Foo( null ); //should throw an ApplicationException
    }
  • Google Test (C++) places a macro around the method call that is expecting an exception to be thrown:
     ASSERT_THROW(Foo(5), bar_exception);
  • unittest (Python) has a special assert to test if an exception was raised, similar to Google Test:
    def testsample(self):
       self.assertRaises(ValueError, random.sample, self.seq, 20)

The answer that Ron gave wasn’t expected, but led me to think about the way that we use exceptions within our codebase.

From what I remember, Ron said that he simply doesn’t test exceptional cases. How? He doesn’t have exceptional cases. He recommended using the Null Object pattern to get away from most of the exceptional cases.

Developers often have to pollute their code with checks to see if a value is null, like so:

XmlNode* userNode = xmlParser.GetNode( "user" );
return ( userNode == NULL ) ? "" : userNode->Text();

In the case above, the call to GetNode would search the XML document for a node with a tagName of “user”. If it finds it, it will return a pointer to the data. If not, then it will return NULL. This makes a lot of sense to developers, but it ruins some of the value of abstraction and good object-oriented programming.

First, any client who makes this call to GetNode has to know how GetNode works (they have to know that GetNode will return NULL if it can’t find the node). Second, there is a type check going on here, basically a runtime check to see if the type returned from GetNode is an actual XmlNode or if it is not.

One way to refactor away from this is to use the Null Object pattern. Martin Fowler talks about the Null Object pattern in his book, Refactoring: Improving the Design of Existing Code. In the section entitled, Introduce Null Object, there is a story told by Ron Jeffries describing how he has used Null Object to treat missing objects as actual objects.

The story goes over how if a record didn’t exist in the database, they would return an object like MissingPersonRecord, which would be derived from PersonRecord. All calls to methods on a MissingPersonRecord would simply return some default behavior and not complain. For example, if the MissingPersonRecord is asked for their name, they would respond with a blank string.

This allows any caller of the methods to not have to worry about missing people in the database. See how the code has changed:

XmlNode* userNode = xmlParser.GetNode( "user" );
return userNode->Text();

There is now no change in behavior if the node doesn’t exist, and there is also no knowledge about how GetNode works (besides the fact that the caller knows that it will never return NULL ;) ).

One library that I have used that follows this pattern is the TinyXml library in C++. When a caller asks for an XML node, there is no checking to see if that node really exists before making the call. This allows you to write code like so:

TiXmlNode* userNode = xmlParser.GetNode( "user" );
return userNode->FirstChild()->FirstChild()->Text();

A disadvantage to this is that when a Null Object has to be used, the error in the code can take a while to find. You may not know that there is a problem until a user object is displayed on screen without a required field like a “name”.

To solve this, you would have to debug and find the Null Object to determine why it is not a real object. Another approach could be to have the Null Object log a message in the Event Log when specific methods are called on it, but this can create too much noise, especially if you are using the Null Object pattern in the way that TinyXml does.

So, that is a little introduction on the Null Object pattern. To summarize, you can use the Null Object pattern to improve abstraction and polymorphism, and to reduce the lines of code per method.

Using Google Mock outside of Google Test

14 May, 2009 § 8 Comments

I’ve just finished getting an implementation going of MSTest with GMock, and I wanted to document all things neccessary to use GMock in a testing framework other than GTest.

To get GMock to build in VS2008, I had to install the Visual Studio 2008 Feature Pack. This was so I could get full TR1 support, specifically std::tr1::tuple. Other coworkers of mine haven’t needed to install the Feature Pack, though I presume that they didn’t need to because they had SP1 installed (which I thought I had installed). After installing the Feature Pack, I then had to reinstall SP1 due to some odd compiler errors that were occurring due to mismatched compiler versions.

Now some quick gotchas with GMock:

  • If you use the included gmock_gen.py script to generate your mocks from your pure abstract classes, then you will need to make sure that there are no default arguments in any of your mock’s methods.
  • When you add expectations to a mock, you should call VerifyAndClearExpectations at the end of your test method. You should assert that this method returns True. If you don’t include this method call, then your testing framework won’t be able to catch the GTest exceptions that will get thrown if the GMock’s expectations aren’t met.

Those are the only points I wanted to put out there at this point. When I run in to more bumps in the road I’ll be sure to publish them.

Where Am I?

You are currently browsing entries tagged with unit testing at JAWS.

Follow

Get every new post delivered to your Inbox.

Join 1,015 other followers