Mar 16 2016

NBench Performance Testing – NUnit and ReSharper Integration

You can run your NBench performance tests using NUnit and ReSharper.

All the logic to run performance tests in NBench is actually embedded in the core library and for this reason an external runner is not strictly required. You can easily integrate your unit testing framework and use your favorite tools to run your performance tests.

In this post, I show you how you can run NBench tests with NUnit and ReSharper.

Full source code is available here

How to create your NBench performance tests

You can create performance tests using attributes as usual.

The only difference is to inherit from a new base class.

PerformanceTestSuite in NBench

The PerformanceTestSuite base class

The abstract class PerformanceTestStuite<T> defines a parametric test in NUnit that run a single performance test (benchmark). The TestCaseSource attribute point to a method that returns all the performance tests so this single NUnit test basically runs all the performance tests defined in the type T.

PerformanceTestSuite to run NBench tests using NUnit and ReSharper

The Benchmarks method use an instance of the ReflectionDiscovery class to find all the benchmarks and then return a TestCaseData for each benchmark found.

The discovery class takes an IBenchmarkOutput instance that is used by the benchmark.Finish() method to notify the result of the performance test. For each benchmark NBench tells us all the assertion results and we can use those information to run a NUnit assert and print useful information to the console.

Integrate NBench with NUnit

Note: the reason why I had to use generics is because NUnit requires the TestCaseSource method to be static and in a static method I don’t have a way to get type information.

Running the tests in ReSharper

This is how running the tests looks like in the ReSharper test runner.

Show NBench tests results in ReSharper test runner

For a failing test, you get a clear error message.

Message from a failed NBench test

For a passing test, you still get information about the actual results of your performance tests.

Messaged from passed NBench tests

Conclusions

I think this is great! You can add this base class into your own project and very easily get your team to start writing automatic performance tests. Of course, the biggest problem is to define what to test but it’s great that now we have some tools that helps us to do it so easily. The learning curve is so slow.

NBench vision is to become a complete, user-friendly and fully extensible framework for automatic performance testing in .NET and more importantly, to make automatic performance testing a regular software development practice. I think this framework is a very good step forward in this direction.

NBench is an open source project. If you like the vision, you can help. Contribution is very welcome. The github page of the project is https://github.com/petabridge/NBench.

Previous posts

 

 

I am a Microsoft MVP and Senior .NET Engineer in Redgate Software. My peers describe me as really enthusiastic, motivated with an infinite passion for software development, a relentless desire to keep learning and an amazing ability to involve others in my pursuit of excellence. I am a proficient C# developer and founder of the Cambridge .NET User Group: the official community on .NET in Cambridge.

Facebook Twitter LinkedIn Google+ YouTube  

Permanent link to this article: https://www.productivecsharp.com/2016/03/nbench-performance-testing-nunit-resharper-integration/