You can run your NBench performance tests using NUnit and ReSharper.
All the logic to run performance tests in NBench is actually embedded in the core library and for this reason an external runner is not strictly required. You can easily integrate your unit testing framework and use your favorite tools to run your performance tests.
In this post, I show you how you can run NBench tests with NUnit and ReSharper.
How to create your NBench performance tests
You can create performance tests using attributes as usual.
The only difference is to inherit from a new base class.
The PerformanceTestSuite base class
The abstract class PerformanceTestStuite<T> defines a parametric test in NUnit that run a single performance test (benchmark). The TestCaseSource attribute point to a method that returns all the performance tests so this single NUnit test basically runs all the performance tests defined in the type T.
The Benchmarks method use an instance of the ReflectionDiscovery class to find all the benchmarks and then return a TestCaseData for each benchmark found.
The discovery class takes an IBenchmarkOutput instance that is used by the benchmark.Finish() method to notify the result of the performance test. For each benchmark NBench tells us all the assertion results and we can use those information to run a NUnit assert and print useful information to the console.
Note: the reason why I had to use generics is because NUnit requires the TestCaseSource method to be static and in a static method I don’t have a way to get type information.
Running the tests in ReSharper
This is how running the tests looks like in the ReSharper test runner.
For a failing test, you get a clear error message.
For a passing test, you still get information about the actual results of your performance tests.
I think this is great! You can add this base class into your own project and very easily get your team to start writing automatic performance tests. Of course, the biggest problem is to define what to test but it’s great that now we have some tools that helps us to do it so easily. The learning curve is so slow.
NBench vision is to become a complete, user-friendly and fully extensible framework for automatic performance testing in .NET and more importantly, to make automatic performance testing a regular software development practice. I think this framework is a very good step forward in this direction.
NBench is an open source project. If you like the vision, you can help. Contribution is very welcome. The github page of the project is https://github.com/petabridge/NBench.
- NBench Performance Testing – Code Throughput
- NBench Testing – Memory Allocations
- NBench Testing – Garbage Collection