Feb 29 2016

NBench Testing – Garbage collection

Full source code is available here

This post describes how to create garbage collection tests using the NBench framework. If you are interested in measuring performance or memory usage check out the previous posts:

Method Under Test

Let’s take a method that intentionally create a lot of garbage on the heap and indirectly force garbage collection to occurs.

2016-02-28 23_54_21-Start

The first allocation of the variable data inside the inner loop creates a lot of garbage. This garbage will be collected pretty soon by the garbage collection at generation 0.

We also create a copy of the data and append it into a list. The elements inside the list will survive for quite some time (for the duration of the inner loop) and so will likely survive garbage collection at generation 0. These elements will be move into the generation 1 and will be collected as soon as the garbage collection at generation 1 kicks in. However, some of them might survive and be collected only during a full garbage collection (generation 2).

For more information about garbage collection I recommend reading the article Fundamentals of Garbage Collection.

Measuring Garbage Collection

You can use NBench to easily measure the total number of garbage collections happening at each generation. To do so you need to use the TestMode.Measurement and the GcMeasurement attribute.

2016-02-28 23_40_56-Start

After running the NBench runner, you can see the max, average, min and standard deviation of collections for each generation.

2016-02-28 23_32_49-Photos

Testing Garbage Collections

If you use the TestMode.Test you can assert the garbage collection throughput and/or the total collections at each generation.2016-02-28 23_41_05-Start

The GcThroughputAssertion attribute allows you to assert that the number of collections per second for a specific generation must be less than a certain value.

The GcTotalAssertion attribute allows you to assert that the total number of collections of a specific generation is less than a certain value. Ideally you would like to minimize the number of generation 2 collections.

You can see that all the test pass.

2016-02-28 23_39_06-StartConclusions

I think that NBench assertions for garbage collections can be useful and they are very easy to use in order to create a set of regression tests.

Most of the time you don’t want to test generation 0 and 1 collections (because they are usually relatively quick) and you want to just focus on making sure that full garbage collections at generation 2 do not happen or are very few. Repeated generation 2 collections is what usually generates serious performance problems. NBench can surely be one of your tool to identify those problems and avoid future regressions.

Previous posts



Permanent link to this article: https://www.productivecsharp.com/2016/02/nbench-testing-garbage-collection/

  • […] NBench testing garbage collection by Andrea Angella. […]

  • >