Performance profiling Nunit tests with ANTS Performance Profiler

I was writing up a Dev at Work blog post when I realized that the post was getting rather long, and that it probably would be better to post a section of it separately. Yesterday, and today, I’ve been working on a story regarding our build service. For some reason our deployment builds have gotten slower and slower, and we are at a point where waiting for the packages to be ready for QA is a pain. I started investigating, and noticed that our integration tests are very slow, and are not scaling well. It takes nearly 20 minutes to run 400+ tests, and considering that we want to more than double our test coverage that wont work long term for us.

But how do you profile tests?

If you have Visual Studio Enterprise, the you can profile and analyze separate tests by right clicking on the test in the Test Explorer window.

However, we are using NUnit and Test Exploring is having trouble picking up the tests. There are other ways, using Visual Studio, but I’m a fan of RedGate profiling tools, and decided to use the Performance Profiler instead as I know how it works. This is my workflow and what I discovered.

You can do this using the NUnit GUI tool or the console

I started the profiler, created a new session and pointed to the NUnit executable path

performance profiling integration tests with nunit 00

performance profiling integration tests with nunit 0

If you don’t have the GUI tool installed you can also pass in the console with commandline arguments (path to the test dll and filters if you need them)

performance profiling integration tests with nunit 1_0

The first thing I noticed it that it’s not CPU, memory or disk that is the problem. I already knew this from looking at the PerfMon data in TeamCity. Using the treeview, and wall-clock time as a metric I could see that every test class had an initial SQL preparation that takes a lot of time.

performance profiling integration tests with nunit

The problem seems to be that each collection of tests ruins a setup script that takes around 20 seconds. This adds up quickly, and wont scale. I have a few ideas in regards to how we can solve this, but I’ll save that for a different blog post :)

Per request: This is how you can use NUnit filters to run only parts of a test assembly (example with PowerShell)

 
$testDll = ""
$nunitExePath =""

Write-Host "***** Test Start *****" -Foreground Yellow

# Konstrukt.IntegrationTest.Given_ApiKeyValidator is the namespace I'm filtering on
# You can also pass in class
$result = &amp;"$($nunitExePath)" "$($testDll)" --test:Konstrukt.IntegrationTest.Given_ApiKeyValidator

$duration = $result | where {$_ -like "*Duration*"}
$testCount = $result | where {$_ -like "*Test Count*"}

echo $duration.Trim()
echo $testCount.Trim()

Write-Host "***** Test Stop *****" -Foreground Yellow

Performance profiling Nunit tests with ANTS Performance Profiler

Comments