Profiling MSTest (and why I had to do it)
I got an email earlier this week from a member of our team saying that they had noticed odd behavior in our unit tests. They said when the ran the tests from Visual Studio, the memory allocation increased by about 500MB. When they ran the tests again, the memory increased another 500MB, and so forth. The system component that developer was working on had the most tests out of all of our components, so he was noticing this before others did.
First of all, it seemed odd that unit tests would cause Visual Studio’s memory to increase since the tests are run in another process. I knew it would probably be impossible to profile Visual Studio itself, so I profiled out tests from the command line to see what if anything I could figure out. A couple of tricks to profiling MSTest.exe:
- Launch mstest.exe with the /noisolation option so that the tests are run in process
- When you use the /noisolation option, you have to replace \Program Files\Microsoft Visual Studio 9.0\Common7\IDE\MSTest.exe.config with the App.config for your tests. You also need to add the this:
<runtime>
<assemblyBinding xmlns=”urn:schemas-microsoft-com:asm.v1″>
<probing privatePath=”PrivateAssemblies;PublicAssemblies”/>
</assemblyBinding>
</runtime>
The first thing I noticed after all of the tests ran was that 98% of the memory was being held by System.String. When I looked to see what objects were referencing System.String, one thing blaringly stood out:
All of the standard out was being held in a single member variable from the whole test run. Since I was not sure if Visual Studio was doing the same thing, I turned off logging in our application and ran the tests again. Surprise, that worked. Visual Studio no longer was increasing in memory usage.
Yes, I realize 800 MB of log output is a lot but that is not really the issue – I’m not sure why VS keeps that output in memory. Once it has the test results, I believe it writes it out to a file so why keep it around?
