Eric Hauser’s Blog

I have had the privilege of doing some work for Ayende for the past month to add support for Hibernate to his excellent NHibernate Profiler.  What does this mean exactly?  Well, Hibernate is now supported as a first class citizen in the profiler.  We have tried to make sure that there is a 1:1 feature mapping between the NHibernate and Hibernate support.  I am happy to report that all applicable warnings and information that you receive for NHibernate is also displayed for Hibernate.  There are of course differences between NHibernate and Hibernate — such as Hibernate does not support statement level batching options, multiquery, or future queries – which is reflected in the documentation.

Since many developers use Hiberante with Spring, we have tried to make Spring integration as seamless as possible.  If you are using  Hibernate with Spring, configuring your application to be profiled is as simple adding a JAR and a custom attribute to your Spring configuration.  Here you can see where I have added support for profiling to an Appfuse demo project:

   1: <!-- Hibernate SessionFactory -->

   2: <bean id="sessionFactory" class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean">

   3:     <property name="dataSource" ref="dataSource"/>

   4:     <property name="configLocation" value="classpath:hibernate.cfg.xml"/>

   5:     <property name="hibernateProperties">

   6:         <value>

   7:             hibernate.dialect=${hibernate.dialect}

   8:             hibernate.query.substitutions=true 'Y', false 'N'

   9:             hibernate.cache.use_second_level_cache=true

  10:             hibernate.cache.provider_class=org.hibernate.cache.EhCacheProvider

  11:         </value>

  12:         <!-- Turn batching off for better error messages under PostgreSQL -->

  13:         <!-- hibernate.jdbc.batch_size=0 -->

  14:     </property>

  15:     <hprof:profiler />

  16: </bean>

This, of course, is just one of the many different ways that you can configure the profiler.  Here is a screenshot of me profiling an Appfuse application.  The top few sessions are profiling unit tests and the remaining sessions (with the URLs) are profiling the web application:

Currently, the product is in closed beta and we are looking for beta testers.  If you are interested, then please contact either Ayende through his blog and he will get you all setup.  Other places you can provide feedback are the user group and the feedback site.  Although I may be a little biased, I think this application can add a lot of value for anyone who is using NHibernate and Hibernate.

I was recently talking with a coworker who said that they write their unit tests in Groovy because unit tests are not production code so the language does not matter.  Of course, I think Groovy is a fine choice for producation applications, but I am ignoring that point for the purposes of this post.  I could not disagree with my coworkers comments more.  Unit tests are as important as your code itself.  I do not think that comment needs more explanation than that, only that I find it interesting that many people still see unit tests as a second class citizen in a code base.

The other interesting thing is that the current goal is to get 80% unit tests passing on all projects (the projects that I work on have 100% all the time, otherwise I would lose my mind).  This is a strategy I disagree with.  Let’s say that over time your project has gotten to a state where you only have 50% of the unit tests passing.  As far as I am concerned, this is equivalent to having 0% of the tests passing.  Why?  Because developers see tests in two colors: red and green.  You will quickly find that the 50% will continue to down because no one is actually noticing that they are breaking tests as they are making code changes.  On a few large projects I have worked on, I have seen where a single failing unit test quickly becomes 20. 

Assuming that because of business goals and quantity of tests to fix, that just fixing all of the tests immediately is not an option.  A much better strategy, in my opinion, would be to ignore the tests that are failing and say that all projects must have 100% unit tests passing.  Then, make your goal to be to work your way down from 50% ignored tests to 0% ignored tests over a reasonable period of time. 

My company has decided to close its facility here in Indianapolis, so I am currently job hunting.  Its turns out that suddenly finding a bunch of free time on your hands is bad for the wallet.  It also does not help that a large number of engineers in our office now have very little to do for the next 60 days but talk about things like computer hardware (in between interviews of course).  My machine at home is close to being three years old, so I have decided that it is fully depreciated.  Inspired by the likes of Scott Hanselmann and James Kovacs I have decided to build my own Ultimate Devloper Rig and write about it.

The System

Case - Antec P182

Pretty original choice, huh?  I really wanted to buy the Cooler Master HAF 932 after reading the review, but then I went to Fry’s.  Here is the size comparison between the P182 and the 932:

I was a little concerned that the 932 might not fit under my desk so I went with the P182.  My other concern was that the the amount of fans in the Cooler Master would  make the noise level of the case unacceptable, but a coworker of mine has one and he says that it is not that bad.  If you are not overclocking your machine, you probably do not even need to hook all of the fans in that case up.

 Processor - Intel Core i7 920

When I first starting looking, I assumed that I would be going with a Core 2 Quad system.  If you go Core i7, you are going to pay a lot more for the motherboard, processor, and RAM, right?  Turns out that you won’t be paying quite the premium you would have paid 3 months ago.  My office is located right across the street from Fry’s and they had an great deal on the Core i7 920 - $229.  NewEgg’s current price for a i7 920 is $289.99 and an entry level quad core processor, the Q8300, is currently running $189.99.  If you look at the benchmarks for the Core i7 920 vs. say the Core 2 Quad Q9650 (currently $339), the 920 performs significantly better in a lot of benchmarks and slightly better in some others.  Also, the i7 is a quad core processor with hyper threading (2 threads per core) which means that the operating system will think that the processor has 8 cores.

 Motherboard - ASUS P6T Deluxe LGA 1366 Intel X58 ATX

The motherboard is definately the place where going with the i7 will hurt you on price.  On average, X58 motherboards are about twice as expensive as their comparable P45 counterparts.  In fact, some of the prices for the Core 2 boards are moving closer to 3 times cheaper.  If you think of your machine as a one time sunk cost, then saving $150-$200 on a motherboard is fairly significant.  However, if you are planning on upgraded your processor and RAM a couple of years down the road, you definately do not want to be stuck with a motherboard that only supports Core 2 processors and DDR2 RAM.

After reading some reviews on the current choices for i7 motherboards, I decided to go with the ASUS board.  The board has features such as dual Gigabit LAN, power on bios, 8 SATA ports, 6 DIMM slots, and 3 PCI express slots.  This board supports both Crossfire and SLI — so you have flexibility in which video card manufacturer you go with.  My choice really came down between this board and the Gigabyte GA-EX58-UD4P.  You could probably save yourself an extra $50-$75 bucks by going with this board or the cheaper ASUS if you don’t want all the bells and whistles of the P6T Deluxe.

 Power Supply - CORSAIR 650TX 650W

After picking my motherboard, I moved on to the power supply.  It is important to make sure that your PSU is certified for both SLI and Crossfire, and this one is.  It has connectors for eight SATA drives.  NewEgg had a really good deal on this particular PSU after instant and mail-in rebates ($79), and I could not find anyone with anything bad to say about it when searching. 

 RAM - G.SKILL 6GB (3xGB) DDR3 1333 Triple Channel Memory

Since there are so many different options for RAM, I decided to use the poll the audience approach for choosing.  I searched for "DDR3 1333 triple channel 6GB" on NewEgg and looked at the results.  There about 10 options, all within $20-$30 of each other.  I decided to go with the G.SKILL because it was inexpensive, had the most reviews, and had a 84% excellent rating.

 Boot Drive - Western Digital VelociRaptor WD3000HLFS 3000 GB 10,000 RPM

This is the item that I thought about the most.  It is probably going to be a few months before there is some good data on reliability and performance for SSDs.  I was actually reading a i7 builder’s guide on AnandTech this morning (which incidentally came out after I made my purchases) and it says that they are working on reviewing the latest cheaper SSD drives.  You can get a 64GB GSKILL SSD for $139 or a 128GB for $234.  Even a 64 GB drive would give you plenty of room as a boot drive.  The Intel X25-M SSDs currently boost much higher read times than the cheaper SSDs and they are significantly more expensive, 80 GB for $339, than the cheaper models.  So, I decided to go with the tried and true VelociRaptor … for now.

 Storage Drive - Samsung Spinpoint F1 HD103UJ 1TB 7200 RPM

Again, I spent some time thinking about what to do for storage and I considered setting up a RAID array for data drives.  However, I did not want to put a huge investment into storage in case we see SSD drives that significantly outperform HDD (and are affordable) in the near future, in which case my VelociRaptor would be fine for holding my data.

I do image backups of all of my drives, so data loss is not a real concern for me.   7200 RPM drives are a dime a dozen, so brand does not really matter that much here.

 Video Card - EVGA 512-P3-N975-AR GeForce 9800GT 512 MB 

This is one area where you will probably want to do differently than what I did.  I am not really into playing games on my computer so I do not really care about the video card that much.  The previous generation video card will do just fine for Visual Studio so I am skimping on this item to save about $100.  In reality, I could probably go with a cheaper card that works just as good, but I couldn’t bring myself to buy the cheapest cards available. 

For anyone who plays games, I would recommend the MSI R4870-T2D1G Radeon HD 4870 1GB per the AnandTech guide.  For about a hundred dollars more than my card, you get a card with a 1GB of RAM that is based on the newer chipset from ATI.  With most 4870 X2 boards costing about double what that card will cost you, that card is definately a better buy.  One video card will do fine for me since it has dual DVI for my two Dell 2407WFP (24" widescreen) monitors.

CPU Cooler – Stock

Really?  No after market cooler you say?  I am not planning on overclocking the system so I don’t see a need for the after market cooler.  I am planning on doing some testing when I get the system setup, so I will let you know what the temperatures are like.  Using this same thought process, I am going to go with the stock case fans and see what happen as well.  Maybe I will be wrong and find out that the temperatures are unacceptable; and I’ll have to get a shiny aftermarket cooler and some new case fans.  But nobody ever really discusses what happens with the stock gear, so I think it will be an interesting experiment.

Totals

About $1300 for the system is not a bad price at all.  You could probably get a very similar system for around $1000 if you go with a Core 2 Quad system.

Benchmarks

Assuming I get all the components in the mail, get the system built, and everything works properly; I will post some benchmarks next week.  I will probably get some stats on how long it takes to build and run the unit tests for Castle, but I would also like to find a big project that uses MSBuild so that I can test some times using its parallel build support (NAnt does not support this).  If there are any suggestions on some other pointless benchmarks, let me know. =)

Last year, Hammett posted a blog entry on combining Javascript files into a single file using Monorail.  I was looking to enhance the code so that it would minify Javascript and CSS files on the fly, so I took his original code and enhanced it.  Alex Henderson was also looking for the same functionality, so he contributed to the patch as well.  You can download it here:  http://support.castleproject.org/projects/MR/issues/view/MR-ISSUE-457

The new component uses a .NET port of the Yahoo! UI Library Compressor that is available on Codeplex.  The component worked great, but they did not have a strongly signed version of the library release.  I made a request and they quickly turned out a release that same day, which was awesome response time.

You can now have all of your Javascript and CSS combined for you on the fly by using the component:

   1: #blockcomponent(BuildJS with "key=layout")

   2:     $jsBuilder.Add("Content/css/main.css")

   3:     $jsBuilder.Add("Content/css/someOther.css")

   4:     $jsBuilder.Add("Content/js/prototype.js")

   5:     $jsBuilder.Add("Content/js/someOther.js")

   6: #end

Although there are a number of good continuous integration servers out there, TeamCity is one of my favorites.  It has great support for .NET — supporting MSBuild, NAnt, Solution builds, NUnit, MSTest, and FxCop.  The price is right if you have less than 20 users and 20 build configurations – free.  I was also able to a plugin to integrate with our issue tracking systems in a couple of hours timespan (my Spring MVC experience definately helped though).

One of thing that you want to do with a release build is automatically increment your version numbers.  Most approaches to this problem recommend using MSBuild to autogenerate the AssemblyInfo.cs file.  Although that certainly works, I prefer not to have to do that for every project.  Instead, each of our projects references a single CommonAssemblyInfo.cs file.

When TeamCity builds a release build, it is configured to the IncrementVersion task:

   1: <Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

   2:     ...

   3:     <Target Name="UnrestrictedExecutionPolicy">

   4:         <Exec Command="powershell set-executionPolicy unrestricted&quot;" />

   5:     </Target>

   6:     <Target Name="IncrementVersion" DependsOnTargets="UnrestrictedExecutionPolicy">

   7:         <Exec Command="PowerShell .\incrementVersion.ps1 $(build_vcs_number_1)" />

   8:     </Target>

   9:     ...

  10: </Project>

The $(build_vcs_number_1) property tells TeamCity to the current revision number from the first source repository attached – in my case SVN — to the project (most projects will only have one repository attached).  Then, we execute a simple Powershell script to replace the file version number:

   1: $buildNumber = $args[0]

   2: (Get-Content ..\src\CommonAssemblyInfo.cs) | Foreach-Object {$_ -replace "(\d\.\d\.\d)\.\d*", "`$1.$buildNumber"} | Set-Content ..\src\CommonAssemblyInfo.cs

If you are more comfortable using a MSBuild task for regex replacement, there is a RegexReplace task that is part of the MSBuild Community Tasks.  For my build, we do not need any other custom tasks, so it was preferable to just use a simple PS script.

There are a lot of times in your application where you will load a lot of entities in a single session (to display a lot of data on a page, build a report, etc).  Since you are not planning on doing a mutable operation on an entity, there is a big saving that you can get from not flushing the session.  To get an idea on everything that happens when you do a flush, check out the AbstractionFlushingEventListener from the NHibernate source.

If you are using explicit SessionScope’s, not flushing the session is trivial:

   1: using (new SessionScope(FlushAction.Never))

   2: {

   3:     repository.DoSomeReadOperation();

   4: }

This will execute the operation without flushing the session at the end.

Another way to control this behavior is by using an interceptor.  For example, if you are using the Castle’s Automated Transaction Facility, you probably already have a way of determining whether or not your methods do mutable operations by their transaction attributes.  You can then use an interceptor to control flushing:

   1: public class UnitOfWorkInterceptor : IInterceptor

   2: {

   3:     private readonly ITransactionManager transactionManager;

   4:

   5:     /// <summary>

   6:     /// Initializes a new instance of the <see cref="UnitOfWorkInterceptor"/> class.

   7:     /// </summary>

   8:     /// <param name="transactionManager">The transaction manager.</param>

   9:     public UnitOfWorkInterceptor(ITransactionManager transactionManager)

  10:     {

  11:         this.transactionManager = transactionManager;

  12:     }

  13:

  14:     #region Implementation of IInterceptor

  15:

  16:     /// <summary>

  17:     /// Intercepts the specified invocation.

  18:     /// </summary>

  19:     /// <param name="invocation">The invocation.</param>

  20:     public void Intercept(IInvocation invocation)

  21:     {

  22:         ITransaction transaction = transactionManager.CurrentTransaction;

  23:         FlushAction flushAction = transaction == null ? FlushAction.Never : FlushAction.Config;

  24:

  25:         ActiveRecordUnitOfWork.Before(flushAction);

  26:         invocation.Proceed();

  27:         ActiveRecordUnitOfWork.After();

  28:     }

  29:

  30:     #endregion

  31: }

Note that in this example, interceptor ordering is important.  If you register the unit of work interceptor before the transaction interceptor in the ATM facility, then transactionManager.CurrentTransaction will always return null.  (The ActiveRecordUnitOfWork class in the above example is just a facade for controlling the session in a thread local as discussed in this post http://erichauser.net/2008/08/06/activerecord-session-scope-and-wcf-redux/).  If you do not want to tie your flushing to transactions, then you can always create your own metadata and read that metadata at runtime in the interceptor.  The ATM facility code is a great example of how to inspect metadata on initialization and use that metadata in your interceptor.

Hopefully, this little trick will some increased performance for your queries that return a lot of results.

Been pretty busy at work, so short post. 

I’ve added a facility in Castle contrib for validating method parameters using Castle Validator.  The goal is to eventually allow have this used for validate parameters in Monorail, but right now I’m using it to do input validation for web service parameters.  I wrote up a brief blurb on how to use it here: http://using.castleproject.org/display/Contrib/Castle.Facilities.MethodValidator.  Should be pretty self explanatory.

If anyone finds any issues, let me know.

I have contributed a couple of enhancements to Castle.Components.Validator that have been committed to the trunk.  Besides using attributes, validations can now be supplied in code using the [ValidateSelf] attribute:

Each of the above validations *could* be done by using an attribute and a custom validator, but expressing validations in code is much simpler for these types of one off validations. You can have as many methods decorated with [ValidateSelf] on an object you want as long as they have the above method signature (void return and one ErrorSummary parameter).  You can also specify the RunWhen and ExecutionOrder just like regular validators.

The second enhancement is the IValidationContributor interface. This allows you contribute to the validation of an object beyond the default validation. The interface is fairly simplistic:

You can extend AbstractValidationContributor so that you can perform initialization for a given type.  The SelfValidationContributor implements the logic for recognizing and executing the self validation feature above.  You can write custom contributors that can be injected into the DefaultValidatorRunner for things like retrieving validations from the container and invoking them on the object.

Enjoy!

I have added a couple of additions to Ayende’s NServiceBus Facility:

• Message handlers and sagas are automatically registered with the container for the defined assemblies
• Message handlers are proxied
• You can register IHandlerFilter instances with the kernel that allow for interception of messages as they are processed
• Using my replay strategy implementation, you can decorate a message handler as [Idempotent(Attemps = 5, Delay = 2000)].  That means that a message will be retried when an exception is thrown processing the message 5 times, delaying 2 seconds between each retry.  This could also be extended so that depending on the type of exception that is being thrown, a default retry strategy is applied.

• The bus is started automatically once all handler dependencies have been satisfied

There were a couple of interesting things that I found when implementing this.  One, NServiceBus retrieives handlers from the kernel using the concrete class instance (looking back Ayende noted this as “yuck”).  That means that we cannot proxy the interface, but instead have to create a class proxy for our message handlers.  That means that when we the message handlers, we have to verify that the message handler methods are virtual and give a nice error if they are not:

Very easy to do with Castle’s fluent interface for registration.  Sagas, on the other hand, have to be registered in the container by each of their interfaces because saga instances are retrieve by calling builder.Build<ISaga<SomeMessage>>().

Out of the box, the way to do message interception with NSB is to have message handlers chained in a specified order.  The IHandlerFilter provides another method for message interception.  If you have a new filter to add, all you have to do is register the IHandlerFilter instance with the container:

The filters follow the chain of responsibility pattern, so they can be short circuited — which is the same as calling bus.DoNotContinueDispatchingCurrentMessageToHandlers().

I have attached the updates to the post: nservicebus-fullduplex-xml-update.  Enjoy.

I got an email earlier this week from a member of our team saying that they had noticed odd behavior in our unit tests.  They said when the ran the tests from Visual Studio, the memory allocation increased by about 500MB.  When they ran the tests again, the memory increased another 500MB, and so forth.  The system component that developer was working on had the most tests out of all of our components, so he was noticing this before others did. 

First of all, it seemed odd that unit tests would cause Visual Studio’s memory to increase since the tests are run in another process.  I knew it would probably be impossible to profile Visual Studio itself, so I profiled out tests from the command line to see what if anything I could figure out.  A couple of tricks to profiling MSTest.exe:

• Launch mstest.exe with the /noisolation option so that the tests are run in process
• When you use the /noisolation option, you have to replace \Program Files\Microsoft Visual Studio 9.0\Common7\IDE\MSTest.exe.config with the App.config for your tests.  You also need to add the this:

    <runtime>
        <assemblyBinding xmlns=”urn:schemas-microsoft-com:asm.v1″>
            <probing privatePath=”PrivateAssemblies;PublicAssemblies”/>
        </assemblyBinding>
    </runtime>

The first thing I noticed after all of the tests ran was that 98% of the memory was being held by System.String.  When I looked to see what objects were referencing System.String, one thing blaringly stood out:

All of the standard out was being held in a single member variable from the whole test run.  Since I was not sure if Visual Studio was doing the same thing, I turned off logging in our application and ran the tests again.  Surprise, that worked.  Visual Studio no longer was increasing in memory usage.

Yes, I realize 800 MB of log output is a lot but that is not really the issue – I’m not sure why VS keeps that output in memory.  Once it has the test results, I believe it writes it out to a file so why keep it around?