Package update performance improvements

Topics: Ecosystem
Oct 26, 2012 at 8:17 AM
Edited Oct 26, 2012 at 9:41 AM

We've started to roll out NuGet in our company rather successfully. The only trouble we have is the solution wide update experience which is absolutely catastrophic. When running a solution wide update through powershell using:

"Update-Package -IncludePrerelease"

We had to wait about 35 minutes for the process to complete when all of the base packages were updated. Even for smaller updates we are talking about 12 minutes time. This is just not acceptable.

Since I already dug a bit through the NuGet source code, I started to look at the places where some optimizations could be made. I've found very interesting spots, and achieved a significant performance boost: 10x ! Got down from 12 minutes to 1 minute 20 seconds in the smallest case I used for the tests.

Here are the big lines of what I did:

  • Add package dependency caching
  • Add command scope package resolution cache
  • Found out that AddBindRedirect costs 20% of the time !! I havent found a good improvement yet
  • Adding references through the IVsProjectSystem is very very slow ! Changing this to use the MsBuild project interface brings a huge improvement
  • EndBatchEdit costs a lot -> moving the EndBatchEdit on a much higher scope (command scope) cut's down the update time by 2 !!
  • The VSProject property UniqueName access is horribly slow !!! There must be a better way to manage that

I need to clean up the modifications I made a bit, before I can show some code to discuss this further.

Oct 26, 2012 at 5:10 PM

Hmm, 35 minutes is unusually slow. Can I ask how big your solution is and how many packages in total are installed?

I will comment further on your changes when I see the actual code.

Oct 26, 2012 at 5:35 PM

Adding references through the IVsProjectSystem is very very slow ! Changing this to use the MsBuild project interface brings a huge improvement

Wouldn't that cause VS to prompt for a project reload?

Oct 26, 2012 at 7:57 PM

@dotnetjunky: The biggest solution we use has about 50 projects which link to about 85 packages. The packages being referenced have many interdependencies:

For example we have for many packages situations where A depends on B, C, D, E and D depends on B, C, E. 

@pranvkm: The spot I changed is in VsProjectSystem.cs, on one line the reference is added through the IVsProjectSystem, and 20 lines below the Hint path is corrected through the MsBuild project class. In my tests it didn't pose any problem.

Oct 26, 2012 at 8:00 PM

Thanks for the info.

May I ask what types of projects they are? Have you tried your changes with different types of the projects and different types of languages, including F#? As pranavkm said, we experienced from the past that calling MsBuild.Save() will show a prompt dialog. If we didn't call Save(), some changes may be lost if you unload the project and reload it.

Oct 30, 2012 at 9:17 PM

I got things cleaned up a bit, so you can look into the changes I made and tell me how much nonsense I produced :)

The code can be found here: 

I haven't addressed the AddBindingRedirect yet. I still have find an approach on this.

@dotnetjunky: I don't have any F# projects at hand to make any pertinent tests at all (this is only noticeable on big projects with lots of dependencies). The only project types I tested were C# projects. There isn't anything C# specific in what i modified, so I hope it will work out with other project types. Please also note that all the tests I made so far were in VS2012.

I'll get to testing this in VS2010 shortly, I'll let you know how it turns out.

Nov 3, 2012 at 9:41 PM
Edited Nov 4, 2012 at 7:24 AM

I just finished some testing using VS2010 and can confirm the major performance improvements achieved with VS2012. Using the MSBuildProject system worked fine also here. I had no dialog boxes which asked if projects should be reloaded.