Update-Package performance

Topics: General
May 10, 2012 at 7:44 AM

Hi,

This week i have been migrating our solutions at work to use nuget. So far so good. The only issue i have at the moment is the performance when doing an Update-package from within visual studio or via the NuGet.exe application (when pointing to .sln file).

If i have the projects below:

Project A depends on aaaa,bbbb,cccc,dddd
Project B depends on bbbb,dddd,eeee
Project C depends on aaaa,bbbb,cccc,dddd,eeee
Then when i do an update the following web requests will be made:

http://myserver/nuget/Packages()?$filter=tolower(Id)%20eq%20'aaaa'&$orderby=Id
http://myserver/nuget/Packages()?$filter=tolower(Id)%20eq%20'bbbb'&$orderby=Id
http://myserver/nuget/Packages()?$filter=tolower(Id)%20eq%20'cccc'&$orderby=Id
http://myserver/nuget/Packages()?$filter=tolower(Id)%20eq%20'dddd'&$orderby=Id

http://myserver/nuget/Packages()?$filter=tolower(Id)%20eq%20'bbbb'&$orderby=Id
http://myserver/nuget/Packages()?$filter=tolower(Id)%20eq%20'dddd'&$orderby=Id
http://myserver/nuget/Packages()?$filter=tolower(Id)%20eq%20'eeee'&$orderby=Id

http://myserver/nuget/Packages()?$filter=tolower(Id)%20eq%20'aaaa'&$orderby=Id
http://myserver/nuget/Packages()?$filter=tolower(Id)%20eq%20'bbbb'&$orderby=Id
http://myserver/nuget/Packages()?$filter=tolower(Id)%20eq%20'cccc'&$orderby=Id
http://myserver/nuget/Packages()?$filter=tolower(Id)%20eq%20'dddd'&$orderby=Id
http://myserver/nuget/Packages()?$filter=tolower(Id)%20eq%20'eeee'&$orderby=Id

This really slow things down. Is there a reason why the result of the package lookup is not cached? I perhaps understand the reason for this not being cached in VS but via the command line it would be nice if the result was cached to speed up the update.

Have others noticed this performance issue?

Thanks in advance,

Dom

May 10, 2012 at 2:18 PM

We found similar and worse, as some of our packages have many hundreds of versions resulting in many return trips for a single ID.  It was particularly frustrating when we wanted to only refer to the very latest package version, which is a single call with a single package response.  With a very large set of packages and versions, and a lot of developers, this was bringing our internal NuGet Gallery servers to a standstill.

We have put together a really simple set of caches and resolvers that reduces this considerably, however in their current form it may be a bit hard to incorporate them directly into NuGet as you would need to explicitly modify each console command class you wanted to use them in.  We use them in a few extension commands which we have not yet released.  

The code for the caches can be found here: http://github.com/BenPhegan/NuGet.Extras, however if you wanted to inject into the update command you would need to either create an extension method on the ProjectManager or modify the ProjectManager to have another implementation of UpdatePackageReference that used both of the following:

http://github.com/BenPhegan/NuGet.Extras/blob/master/NuGet.Extras/Packages/PackageResolutionManager.cs

http://github.com/BenPhegan/NuGet.Extras/blob/master/NuGet.Extras/Caches/MemoryBasedPackageCache.cs

Not sure if any of that helps or not, but at least you know its not just you!

Developer
May 10, 2012 at 3:05 PM

We've fixed this issue in 2.0. We're moving all the filtering to the server to make the result much smaller and where we wouldn't have to do package lookups from the client.

May 10, 2012 at 10:39 PM

I've just released changes to NuGet.Server that should speed up these queries despite the redundancy.  You can get a zip file of the binaries here:

https://github.com/themotleyfool/NuGet/downloads

May 22, 2012 at 4:00 AM

Thanks for your replies.  I managed to hack something into the nuget.exe to do some nasty caching but i am happy to hear this is resolved in v2.0. Any idea when 2.0 will be available?  I have just migrated our code base to nuget so keen to make it as slick as possible for the devs i work with.