Package Updates without package.config modifications and TFS/NuGet.exe

Aug 7, 2011 at 1:57 PM

Very similar to that expressed by LeeCampbell in this thread:  

In that, he said that the workflow he wanted was:

  1. when an internal library is modified the build server will version, build, test, package and publish it into our internal NuGet Repository
  2. the build server will then activate dependent builds (4 modules, 5 services, fakes, test tools and monitoring tools)
  3. builds for projects that consume a NuGet dependency will check the internal NuGet repository for newer version and get the latest
  4. they will update the csproj files and packages.config to the latest versions and check in the changes.

I would amend 4., as typically we would not want our CI build modifying files under the same root that it is watching.  Instead, a concept of a "floating" or "latest" version of the dependency, and this would probably need to be explicit (both when selecting in the gui and in the packages.config).  This is so we can have a reference to "Internal.Library1" and during development always update to the "Latest" from the feed.  This would then update the package from the feed with the latest available and install in \packages, note it in the output (for capture as part of the CI build output - so we can easily recreate at a later point even without binary output of build) and _not_ modify any existing packages.config files.  As we get close to locking down a release, we can change this to an explicit versioning of this dependency - updates would no longer modify the packages.config as the version would be explicit or the packages stable.  This would require the use of the ExcludeVersions flag (no versioned directories in Packages).  Currently the ExcludeVersions functionality does not appear to work for updates, or be available in the gui or powershell, although command line installs appear to work.

This would also alleviate an issue where multiple people are doing an "update" at a similar time, with a dependencies build occuring between their updates.  This results in two different versions being persisted into the packages.config file, and subsequent merge issues.

Also, using TFS 2010 by default you get  read-only files on checkout.  This will cause updates to fail and make it difficult to enable automated NuGet changes that can then be committed as you would need to externally script a "checkout", check for changes, then a "commit".  The base class that MSBuildProjectSystem inherits from is PhysicalFileSystem, would need to be able to use TfsFileSystem here instead somehow.

Aug 7, 2011 at 11:48 PM
Edited Aug 7, 2011 at 11:49 PM

Agreed on all counts.

Currently (with v1.4) the only real way to bake NuGet.exe operations such as 'update' in to a set of MSBuild CI scripts is to make use of solution / project build events, i.e. update your packages pre-build.  This leads to a suboptimal developer experience; build events that modify the csproj file that you're currently compiling represents a workflow that is in need of a rethink.  Plus, as you mention, when coupled with TFS the process becomes even more cumbersome.  Basically having to change anything outside of the bin folder when compiling an assembly is IMO best avoided...

I totally understand this is a workaround resulting from an increase in scope & requirements, which is a good thing as the fundamental principles on which NuGet is based are excellent.  For NuGet to realise its full potential in a corporate CI build process the operations need to be supported directly from the MSBuild system, preferably removing the Visual Studio dependency the current version demands.  I'm a firm believer of not installing VS on a build server... (another topic entirely :-)).

Modifying project MSBuild files during compilation isn't acceptable, as you're effectively compiling against an indeterminate code-base (minimal but true nonetheless), which can be partially solved by -ExcludeVersion (which isn't fully functional as you say - need to test this out more before reporting bugs though).  The -Safe option on "update" already allows us to manage NuGet updates through a disciplined versioning scheme, but this still requires a fixed version (via packages.config) to be checked back in to your VCS.  So a variation / hybrid of -ExcludeVersion and -Safe could go some way to provide a solution.

A potential idea (and this IS just an idea that hasn't been fully thought through...):  The build version number could be tagged against the referenced versions of NuGet packages on the feed server instead of being stored in the project/s.  The NuGet reference version number in the project would be the 'maven' style, i.e. (1.0,1.6], but the "Build" version would be the link between VCS and the NuGet package feed version. May not work in practise as it would create another dependency on the NuGet feed, also retrospectively building shrink-wrapped software could be interesting - definitely needs more thought. In terms of the developer's workflow you'd simply pick up the latest safe version each time.

Jul 12, 2012 at 5:39 PM

Well I'm not alone ) I have similar requirements as BenPhegan and LeeCampbell. In general it's "using NuGet as an engine for managing and resolving dependencies between parts of a system". But not just a package manager of 3rd-party libraries.

I asked here and here about updating packages and versioning. But after reading Ben's and Lee's posts I've finally realized my requirements (thanks to them) )).

It's passed a year since original posts. Today nuget.exe has -ExcludeVersion option. But it doens't solve all problems.

Let me describe my scenario. I have a package which I publish into our internal nuget-repo. This package is used by developers of different projects. Here they act as "users". They should be able to update the package. That's because I have to increment package's version. If I don't then update won't work. Developers'll have to delete local package-folder under "packages" folder and do restore. Restore isn't suitable for us as my package has content (javascript files). And that content isn't updated during package restore. So I have to increment package version.

But besides users there're internal projects which uses my package. They are kind of demo/samples projects. They are built and deployed by CI Build Server automatically (by builds dependant on main project's build). And here I have problems. As different builds of my package have different versions "package restore" won't work. Because nuget will be trying to download exactly the same version of my package which specified in demo-project's packages.config file. And this will the same version every time. I can't it update manually each time obviously. So it makes no sence to build demo-project every time.

This' a deadlock. I can't understand what to do really. I'd happy to hear any ideas, thanks.

Jul 24, 2013 at 12:47 PM
Has anyone found a resolution to this issue? I'm using the latest version of NuGet (2.6.1) and having this exact same problem. I have a set of shared libraries that I pack and publish into NuGet packages in an internal package source. I want my dependent builds to always pull in the latest build/version of these libraries. But it always pulls in the exact version specified in the packages.config file. I tried using a constant version number (i.e. all builds would be versioned the same) hoping it would pick up the newest build for that version, but it thinks all packages are already installed and up-to-date for that version. Any help would be greatly appreciated.
Jan 19, 2015 at 7:13 PM
Edited Jan 19, 2015 at 7:14 PM
Our use case is an msbuild system on TFS for a desktop application that needs to first restore existing packages and then update certain internal dependencies automatically. We have several solutions and projects sharing the same directory tree (some projects are shared or have different build configurations for different solutions).

Currently we can restore packages, but can't udpate individual packages solution-wide (since our projects depend on each other, they all have to have the same version of the same dependencies within a solution).
Our working solution is:
  1. call nuget restore X.sln
  2. search for all csproj files in the sln file
  3. for each Y.csproj file, re-write its packages.config file to set allowedVersions=[version] for the packages we're not updating, so that we can update certain packages.
  4. for each Y.csproj file, call nuget.exe update Y.csproj
It would be quite helpful if steps 2-4 could be replaced with "nuget,exe update PackageName X.sln".