NuGet Package Restore doesn't play well with multi-sln projects

Topics: General
Feb 1, 2013 at 1:07 PM
Edited Feb 1, 2013 at 1:19 PM
When using projects configured for NuGet package restore, I discovered issues that break when I try to reuse the same projects in another solution.

Here's the scenario:

ProjA\ProjA.sln -> references 4-5 projects underneath ProjA (ProjA\ProjA.Dal\dal.csproj,ProjA\ProjA.Foo\foo.csproj, etc )

That's a standalone solution, and works fine by itself when configured for package restore. It puts the packages dir in ProjA\packages.

Suppose I have the same thing with ProjB. I have a top level ProjB folder with a ProjB.sln that has its own projects.

So we have

\ProjA\ProjA.sln
\ProjA\ProjA.Dal...
\ProjA\ProjA.Foo...
\ProjA\.nuget\
\ProjA\packages\
\ProjB\ProjB.sln
\ProjB\.nuget\
\ProjB\packages\
\ProjB\Bar...
\ProjB\Frob...

By themselves, so far so good. The problem arises when I want to create a "master" solution file for all of the projects.

\Master.sln

The idea is that teams can work on their components in isolation, but it's still easy to do refactoring or testing cross-component by loading up the master.sln.

When I load up the projects two bad things happen.

1) the nuget.targets are missing because they are referenced via $(SolutionDir) and that dir is different

2) If I enable NuGet package restore for the master.sln, it will put the .nuget dir, but building doesn't correctly restore the packages. It puts them into \packages instead of \ProjA\packages and \ProjB\packages and the projects' hint paths don't find it.

I don't want to change the hint paths because I still want to have isolated solutions.

Couple of ideas here. Can the nuget.targets file be included relative to $(ProjectDir) instead of $(SolutionDir)? That would prevent the first issue.

Second, the \packages dir would need to be relative to the .nuget directory. Package restore would then put the files in to the correct locations (within \ProjA and \ProjB) so that the references resolve correctly.

Alternatively, the <NugetPackageReference> stuff could help with this redirection problem as the resolution wouldn't depend on the HintPath.
Feb 1, 2013 at 8:31 PM
It does work well, so long as you don't actuall use the inbuilt implementation.
The current process assumes one solution and every project is nested, aka a sandboxed project, forget the real world.

To make it work you need to manually do the following, but thankfully you only have to do it once.
  • FIRST, You need a nuget.config in the common root folder with something like the following content:
<configuration>
    <config>
        <add key="repositoryPath" value=".\Packages" />
    </config>
    <!-- this should work but doesn't
    <solution>
        <add key="disableSourceControlIntegration" value="true" />
    </solution> -->
</configuration>
This hierarchal config will make one common folder for all nested projects/solutions.
After this is in place, either manually move your packages here and update project references, or preferably uninstall and reinstall your packages.
This will put in all references relative to the common Packages folder.
  • SECOND, Now that you have a common packages location, you should have a common restore implementation.
    I have a 'NuGet' folder next to the common shared packages folder.
    In it I have NuGet.exe and the following simplified NuGet.Restore.targets:
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
    <PropertyGroup>
        <!-- Enable the restore command to run before builds -->
        <RestorePackages Condition=" '$(RestorePackages)' == '' ">true</RestorePackages>
        <PackagesConfig>$([System.IO.Path]::Combine($(ProjectDir), "packages.config"))</PackagesConfig>
    </PropertyGroup>

    <PropertyGroup>
        <!-- Commands -->
        <RestoreCommand>"$(MSBuildThisFileDirectory)\nuget.exe" install "$(PackagesConfig)"</RestoreCommand>

        <!-- We need to ensure packages are restored prior to assembly resolve -->
        <ResolveReferencesDependsOn Condition="$(RestorePackages) == 'true'">
            RestorePackages;
            $(ResolveReferencesDependsOn);
        </ResolveReferencesDependsOn>
    </PropertyGroup>

    <Target Name="RestorePackages">
        <Exec Command="$(RestoreCommand)" LogStandardErrorAsError="true" Condition="Exists('$(PackagesConfig)')" />
    </Target>
</Project>
All that is left is to add/change the <Import /> in each project file that needs restore to point to your new shared restore target.
Feb 1, 2013 at 10:02 PM
Unfortunately, I don't think that workaround is going to work as I left out a few details in the name of simplicity.

There's no common root location for the master solution; it's actually parallel to the other solutions.

Here's the real structure:

$/WebSite/Main/website.sln (and other projects)
$/WebSite/Dev/...

$/Service1/Main/Service1.sln (and other projects)
$/Service1/Dev/...

$/Service2/Main/Service2.sln (and other projects)
...

$/Solutions/Main/Master.sln

Each solution has its own versioning, etc.

In TFS, we use a workspace to map down the branches to be like so:

$/WebSite/Main -> C:\dev\WebSite
$/Service1/Main -> C:\dev\Service1
$/Service2/Main -> c:\dev\Service2
$/Solutions/Main -> c:\dev\Solutions

This provides the flexibility such that a dev/tester can remap one of the services to be from a different branch, etc. The master.sln file is setup to depend on those local directory structures.

There's no real way to put anything hierarchically in c:\dev. Not sure about other version control products, but TFS gets confused by overlaying different mappings like that. That being the case, I don't think your proposed workaround would work for this scenario.

Thanks
Feb 1, 2013 at 10:19 PM
Sure there is a common root folder, its your C;\dev folder. The one nuget.config file in this folder should never really change. As you are manually remapping your dev branches into a common working space, why not add a $/NuGet/Main and map it down to c:\dev\nuget. Also add a very simple batch file in this folder to copy your nuget config up to c:\dev. which only has to be run once.
Feb 1, 2013 at 10:28 PM
Hmm....I suppose something like that could work; I was thinking that there's no common root that's in source control or directly mapped, but a one-time batch file could work to setup the structure.

I'll give it a further look on Monday.

Thanks,
Oren
Feb 1, 2013 at 10:32 PM
Also, do you have a Company.DotSettings file? If so, how do you share it? We put ours in this same root folder and then add it as an item of the solution-team-shared layer for all solutions.
Feb 5, 2013 at 3:32 PM
Hi Eddie,

I was looking into this yesterday/today but still am running into the HintPath issue.

The real issue is that most of the devs do NOT want to map more than their project. Nor do they want to have to run the bat file to put the common nuget.config file somewhere.

I'm okay with that if it's only for people who want the master.sln, but the problem is that the hint paths won't match. If I force packages to be in c:\dev\packages, then that won't work for people who just want to load their single project solution.

Another NuGet proposal around creating a <NugetReference> element in MSBuild could work as that'd enable NuGet to dynamically resolve references without a hint path, but that's probably 2.3 or 2.4 before that happens.

What I really want is logic to specify multiple packages dirs., one per sln.

If the nuget.targets were imported relative to the project file instead of sln file, then each of the project sln's would have their own .nuget dir (which they do). What I want is a way to say that the \packages dir should be relative to the .nuget dir...i.e., alongside it. Then it'd work in all scenarios. Package restore should only consider a \packages dir that's relative to the included nuget.targets file.

I realize that it could mean the same package is downloaded multiple times, but I'm okay with that.

Oren
Feb 5, 2013 at 4:57 PM
Well if thats the case, you could always manually (ideally automatically) set up your hint paths as such:
Instaed of
      <HintPath>..\packages\Newtonsoft.Json.4.5.11\lib\net40\Newtonsoft.Json.dll</HintPath>
you would have
      <HintPath>$(PackagesDir)\Newtonsoft.Json.4.5.11\lib\net40\Newtonsoft.Json.dll</HintPath>
where $(PackagesDir) is defined in your nuget.targets.
Sep 18, 2013 at 10:51 PM
Hi Eddie,

I had a question on disableSourceControlIntegration in the scenario when we have the nuget.config at the TFS root folder. This is my current folder structure

NuGet.config location is : $/Nuget.config
Nuget folder location is : $/Nuget

The NuGet.exe and Nuget.targets are under $/Nuget.

I modified the project to point to the Nuget.exe in $/Nuget.

What I am seeing is that Nuget.exe picks the updated repository path in Nuget.config but never reads the key disableSourceControlIntegration so always adds the packages folder as a pending checkin. Is this because Nuget.config is outside the solution folder ?

Regards,
Sai
Apr 21, 2015 at 4:16 AM
Edited Apr 21, 2015 at 4:17 AM
NuGet only seems to support the scenario where all .sln files are in the same folder. If you do that, you will not encounter this problem. It’s worth it just to avoid the hassle of manually editing in <HintPath/>s every time you install a nuget package.