50
Vote

Nuget.exe install : copy "Content" files if missing

description

As explained here http://blog.davidebbo.com/2011/05/thoughts-on-installing-and-updating.html," Nuget.exe install" is not intended to install a package outside Visual Studio, but rather to restore a package in order to avoid to commit this package in source code control.

This means "nuget install" doesn't make modifications to the csproj file, does not run perform config transformations, and does not copy the files in the "Content" directory to the target directory.

I've a package with large files in the content directory that I'd like not to commit, and it would be nice if "nuget install" could additionally copy these files if they are missing in the target directory.

Actually, maybe a new command like "nuget restore" would be more appropriate than "nuget install".

Thanks

comments

davidebbo wrote Jun 27, 2011 at 6:30 PM

Yes, I could see this making sense. The restore would simply copy over the content files if they're missing.

rogerkratz wrote Jun 4, 2012 at 1:38 PM

+1
This would enable us to not commit a lot of js files in web projects (good if a DVCS is used).

m0thr4 wrote Sep 25, 2012 at 2:20 PM

The lack of this behaviour is currently causing my C# project to fail when run from TeamCity. It depends System.Data.SQLite.1.0.82.0 which has a content directory containing additional libraries that need to be copied to the project folder as follows:

x86\SQLite.Interop.dll
x64\SQLite.Interop.dll

These libraries are then copied to the build output directory. However, when I run this run from the command-line, or under TeamCity Nuget build runner, the content directory is not unpacked, and the build fails.

eazran wrote Mar 10, 2013 at 3:09 PM

This is important for TeamCity builds.

It seems that when NuGet is executed as part of a TeamCity build, the files in "Content" folders are not copied to folder root.

see post here: http://devnet.jetbrains.com/thread/438726

Eyal

dotnetjunky wrote Mar 11, 2013 at 6:18 PM

Can you explain why this is important for TeamCity builds? Why don't you include the content files in your projects before pushing them to TeamCity?

m0thr4 wrote Mar 11, 2013 at 7:23 PM

I think there is some confusion here -

TeamCity's "Nuget Installer" build step only runs command-line Nuget, and therefore is quite limited in what it can do. What NuGet command-line (and therefore TeamCity) certainly cannot do is run the post install script bundled with System.Data.SQLite, which copies the two SQLite.Interop.dll files into the solution root, under folders called x86 and x64.

whut wrote Jun 20, 2013 at 6:59 AM

Such "nuget restore" should copy content files, but should not run install.ps1 or update csproj files.

For the corner cases, there are some packages, that :
  • put in content\ files that aren't really needed (like Elmah with App_Readme\Elmah.txt), this would be handled by not updating csproj
  • open readme.txt in Visual Studio tab (like Castle.Windsor), this will be handled by not running install.ps1

rfrizzel wrote Jun 25, 2013 at 5:21 PM

I will also "+1" this. This is something we are struggling with right now.

We are mainly trying to find a good way to distribute common master pages across many different solutions/projects/websites/etc and we do not want to source many different copies of a set of "standard" master pages for our organization.

I know there are other ways to solve this besides NuGet, but with this issue addressed with Content automatically being checked on package restore, NuGet makes the most sense.

JeffHandley wrote Aug 14, 2013 at 5:45 PM

This feature could also repair broken or missing assembly references and msbuild imports.

since we now have a nuget.exe restore for pure package restore, that opens nuget.exe install up for actually installing/fixing packages.

hrdcoder wrote Jun 5, 2014 at 7:02 AM

My scenario is thus:

I have a fake server .exe, that (using WebApi) pretends to be an external restful service. I use it in multiple regression test projects across different solutions. I packaged it up as a Nuget package, so it can be pulled into the reg test project prior to running. Using 'content' means it is easy to locate the binary to launch it (it could be launched from the packages folder directly, but that is somewhat messy).

On TeamCity, since nuget.exe is not placing content into the target reg test project, the tests fail.

If there is a better solution for this, I'm happy to do it another way.

JeffHandley wrote Jun 5, 2014 at 7:22 PM

Sounds like you should be authoring that workflow to grab files out of the package's /tools/ folder instead of the /content/ folder. And do the files actually need to be copied into the project directory at all?

hrdcoder wrote Jun 5, 2014 at 11:41 PM

Thanks @JeffHandley - It it not necessary for it to be in the project directory, and what you say is correct. It's more that it's cumbersome to create some discovery mechanism, rather than just having a well-known path (in this case a relative path) to launch the tool.

mickdelaney wrote Jun 25, 2014 at 10:32 AM

+1 for Content Restore like workflows.

undecided wrote Jul 31, 2014 at 7:03 PM

+1. I packaged some 3rd party libraries into a nuget package. 1st time the package is installed, all the content files are there. When a new developer does the nuget restore, the content files are omitted.

JeffHandley wrote Jul 31, 2014 at 9:03 PM

@undecided - can you elaborate on why the content files weren't there? Did you explicitly exclude them from source control in your project?

undecided wrote Aug 1, 2014 at 12:49 AM

Yes, I do not add them to source control. They are not source files. They are 2 c/c+ unmanaged dlls and a flat file that those dlls use at run-time. They are provided by a 3rd party.

JeffHandley wrote Aug 1, 2014 at 2:38 AM

@undecided - can you clarify something though...

Are these files coming into your project through the package's "/content/" folder where NuGet is copying them into your project, and then you explicitly exclude them from source control? Or are these files in the /packages folder and just referenced somehow?

Those don't sound like the kinds of files a package would put into the /content folder. Is this a public package you can point me to?

undecided wrote Aug 1, 2014 at 3:22 PM

Jeff, here is the nuspec (renamed everything as this is proprietary)
<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2011/08/nuspec.xsd">
<metadata>
    <id>Some Package</id>
    <version>1.0.0.0</version>
    <title>Some Package</title>
    <authors>Some Author</authors>
    <owners>Some Owner</owners>
    <requireLicenseAcceptance>false</requireLicenseAcceptance>
    <description>Some Package</description>
    <copyright>Some Author</copyright>
</metadata>
<files>
    <file src="content\aDefFile.def" target="content\aDefFile.def" />
    <file src="content\Example.cfg" target="content\Example.cfg" />
    <file src="content\AFlatFileNeededByBelow" target="content\AFlatFileNeededByBelow" />
    <file src="content\SomeCPPUnmanaged.dll" target="content\SomeCPPUnmanaged.dll" />
    <file src="content\SomeCPPUnmanaged2.dll" target="content\SomeCPPUnmanaged2.dll" />
    <file src="lib\ManagedCSharp.dll" target="lib\ManagedCSharp.dll" />
</files>
</package>


The owners of this do not publish this library through nuget. I packaged it up for our internal nuget feed so that all developers can use this library as is with no need to check it in repeatedly with each project.
The way we accomplished this in the past was through the use of externals in subversion.

cameron314 wrote Aug 5, 2014 at 10:23 PM

Like @m0thr5, we're using System.Data.SQLite.Core, which puts the native DLLs in the content part of the package, which is copied automatically to the project directory when the package is installed (note that there is no Powershell script that does this, it seems to be nuget.exe doing this on its own. The script merely adds the file into the project.).

It doesn't make sense to commit these files, since they're binaries that are never edited, and change with every package update. What we've done as a workaround is to change the installation script (yes, we forked the code and privately hosted our own NuGet) to reference (as a link) the file from the unpacked package itself (under packages/content/), instead of from the (copy) of its content for that project.

undecided wrote Sep 5, 2014 at 3:52 PM

Is any progress being made on this?

Thx

JeffHandley wrote Sep 6, 2014 at 1:52 AM

It sounds like what folks want is this:
  • A new nuget.exe restore switch for copying content files into the project folder
  • Integration into Automatic Package Restore in Visual Studio to use the switch
  • It would copy in the types of files that can be treated statically and unmodified (details on this below)
This approach would cover the 80% cases that folks seem interested in:
  • JavaScript
  • CSS
  • Images
But there are some files that we'd have to omit from this copy process:
The command would skip those files and emit warnings about them being skipped. But this would allow package consumers to go delete files that came from packages from source control and allow package restore to bring them back, with the limitation of only being able to delete files that came in statically from the package.

The project file would be left alone. There would still be project file references to the files that are omitted from source control, and solution explorer would show the files missing until package restore was fired off. Then the files would show up on disk again and all would be swell.

Does this capture the request thoroughly?

undecided wrote Sep 6, 2014 at 3:59 AM

Besides
JavaScript
CSS
Images

I would also add unmanaged dlls, otherwise the feature sounds good.

Thx

p.s. We are now handling our case as a pre-build msbuild task. We copy content, scripts, etc folders on pre-build. We exclude the same transform files you are suggesting to exclude.

davidebbo wrote Sep 6, 2014 at 4:47 AM

That sounds like a good feature to me. Would be great to get all those big jquery files out of source control!

panchilo wrote Sep 7, 2014 at 2:51 PM

I've created an MSBuild task that takes care of restoring your Nuget package content files.

Here is the Nuget Package site: https://www.nuget.org/packages/MSBuild.NugetContentRestore/
And the GitHub page: https://github.com/panchilo/MSBuild.NugetContentRestore

I've developed it following the .nuspec specification, ignoring transformation files (.config.transform, .install.xdt), and only copying some specific folders like Scripts, Fonts, Images, Content. This is configurable through task parameters.

@JeffHandley - Even though this is doing the work for me, I still believe it'd be perfect to have another "switch" in nuget.exe so it also copies the content files.

dromero wrote Sep 9, 2014 at 5:31 AM

I've created a nuget solution level package which does this, it's also managing the git ignore file, to be able to exclude content files from the git repository. In VisualStudio the copy process happens automatically and there is a powershell script to run on build servers.

http://www.nuget.org/packages/Baseclass.Contrib.Nuget.GitIgnoreContent/

https://github.com/baseclass/Contrib.Nuget

Step by step usage:

http://www.baseclass.ch/blog/Lists/Beitraege/Post.aspx?ID=9&mobile=0

bartelink wrote Oct 24, 2014 at 12:04 PM

@dromero @panchilo You guys might find http://fsprojects.github.io/Paket/ interesting and/or useful

@JeffHandley Would you mind commenting on some of the following assertions/ideas around this please?

1 NuGet does/will continue to keep a redundant copy of the /content under the package's dir in /packages (despite it actually 'living' in each project)
1a NuGet does/will work if one doesn't commit the packages//content/*/* (ideally to minimise churn in DVCS, the files would not be expanded in the first place)
2 It considered impossible/out of scope for now/unlikely to ever happen for content files to be 'links', i.e. use the same scheme as for DLLs where a single copy lives in the package folder and each project links to it there
2a if a scheme like this was to be worked out - would Package Restore magically step in if the files were not committed

(I understand #2 is obviated on newer platforms that will consume NuGets at compiler/build system level)

deep_light77 wrote Dec 21, 2014 at 3:12 AM

Here's a MSBuild task that uses Nuget ProjectManager to re-install your packages before build, but after Automatic Package Restore has restored them to your packages folder. It can also add the Content reference to your project file:

github page: https://github.com/deeplight77/NugetPackageRestore

This work is based on panchilo and dromero's, but modified so it applies to all types of packages, not just content packages, it even works with .pp and .transform files, and it uses the local packages folder as a source repository, to enable it to work in conjunction with Nuget's Automatic Package Restore.

More information on the github page.