37

Closed

Package Creation from Project File doesn't handle Project References

description

Related to http://nuget.codeplex.com/workitem/675
Duped by http://nuget.codeplex.com/workitem/1058

I have a solution with two Class Library projects, each with NuSpec files in there: Project B -> Project A
I want to turn them both into packages.

When I run nuget pack ProjectB.csproj, ideally it would recognize that ProjectB has a project reference to ProjectA which has its own nuspec file and would automatically add a <dependency id="ProjectB" version="{ProjectBVersion}" />.

Right now, I have to manually put the dependency node in ProjectB and remember to keep it in sync with the ProjectA assembly version.

file attachments

Closed Mar 11, 2013 at 6:07 AM by deepakverma
following issues were opened while verifying this from the nightly build.
portable library reference not added correctly with nuget.exe –includereferencedprojects http://nuget.codeplex.com/workitem/3111

winmd is not included with nuget.exe -includereferencedprojects switch http://nuget.codeplex.com/workitem/3112

includereferencedprojects need to parse id token before adding the nuspec dependency, throws an exception http://nuget.codeplex.com/workitem/3121

includereferencedprojects doesn't use the metadata from the nuspec dependency http://nuget.codeplex.com/workitem/3124

includereferencedprojects -symbols doesn't add sources for referenced project http://nuget.codeplex.com/workitem/3125

includereferencedprojects need to check if a project already exists in the projectCollection, throws an exception http://nuget.codeplex.com/workitem/3116

comments

Haacked wrote Apr 19, 2011 at 6:45 PM

Need to figure out the design. Do we create all packages or just the one?

rekna wrote May 18, 2011 at 8:00 AM

In the case of a project reference, I would prefer having one package...
You can not tell for sure, if project A is meant to be used as a standalone package ( as a matter of fact, I have a solution, where that is definitely not the case, they have to be released as one package, because A will not work as standalone package)...
So single package scenario should certainly be implemented, multiple packages can be solved by changing project reference to package reference.

dmal wrote Jul 25, 2011 at 2:41 PM

This would be killer as i have a project which i intend to turn into a package, which has 6-7 dependencies (our own projects) so it wouldn't matter to which way it went :)

2 thumbs up for the good work
//Dennis

dandreica wrote Nov 15, 2011 at 12:29 PM

I'm also struggling with a similar scenario, and I have a suggestion on how NuGet could improve support for it. I use "nuget pack ProjectB.nuspec" (note, .nuspec file used , not the .csproj file). In the nuspec file, I list all project dependencies, like so (as all projects and packages in this solution are built at the same time, my build script can pass the Version property so the correct dependencies are output):

<dependencies>
  <dependency id="ProjectA" version="$version$" />
  <dependency id="ProjectC" version="$version$" />
</dependencies>

However, because there's a "packages.config" file in ProjectB, NuGet ignores all the dependencies I've written in the nuspec file, and uses the content of "packages.config" instead. If this were changed to merge/add the contents of "packages.config" to the manually-declared dependencies, then packages with project references could easily be built.

I would therefore like to suggest that when creating a package from a nuspec file, it should be possible to state if the dependencies declared in the nuspec file should be replaced by or merged with the ones found in the "packages.config". I believe this would allow handling project dependencies quite nicely, until NuGet is equipped to handle them natively.

lotharlinux wrote Dec 5, 2011 at 9:59 PM

I need this one too. I like the package to be created with dependencies to both the project references (to projects that have their own nuspec file) and to the packages.config. It requires traversing the folder of the referenced projects and look for the nuspec file etc.

lotharlinux wrote Dec 7, 2011 at 10:14 AM

http://nuget.codeplex.com/SourceControl/network/Forks/lotharlinux/936ProjectReferences
This fork adds tentative support for automatically adding dependencies to referenced projects if they have a .nuspec file in the referenced project folder. How someone will tidy it up and include in the main branch for vNext release.

thargy wrote Dec 22, 2011 at 2:19 PM

Back to what rekna wrote, I'm not sure this is the right approach. The project reference should actually be a package reference, and the two packages should probably be in seperate solutions. The real issue here is lack of support for cascading package dependency builds (I have posted a proposal to address this via an extension http://nuget.codeplex.com/discussions/283866).

If you really wanted to do this then respecting dependencies in the NuSpec as suggested by dandreica and also expanded on in my proposal for cleaning up NuSpec behaviour (http://nuget.codeplex.com/discussions/283859) would allow what you suggest in a less dangerous (overly 'automatic') way. But again, if Package A is a NuGet in it's own right is should probably be developed and tested in it's own right and not part of the same solution as Package B.

lotharlinux wrote Dec 22, 2011 at 3:24 PM

Well. I would very much like this feature supporting the case mentioned earlier for the scenarioes where one has one package per csproj file. When using nuget inhouse for managing dependencies between projects, the feature I have listed is quite simply essential and it works great! If someone thinks this is too much magic, we could add a configuration setting that would explicit allow for this behaviour. Without this nuget is useless in my scenario. I don't really care how its implemented, but I strongly suggest that one consider my argument.

In my specific example I have a solution with 4 project files with are part of a dependency hierarchy and which will all become packages. In order for a reasonably efficient development, the feature I have implemented seems like the best cause of action. This example is just one of many for our frameworks and utility libraries. We have now a total of maybe 25 packages developed inhouse and distributed through a inhouse nuget feed.

rekna wrote Jan 2, 2012 at 9:30 PM

@thargy: the projects definitely do not belong in different solutions. Creating a package for the project that is referenced just for the cause of being able to add it to my other project and to be picked up by nuget packager, would not make any sense, as the package can not be used alone... publishing it on our internal repository would cause other developers thinking it could be used on its own, and this is not the case. I want both projects to end up in a single package, only this way I can make it clear to other developers both libraries need to be added...

deerchao wrote Jan 6, 2012 at 12:35 PM

This would be the only missing feature for me to use nuget to manage my house-made class libraries. Hope it resolved in v1.7.

JeffHandley wrote Jan 12, 2012 at 11:49 PM

Interesting feature idea. Moving to backlog for now.

JeffHandley wrote Jan 12, 2012 at 11:55 PM

Actually moving into 1.8.

giammin wrote Jan 13, 2012 at 7:58 AM

this useful improvement continues to be delayed.

it has started to slide since version 1.5.

is it so hard to accomplish?

xumix wrote Mar 16, 2012 at 6:30 AM

This feature is a real must have. As an option it would be great to have a choise between single package/multi package generation.

xumix wrote Mar 16, 2012 at 6:30 AM

Moreover, there is a patch for this already. Just merge it FFS.

mysterd429 wrote Apr 30, 2012 at 8:07 PM

Hi, all. Does anyone know the status of this issue?

deerchao wrote Jun 5, 2012 at 2:01 PM

Not fixed in 1.8?

lotharlinux wrote Jun 5, 2012 at 2:58 PM

I had plans to look into tidying up my initial patch that was mentioned earlier, but I have had no time todo it. In the meantime the nuget source have also been moved.

BenPhegan wrote Jul 5, 2012 at 12:17 PM

@lotharlinux is your patch still around somewhere? If you don't have time, do you want to share it and see if someone else might be able to pick it up? We would also like this functionality...was thinking of looking into it myself but if you have already written something...

lotharlinux wrote Jul 5, 2012 at 4:56 PM

@BenPhegan I think I have the code lying around somewhere on my laptop. I shall have a look. It would be great if someone could pick it up and continue where I left off. There were a couple of issue with it that caused it not to be accepted into the main branch, but that should be simple enough to fix.

lotharlinux wrote Jul 5, 2012 at 5:27 PM

Attached is the 'src' folder of the nuget fork listed below(the rest is left out to get below the 4mb limit). After the nuget source was moved my old branch is no longer available. Run a diff agains the core nuget 1.(?) branch to see what is different.

One significant change which I remember was to ProjectFactory.ProcessDependencies class. I remember haacked commenting that one needs to check if the packages.config file is indeed included in the project file, checking for precense on disk is not enough. As summed up in the code comment about the change:
        // for each project reference 
        //      - check if it has nuspec file matching the project csfile
        //      - return as package reference using compiled assembly version and id

lotharlinux wrote Jul 5, 2012 at 5:29 PM

Aware of bad spelling in last post...apologies...

mattme wrote Aug 6, 2012 at 11:53 AM

Project references are ubiquitous. Instructions how to package projects with project references are desperately missing from the Nuget packaging guide. These should be written now, for the current behaviour of Nuget, because this is a common real world difficulty. If changes are eventually made per OP's suggestion then the instructions can be updated again.

http://docs.nuget.org/docs/creating-packages/creating-and-publishing-a-package

NB. If you're reading this page, Codeplex displays comments in reverse chronological order (ie. most recent at the top), so to follow the discussion you need to read this page from the bottom up!

brugidou wrote Aug 21, 2012 at 8:48 AM

I creaed a pull request http://nuget.codeplex.com/SourceControl/network/forks/brugidou/nuget/contribution/3277

It's based on lotharlinux initial work.

However the design is pretty different, I believe we should not (or not so magically) create multiple packages out of a solution with project dependencies. This will create confusion since you have no guarantee that your package is built with the correct dependencies. I created a less sophisticated scenario where it simply creates one package with all the necessary files from other project dependencies. So your package will include all your project dependencies and have the necessary nuget dependencies.

We could further discuss a more complex scenario that generates multiple packages out of a VS solution with the correct version constraints etc... It is probably doable and is not incompatible with my pull request actually. It just needs more work.

lotharlinux wrote Aug 21, 2012 at 11:46 AM

brugidou: I do agree that its is one option to just build one package of everything. However in my scenario that would not be sufficient. I still believe that there is a case for offering both.

My original pull request got rejected by Haacked because there were some issues with code conventions and it had a few shortcomings when it came to verifying the presence of nuspec file in the referenced project. It was NOT rejected because if the anything related to the reservations brugidou is expression. In my opinion if someone has the time to finish the work I started or adding to what brugidou has now proposed, then there are good chances to get this into the next release. Unfortunately I do not currently have any time. Maybe the difference in behavior should be configurable. As in either pack all in one package, add dependencies or do nothing.

brugidou wrote Aug 22, 2012 at 8:22 AM

lotharlinux, we could attempt to patch my pull request to have this behavior:
  • Create one big package if no other nuspec is found
  • Add the project reference as a nuget dependency if a corresponding nuspec file is found in the referenced project
I just don't know how to do it clean in the code because what if you have

A(nuspec) -> B(nuspec) -> C
A(nuspec) -> C

C will get packaged in A and B nuget packages.

The other problem that could occur is if you build your package A that depends on B as a project reference, no one prevents you from modifying the B project without upgrading the version number.

My workflow when I work with multiple packages is to release my package locally to test it with my other packages (just like installing a SNAPSHOT with maven locally for example). I don't really create multiple packages out of one solution. I generally do :
  • 1 nuget package
  • 1 solution
  • 1 source control repository
Dependencies rely on nuget, and i try to sync my git tags with nuget package versions.

I suggest that we first merge my pull request which is a first step, and it doesn't block us from implementing the second step (that checks for nuspec files).

lotharlinux wrote Nov 9, 2012 at 7:09 PM

My workflow is as follows: I have several interdependent projects that again have some nuget dependencies. Some of these are projects that almost never change and some change more often. I choose to ignore any potential version numbering problem (which can happen in any scenario).

When building these projects I have want to generate one nuget package for each source project. These nuget packages should then naturally include dependencies to any external nuget packages and also the dependencies that are found implicit via the project references to other projects that also have nuget specs. All my dependencies in this scenario have either an nuget package reference or have their own nuspec file. In these nuget packages are then one single dll library and some dependencies.

C(nuspec) -> B(nuspec) -> A(nuget) (external nuget package ala log4net)
D(nuspec) -> B(nuspec) -> A(nuget)
E(nuspec) -> A(nuget)

A is i.e log4net, a nuget package from the nuget gallery
B, C, D, E are all inhouse developed libraries published at an inhouse gallery. Developers use these libraries.

Now I can modify B and release a new version of package . VisualStudio automatically updates the assembly override for those who choose to use the new version of B.

This works very well for us and we have a "hacked" version of nuget in order to generate the packages using the sources I have posted earlier.

It would be nice if there was an option for nuget.exe which could then tell it to either auto include project dependencies to projects with nuspec or to just ignore them etc.

Lukazoid wrote Nov 27, 2012 at 12:57 PM

The functionality described by lotharlinux would be an absolute dream come true. I can see brugidou's pull request incorporates part of this functionality, but would love for it to go as far as lotharlinux's workflow, do you think there will be any chance of this appearing in a future version of NuGet? Excellent work, both of you :)

Luke

mlang wrote Dec 27, 2012 at 3:08 PM

I have built a post build command line to build a solution with project references into nuget packages. If projects have a nuspec file it builds the reference nodes in the nuspec accordingly based on the project references. You don't even need to add references to the nuspec file manually.

I wrote a blog article on how to setup a solution here:
http://candordeveloper.com/2012/12/12/nuget-package-build-a-solution-of-projects/

And a solution with this implemented can be found on github here:
https://github.com/michael-lang/candor-common

JeffHandley wrote Jan 23, 2013 at 8:40 PM

Triage: moved into 2.3 due to high vote count.
This is not a guarantee that it will be completed in 2.3 though. It will at least get attention though and design consideration.

dotnetjunky wrote Jan 24, 2013 at 10:11 PM

Fei, please investigate and propose a design before implementing.

EddieGarmon wrote Feb 1, 2013 at 8:07 PM

I implemented this as a msbuild extension for the http://adjunct.codeplex.com/ project. the build tasks are actually already on NuGet for consumption, and have been for over a year.

JeffHandley wrote Feb 14, 2013 at 5:25 AM

It sounds like there has been a lot of effort put into finding ways to pull this off. That adds more weight to wanting to do this natively in NuGet.

I hope that everyone who has found a way to make this work for themselves will review the implementation and try out a CI build when it's checked in. :-)

feiling wrote Feb 14, 2013 at 4:44 PM

This is the design that nuget team plans to implement for this feature:
  • Create a new method AddProjectReferenceDependencies() that adds dependencies for referenced projects by the current project.
  • At the end of ProjectFactory.ProcessDependencies(), add call to AddProjectReferenceDependencies().
  • Method AddProjectReferenceDependencies() works by iterating through every referenced project of the current project, gets the id and version of that project, and adds a dependency using the id and version to the package builder.
  • To get the id and version of a referenced project, following steps are taken:
    • If there exists a .nuspec file with the same name as the referenced project’s name, at the same directory as the referenced project file, the nuspec file will be read to get the id and version;
    • Otherwise, the project’s target file is loaded through reflection to get its metadata,
      and the Name and Version properties of the metadata are returned as id and version. If the target file does not exist, an exception is thrown with error message “the project xxx is not built yet. Please build the project xxx first.” If the target file is not a valid assembly, a warning is displayed, and this referenced project is ignored.

feiling wrote Feb 21, 2013 at 10:42 PM

Fixed. Changeset http://nuget.codeplex.com/SourceControl/changeset/0c488cc9e745

Now nuget.exe pack command processes referenced projects with the following rules:
  1. if the referenced project has corresponding nuspec file, e.g. there is a file called proj1.nuspec in the same directory as proj1.csproj, then this project is added as a dependency to the package, using the id and version read from the nuspec file.
  2. Otherwise, the files of the referenced project are added to the package. Then projects referenced by this project will be processed using the sames rules recursively.
    • All dlls/pdb/exe files are added.
    • All other content files are added.
    • All dependencies are merged.
So basically, a referenced project is treated as a dependency if there is nuspec file, otherwise, it becomes part of the package.

giammin wrote Aug 29, 2013 at 4:45 PM

if the verion in the nuspec file is $version$ no version is used

mlang wrote Aug 29, 2013 at 11:08 PM

Giammin,

This is working in my open source solution. Maybe your nuspec file is malformed? Try running release.cmd from candor-common on github.

Article explaining build scripts:

http://candordeveloper.com/2012/12/12/nuget-package-build-a-solution-of-projects/

Candor common source and build scripts:

https://github.com/michael-lang/candor-common

giammin wrote Aug 30, 2013 at 9:00 AM

Ops i forget to add -includeReferencedProjects ...

But now an other problem raises: why specifying both -IncludeReferencedProjects and -Symbols is not supported?