29
Vote

Relax pre-release package restriction policy

description

Consider the following scenario: there is a package (e.g. Super.Log) which goes through version 0.1 to 0.9 and is now approaching release. Developers publish a version 0.1-rc, and this becomes a problem for other packages that reference Super.Log.

So if we have another package MyComponent version 0.6 that listed Super.Log in its dependencies, it can no longer list the latest (and best) Super.Log version because NuGet policy does not allow referencing pre-release versions of other components from non-pre-release packages. Developers of MyComponent will have assign a version 0.7-beta or something in order to be able to use Super.Log. This will affect all MyComponent users who regularly updates their dependencies using Update-Package command. All pre-release components must be installed or updated by explicitly choosing pre-release option, so by default MyComponent users will not be getting most recent version.

As a workaround some package publishers publish them twice: as pre-release and as a regular version, take a look for example at Simple.Data.Core: https://www.nuget.org/packages/Simple.Data.Core/ It has duplicate versions for all its recent packages because of this. But Simple.Data does not have any external package dependencies, and if it had a dependency on another package with pre-release versions, such workaround would not be possible.

I find this policy to be prohibitive for frequent incremental updates. Moreover, since it has a transitive nature, one little package may decide version numbering for the whole chain of related packages. I can see two ways to improve it:
  1. Turn an error condition into a warning condition, so developers are notified about using pre-release packages but are not forced to change their version policy.
  2. Introduce an additional option to "NuGet Pack" command to suppress an error condition due to use of pre-release packages.

comments

maartenba wrote Feb 15, 2013 at 10:16 AM

My first reaction to this: if you depend on prerelease, you are prerelease.

XavierDecoster wrote Feb 15, 2013 at 10:20 AM

I have a feeling this is opening pandora's box, and really don't like this suggestion...

If a consumer does not explicitly opt-in to get pre-releases, then only stable versions should be listed.
If you are a stable version depending on prereleases, you're basically telling the consumer: "Meh, you'll get prereleases anyway, 'cause I as a package producer didn't like shipping a prerelease".

Because some other package producer decides to ship a prerelease, a depending package producer decides to hide this "instability" in a non-pre-release. I don't like the sound of this. It's a way of forcing something upon package consumers.

In other words: if you would be a release depending on a prerelease, you shouldn't be listed in the search results if prereleases were not explicitly included. I think that's exactly what you're trying to avoid by trying to create a release...

object wrote Feb 15, 2013 at 11:21 AM

Hi Xavier,

I think a lot depends on what meaning we put in the word "pre-release". I disagree with boolean separation of packages on "released" ("everything works great!") and pre-released ("not yet stable"). I've seen packages that have been in pre-release state for more than a year due to perfectionist policy of their owners, as well as packages that had several major releases in a few months. So what would you suggest me to do in the following situation:
  • I have MyPackage 1.0.0.0 that depends on HisPackage 2.0.0.0.
  • Customers report bugs that appear to be in HisPackage. I report them to the HisPackage owner.
  • HisPackage owner fixes the error and publishes HisPackage 2.0.1.0-rc, but then he wants to include some more features, do more tests etc. So he publishes 2.0.2.0-rc, 2.0.3.0-rc, so month after month the only "stable" version is 2.0.0.0 which is in fact not stable at all.
  • Since I can not reference fixed version in my released package MyPackage 1.0.1.0 I have to publish a pre-release version 1.0.1.0-rc and keep on telling my customers that they should be using that one. Those who have a policy of not using unreleased version have to stick with a buggy one. In addition people who are unaware of all these policies and simply write "Install-Package MyPackage" get the buggy version.
As you see, all these formalitites about defining quality based on naming convention led to a lower quality for users who are not specifically instructed about what package version is the best to use. It must be a package owner's resposibility (and mandate) to declare a specific package version as the recommended for use and he should be entitled to publish it so everyone gets it by default unless they specifically ask for something else. Right now this is not possible due to NuGet pre-release restriction.

Vagif

object wrote Feb 15, 2013 at 11:29 AM

Hi Maarten,

See my reply to Xavier. Let's not names fool us. If I depend on a package with known bugs, I must get a better package, even if the buggy package was the official greatest relelease, and the new a better one is just a humble attempt to fix the bug. If the official package breaks my unit tests and the unofficial make the tests pass, why should my customers be forced to get the buggy one by default?

XavierDecoster wrote Feb 15, 2013 at 12:06 PM

@object Don't get me wrong here, I do understand your frustration. Package publishers have a great responsibility: each decision has impact on dependent packages and can create a chain effect on their publishers.

Here are the things you can't control:
  • release cycles of your third-party dependencies
  • consumption policies of your consumers
But as a package publisher, you do have control over:
  • which dependencies you want to rely on
  • your own release cycle
  • communication to your consumers
I strongly believe that "perfectionist policies" and keeping packages in pre-release for over a year should be discouraged instead of embraced by providing workarounds. Consumers eventually will also shift away from using such packages, and you'll end up relying on a dependency nobody wants in the end.

If you feel a certain dependency is becoming an impediment, then try to find a way to shift away from it. This should be enough of encouragement for those third-party package creators to release more often or to listen to their consumers (which you are!).

A pre-release is by definition something which is not a full-blown release, and as such should not be considered "production-grade" (not necessarily unstable, but still). It might be a good candidate for release, but that just doesn't cut it to give it the "production-grade" stamp, and such implies that everyone who depends on it should not be considered "production-grade".

If you think in terms of support:
  • a release is something you should always get support on
  • a pre-release is something you use at your own risk
Extreme illustration:
Would you like to vouch for million dollar claims towards your consumers because of a bug caused by one of your dependencies?

I can't think of any other meaningful definition for release vs pre-release other than the above. Maybe someone else can prove me wrong, but I'm pretty sure I'll ask quite a few questions trying to tackle your approach :-)

object wrote Feb 15, 2013 at 12:48 PM

@XavierDecoster,

I agree with your distinction between released and pre-released version as something that you should always get support on and something you use at your own risk. But in practice, can you say anything about support expectation for PackageA 1.0 vs PackageB 2.0-rc? Not until you know how's the package owner. Moreover, due to NuGet versioning restrictions there are some chances PackageB is in pre-release state ONLY because of the pre-release state of one of its dependencies. And if you try to get support on the last release version 1.9 the response will be "sorry but it had some problems due to its dependencies, so please use 2.0-rc and just ignore its release state". We are back to the responsibility of the package owner to define what's most stable and what has better quality.

To give an absurd example: should you expect better stability from a package with version 0.1 or a package with version 4.0-rc? According to NuGet, the one with version 0.1 is more stable, because it's just fine to include it in any released package. Unlike the one with version 4.0-rc.

And NuGet these days is used for corporate development with continuous delivery. They may simple assign an incremental number to its releases: 1.1, 1.2, ..., 1.35 etc. Until one of its dependencies is comes in a pre-release version. Then they must change their internal versioning policy and use something like 1.36-rc. Does this make much sense? Not in my opinion.

Semantic versioning is about contracts, not about viral quality state enforcement. Some packages are released by individual developers, some by large companies. They all have different development cycle, QA requirements there will be a least common quality denominator between them. We are in the era of continuous changes and continuous delivery, and we should be entitled to define the content of our releases, no matter what naming scheme other vendors prefer.

XavierDecoster wrote Feb 15, 2013 at 2:06 PM

@object Here's where I always say: a tool cannot fix a broken process, it can merely support/encourage you in doing the right thing. In addition, it should prevent you from doing the wrong thing.

That's exactly why I don't think this rule should be relaxed.

A consequence of introducing NuGet is that package producers finally need to think about their versioning strategy. (Incremental) build numbers are just metadata, hence why SemVer suggests using a '+' sign for those after the optional prerelease tag, as the prerelease tag is considered more important than the build number. This is illustrated in the SemVer version precedence algorithm. (NuGet is waiting for SemVer RTM to fully support this AFAIK)
All software has bugs, given enough iterations you'll fix a few and add a few more. Your fixes could even introduce new bugs or cause regressions. Assuming a correlation between version number and (perceived?) quality is wrong, but that's exactly why the pre-release tag comes in handy. This is the flag you can toggle to distinguish quality releases from pre-releases.

Obviously, a repository owner cannot control this for each package publisher, and I don't see how this could be objectively build into the tooling.

We both agree that SemVer is all about indicating changes in your public API / contract, but this has no correlation with software quality.
In your extreme case: assuming v4.0.0-rc1 is more stable/production-grade than v0.1 might be caused by such perception.
On a side note: maybe NuGet should consider versions below 1.0 also as pre-release?. That could make sense to me...

Automated builds + packaging, and automatically pushing to a repository doesn't mean you're doing Continuous Delivery. The repository you'll continuously push to likely won't be the same as the one you'll expose to your consumers (even internally), unless you want to have your (internal) customers getting a package update notification after each commit you or anyone else on the repository does. That's quite a challenge in terms of QA and should not be taken lightly.

Still don't see why this restriction should be relaxed. IMO, relaxing it removes a built-in protection against yourself from doing "the wrong thing".

dotnetjunky wrote Feb 15, 2013 at 2:10 PM

I agree with Xavier that when we added support for prerelease version, we meant it for "unstable" packages. I put it in quote because as you pointed out, 1.0.0-rc can be more stable than 0.1. However, from Nuget's point of view, if you put the "rc' tag to your package, it means it's not "released" quality.

Having said that, NuGet only enforces this restriction at package creation time. At package installation time, NuGet still allow installing a pre-release dependency package if user provides the -Prerelease switch.

object wrote Feb 15, 2013 at 2:40 PM

@dotnetjunkie,

"when we added support for prerelease version, we meant it for "unstable" packages"

Yes it was a good intension but look at the current practice. Packaged marked as "rc" is almost alaways considered to be of a better quality than the previous published package.

"Having said that, NuGet only enforces this restriction at package creation time. At package installation time, NuGet still allow installing a pre-release dependency package if user provides the -Prerelease switch."

That's another problem. It would be easier live with this policy if there was a way to mark pre-release package as recommended, so people will get it by default. As it is now, once package is marked as pre-release, it will not be selected unless people specifically ask for it.

After all, I am not saying this policy should go. It should probably stay as a default option. But developers should be able to overrule it and stay in charge of their versioning scheme. Why don't you arrange a poll to ask NuGet publishers?

object wrote Feb 15, 2013 at 2:56 PM

@XaviewDecoster, my main objective is that current publishing practice is quite different, even for companies like Microsoft. Do we write production code with Visual Studio CTP? Yes we do. Why does Microsoft anounce "GO LIVE" versions of their yet-to-be-released products? Because they consider them stable enough to encourage people to take advantage of new features and fixes. The same with NuGet packages: a "rc" tag in most cases means something of a better quality than the previously published package.

Since it looks like we both are out of more arguments, perhaps it's worth to ask NuGet publishers what do they practice and what they prefer? Take for example, strong naming and signing of assemblies. A few years back it was presented as a good publishing practice. Now developers often try to avoid it for a number of reasons. But imagine it would be a mandatory rule? How much harder development would be? My only point is that developers should be entitiled to choose their preferred versioning scheme.

XavierDecoster wrote Feb 15, 2013 at 3:31 PM

@object I still disagree :-)

There is NO correlation between package version and product quality, at all! If there is, there should be a mathematical prove for it... Unless I see such prove, those arguments are based upon perception and expectations. A version number is not a quality indicator.

Also, the proper way of doing things is not defined nor is it reflected by the number of people that actually apply it in practice today. As much is it may sound surprising, using NuGet comes with a learning curve, which implies practices will also change over time. If a huge part of package authors applies some practice today, it doesn't mean they'll still adhere to it tomorrow, when they've learned about another, maybe better, way.

Haacked wrote Feb 15, 2013 at 4:25 PM

  1. Turn an error condition into a warning condition, so developers are notified about using pre-release packages but are not forced to change their version policy.
I agree with the first part of this proposal wholeheartedly:
  1. Introduce an additional option to "NuGet Pack" command to suppress an error condition due to use of pre-release packages.
I don't think this would be necessary if we did #1.

When we designed the pre-release feature, we wanted to make sure the user knew what they were installing. By default, they will never get packages marked as "pre-release". Not even by transitive dependency.

However, we were too rigid. This hurts a scenario where package authors who have built a release version of their package cannot package it as a release because they depend on a pre-release. Perhaps they've tested their use of it and can vouch that their usage of that pre-release package is stable.

ReactiveUI is an example of this. It depends on the Microsoft.Bcl.Async targeting package which seems to be stuck in perpetual pre-release. Paul Betts tried to create his own version of that package, but the contents are not OSS. So he's effectively stuck and had to resort to a hack to use install.ps1 to install it. This sucks because now that is a hidden dependency. It doesn't show up in the gallery or in the package metadata.

When we have a feature that requires ugly hacks like this from mainstream packages, we need to step back and reconsider our approach.

Louis DeJardin, David Fowler, Paul Betts, Drew Miller, and I had a long conversation about this at MonkeySquare and I really liked the proposal everyone came up with.
  1. NuGet.exe pack gives a warning (not error) when packing a release package that depends on a pre-release package.
  2. The NuGet gallery allows uploading such a package. Again, nuget.exe push could also offer a warning if we care.
  3. a) Package Manager Console: When a user installs a release package with a pre-release dependency without the -PreRelease flag, we fail the install and give them instructions to use the -Prerelease flag.
  4. b) NuGet VS Extension: When a user installs a release package with a pre-release dependency, but didn't have the "Allow prerelease" checkbox set, the installation fails with a dialog that explains the situation clearly and allows them to install it anyways.
This still meets the spirit of our initial design goals. It keeps the power in the hands of the user, but provides flexibility to package authors. In fact, it gives more power to users because more packages will be available to them that might not otherwise be and they have the option to install them.

Haacked wrote Feb 15, 2013 at 4:25 PM

Ack! Markdown fail. ;)

dotnetjunky wrote Feb 15, 2013 at 5:42 PM

Phil, we need to define exactly what it means when "a release package depends on a pre-release package"? I think we have been vague on this. Does it mean the dependency is exact version [1.0.0-rc] or the version range has a pre-release version at either end, e.g. [1.0.0-rc, 2.0.0), [1.0.0, 2.0.0-rc) ?

In case of the exact version, nuget.exe pack should fail right away, because there's no point creating the package that we know it will always fail at installation.

In the latter case, throwing at installation doesn't make sense because there can be a higher release version that satisfies the constraint.

XavierDecoster wrote Feb 15, 2013 at 5:53 PM

@haacked: I can agree with throwing warnings all over the place and requiring explicit consent from the consumer etc, but how can anyone vouch for their usage of a third-party pre-release without constraining their dependency to exactly that specific version only? Wouldn't that be a requirement of doing this?

I'm not a lawyer either, but I'm honestly questioning what implications this could have.

I'm aware of these PowerShell hacks and I agree it's horrible that package authors are pushed into corners like this, but I still believe the issue is mainly caused by a package author and not by the feature.

Also, if you'd provide options to turn errors into warnings, then at least give us the option to throw real errors in custom package analysis rules (they're all treated as warnings today) :-)

xpaulbettsx wrote Feb 15, 2013 at 10:21 PM

I am voting for this bug

jlaanstra wrote Feb 15, 2013 at 11:43 PM

Agree with Phil here. The work-arounds with Install.ps1 are ugly. This pre-release policy should be relaxed.

dotnetjunky wrote Feb 19, 2013 at 6:06 PM

I repeat my question:

I think we have been vague on what it means for a released package to depend on prerelease packages. Does it mean the dependency is exact version [1.0.0-rc] or the version range has a pre-release version at either end, e.g. [1.0.0-rc, 2.0.0), [1.0.0, 2.0.0-rc) ?

dotnetjunky wrote Feb 22, 2013 at 12:27 AM

We need to think through the design.

object wrote Feb 22, 2013 at 8:40 AM

@dotnetjunky,

"Does it mean the dependency is exact version [1.0.0-rc] or the version range has a pre-release version at either end, e.g. [1.0.0-rc, 2.0.0), [1.0.0, 2.0.0-rc) ?"

I agree it's important to have a resonable range semantics for pre-release versions, and IMHO the following rules may apply:

Alternative A.
  • Pre-release versions are only used for explicit match, they can not be included in ranges. If a package is dependent on a pre-release version, it must state it explicitly.
Alternative B.
  • Pre-release versions may be used in ranges
  • A released version is greater than a pre-release version with the same number: N1.N2.N3.N4 > N1.N2.N3.N4-xxx
  • For two pre-release versions N1.N2.N3.N4-xxx and N1.N2.N3.N4-yyy string comparison applies, i.e. if xxx > yyy then the first package has a greater version
Personally I am in favor of the second approach as more flexible: it will be possible to automatically update packages when new pre-release versions with the same number arrive. I understand however concerns of those skeptical to this and treating pre-release versions as temporary exceptions.

flank42 wrote Apr 1, 2013 at 4:41 PM

I'd like to point out that this prerelease issue exists if you're using Unity with .NET 4.5. I really wish Microsoft would update the package since last August :( So there ARE some legitimate cases where this occurs.

JeffHandley wrote Aug 8, 2013 at 7:51 PM

We're talking about this and considering relaxing the restriction. I was thinking of the downstream effects of that though and realized something.

When A depends on B and B is only available in pre-release form, why does A want to be stable?
Because it allows users to install A without having to specify "Allow Prerelease".

If A could be stable and depend on prerelease B, then can users more easily install A?
No, we would fail the installation of A unless the user had specified "Allow Prerelease".

Doesn't that mean that we'd be back to square one?

XavierDecoster wrote Aug 9, 2013 at 11:29 PM

Relaxing this restriction during package creation? Sure, why not. Authors decide what they create.

Relaxing this restriction during package consumption? Please make this opt-in... and allow consumers to decide whether they want this or not. Consumers decide what they consume, and should not be silently enforced to depend on any prerelease packages while they expected to rely on a stable dependency. It's also very likely they did not explicitly used the -IncludePrerelease option, which also implies there should be no prerelease dependencies as a result of this operation.

If a consumer installs a stable package, he won't expect to install any unstable dependencies. That's why I believe this should be opt-in from a consumer POV. The consumer should be made aware of this and explicitly allow it.

Perhaps there should be an option -AllowPrereleaseDependencies (maybe also make this globally configurable similar to the package restore consent option?).

dotnetjunky wrote Aug 10, 2013 at 12:06 AM

This change will only affect package creation. Package installation will keep the current behavior, meaning -IncludePrerelease is still needed to include prerelease packages.

XavierDecoster wrote Aug 10, 2013 at 12:26 AM

@dotnetjunky Excellent! :) Still wondering whether pkg authors need to define such dependency explicit or not, e.g. [1.3.0-alpha]. That's after all the only version they can vouch for given the specific release of their package?

dotnetjunky wrote Aug 10, 2013 at 12:23 PM

it doesn't have to be explicit. it can be min version, e.g. [1.3.0-alpha, ]

FilipDeVos wrote Aug 13, 2013 at 9:01 AM

If by default no pre-releases are installed this looks good to me. (I think it would also solve the ticket https://nuget.codeplex.com/workitem/3017)

JeffHandley wrote Aug 13, 2013 at 11:18 PM

I closed #3017 as a dupe of this one.

JeffHandley wrote Aug 14, 2013 at 12:09 AM

After talking with Paul Betts and @hacked in various forums, we've concluded on the following approach:
  1. Remove the restriction from nuget.exe pack
  2. Ensure that the gallery will accept a stable package that depends on a prerelease package
  3. Console (without -pre): when the stable package has a dependency on a prerelease package that is not already installed, warn (but don't prompt)
  4. Console (without -pre): when the stable package has a dependency on a prerelease package that is already installed, prompt
  5. Dialog (with Stable only): always prompt
We need to be clear on the prompt to not scare people away but also let them know what will happen.

object wrote Aug 14, 2013 at 3:46 AM

Perfect! Thanks for that.

Vagif

XavierDecoster wrote Aug 14, 2013 at 6:45 AM

Sounds good!

JeffHandley wrote Aug 15, 2013 at 6:58 PM

The NuGet team is discussing this further and we need to revise the plan above slightly. We merged the two console scenarios into one and changed it so that instead of prompting, it fails with an explicit error that tells you how to proceed. This is essentially a different kind of prompt--kind of like how git suggests tells you when you need to specify --force.
  1. Remove the restriction from nuget.exe pack
  2. Ensure that the gallery will accept a stable package that depends on a prerelease package
  3. Console (without -pre): when the stable package has a dependency on a prerelease package, the installation will fail with a message the includes the exact console command that needs to be run to get the install to succeed. For instance, "To install version 1.0.0 of Foo, which depends on 2.3.4-beta of Bar, run the following command: Install-Package Foo -Version 1.0.0 -IncludePrerelease". This message will also call out whether you already have Bar installed as a different version.
  4. Dialog (with Stable only): always prompt and allow the user to continue with the installation if they desire
Note that #4 is a lot of work, especially when we work on the package updates tab and the update all button. We will need to further investigate this to estimate the work and ensure a solid and thorough implementation. To justify the work, we may need to see more support (votes) of this.

nhhagen wrote Aug 15, 2013 at 8:38 PM

As as public package consumer and an internal package producer and consumer this is quite a big issue to get right, if to be supported at all. We have currently chained our builds in such a way that a "release build chain" will stop if there is a pre-release decency (the build breaks). I know this is probably not the best way to do this and we are looking at moving to git-flow to deal with releases.

If this feature is implemented I would like experience to be like this.
  • nuget.exe pack - gives an error if there are pre-release dependency (I need the build to fail, so a non zero exit code is needed for the TeamCity NuGet pack stage to fail, if I remember correctly) and a tip on how to get the package to build.
  • nuget.exe pack -pre - warns you that you are including pre-releases.
  • Install-Package|Update-Package - gives an error with a tip on how to get the package with the pre-release dependencies.
  • Install-Package|Update-Package -pre - warns you that are installing pre-release dependencies.
  • nuget.exe install|update- gives an error with a tip on how to get the package with the pre-release dependencies.
  • ´nuget.exe install|update -pre` - warns you that are installing pre-release dependencies.
It looks like I mostly agree with @JeffHandley except for the nuget.exe pack command.

I'm not sure how one should deal with dependencies that are already installed and how to deal with and update after you have said OK to pre-release dependencies for a package, is it then OK forever?

Have you looked at how maven, npm, bower and gem deals with the same issue? I would be nice if all of them including nuget work the same way with regards to versions.

On a side note: I do not take a project that are stuck in an infinite pre-release loop seriously and try to find another project, but that is just me :)

nhhagen wrote Aug 15, 2013 at 8:42 PM

Argh, markdown errors and a couple of typos, where is the edit button?

object wrote Aug 16, 2013 at 3:37 PM

@nhhagen,

"On a side note: I do not take a project that are stuck in an infinite pre-release loop seriously and try to find another project, but that is just me :)"

I believe the quality of the project should be measured by more meaningful parameters than the version assignment policy used by third party teams. It's not uncommon that some projects of a very high quality keep pre-release versions for a long time (more than a year!) to ensure absolutely everything works fine when they can finally put "Version 1" tag on it. On the other hand there are plenty of projects with high frequency of releases that cause troubles to their consumers. By relaxing its pre-release policy NuGet will allow package consumers to have a final judgement on package quality.

Cheers

dotnetjunky wrote Sep 24, 2013 at 12:32 AM

There're several complications that we need to think through before we can do life this restriction in a satisfactory manner. Moving this to 2.9.

johncrim wrote Nov 14, 2013 at 12:14 AM

We use NuGet for internal and external project dependencies.

In our release process, it's very helpful that nuget pack fails if a release package depends on prerelease packages. It reminds the person performing the release that dependencies must be released before the current project can be released. Our release process relies on this restriction.

If this restriction is removed, we'll need to add our own safeguards so that projects are not released with prerelease dependencies. This means parsing packages.xml and version numbers. So I'd prefer that this restriction remain; and I'd support the proposal that a switch be added to allow nuget pack -pre to succeed.

AbbasSyed wrote Jan 6 at 7:11 AM

Hi,

I cant add the Pre-release package in my Nuget. Is this possible to add a Pre-Release package in my own nuget package? if possible please provide the steps to me.

Thanks,
Abbas K