How to keep track of solution level packages when packages not checked into source control?

May 12, 2011 at 10:42 AM

I would prefere to not have packages checked in into source control. Packages can get big, I prefer pacjages to be pulled in on a developer machine or a build server (I know bsimser, build server do not always have internet access, we could solve this with an on-disk packages cache).

I also want to use solution level packages, so packages only installed on the solution level, and not installed on projects. For example for scaffolders that are useable on all my projects, because the scaffolder itself creates a project for example. If I don't install by packages folder into source control (except the repositories.config file) there is no place where we keep track of packages installed at the solution level.

I think this could best bew solved as follows:

  • Manage solution level packages in the repositories.config file
  • Don't place the repositories.config file in the packages folder, so the packages folder can be excluded as a whole from source control
  • Rename the repositories.config file to a more appropriate name, maybe something like myproduct.sln.packages

To go completely over the top: It would also be great if there was an API to add/remove per package information in the repositories.config and packages.config file. This would allow for example chocolatey to keep track of installed application packages that should be shared for the team, so when a new team member checks out a solution from source control, he can update its development mnachine with all required tools from the nuget feeds.

 

May 12, 2011 at 10:55 AM

Note 1: you can install packages at the solution level in two ways:

  1. in a blank solution execute install-package <yourpackage>
  2. using nuget.exe (when there are already projects installed in your solution), I still think there should be an install-package mypackage -solution so the Init.ps1 is executed

Note 2: the current behaviour requires you to check in your packages folder (and of course the packages.config files in all your projects) into source control, directly solves the problem with build servers that have no connection to the internet.

Note 3: the current approach of chocolatey forces you to add tools required in your build process to a tools folder, or include them in a custom package that has the tool in the Tools folder. Would prefer a chocolatey approach to circumvent licensing issues.

 

 

Coordinator
May 12, 2011 at 3:50 PM

Note that if a package does not contain a content folder nor a lib folder, it will install at the solution level only today.

May 12, 2011 at 5:33 PM
Just addressing note #3 about chocolatey: Chocolatey is not about circumventing licensing at all. It embraces and reminds you that you accept licensing by installing a tool every time you run it. It even points you to the full acceptance terms which you can get if you run chocolatey w/out any arguments or with /? :

-------------------------
Package License Acceptance Terms
-------------------------

The act of running chocolatey to install a package constitutes acceptance of the license for the application, executable(s), or other artifacts that are brought to your machine as a result of a chocolatey install. This acceptance occurs whether you know the license terms or not. It is suggested that you read and understand the license terms of any package you plan to install prior to installation through chocolatey. If you do not accept the license of a package you are installing, please uninstall it and any artifacts that end up on your machine as a result of the install.

It is not required that you put your tool in the tools folder. If there are licensing concerns, its better to use the chocolateyInstall.ps1 to download a tool from a source and run an installer instead (or unzip).

A great example is nearly all of the packages in https://github.com/ferventcoder/nugetpackages

Take a look at Skype (https://github.com/ferventcoder/nugetpackages/tree/master/skype). Nowhere does it include the Skype application. Look around in here. Now take a look at the chocolateyInstall.ps1 (https://github.com/ferventcoder/nugetpackages/blob/master/skype/tools/chocolateyInstall.ps1). The contents of the file literally tell chocolatey where to download it from and how to install it.

Install-ChocolateyPackage 'skype' 'exe' '/SILENT /nogoogle /noie /nodesktopicon"' 'http://download.skype.com/SkypeSetup.exe'

Chocoately in this case is doing nothing different than you would do. It downloads the tool like you would and it runs the installer like you would (with silent switches). You still work with licensing of Skype based on it's native installer. If there is a concern with what chocolatey is doing here, then everyone is breaking licensing concerns by manually downloading it because it does nothing differently (just automates the manual process).

Take a look at sysinternals (chocolateyInstall.ps1):

Install-ChocolateyZipPackage 'sysinternals' 'http://download.sysinternals.com/Files/SysinternalsSuite.zip' "$(Split-Path -parent $MyInvocation.MyCommand.Definition)"

It is downloading and unzipping a file to a known location on your computer. It even helps you find where the licensing is located (http://nuget.org/list/packages/sysinternals). Follow the link to the license.

As I mentioned already, chocolatey is about embracing licensing instead of circumventing. I am not a lawyer, but if you have properly addressed licensing for your nuget package, anything in the package is covered by that license. But if you are still concerned about licensing, you can create a native installer (inno setup, msi, other) and point to it instead. Have I addressed your concerns with licensing and chocolatey?

____

Rob
"Be passionate in all you do"

http://devlicio.us/blogs/rob_reynolds
http://ferventcoder.com
http://twitter.com/ferventcoder

May 12, 2011 at 5:41 PM
Pardon my spelling and grammar mistakes.
May 12, 2011 at 6:28 PM

@ferventcoder: sorry that I expressed myself badly in note 3. I mean exactly what you mean. There is a lot of software that you depend on, but can't deliver with your product. In our software factory we depend on these kind of tools. With chocolatey we have a means of making sure the required tools are installed without packaging the tool with our factory and breaking licenses. Circumventing is deffinitly the wrong word for what I tried to express:-)

PS: why did you choose c:\nuget as installation folder, instead of a Tools or Bin or configurable folder next to your solution?

May 12, 2011 at 7:20 PM
For things you would likely never be able to get from chocolatey, we've talked about a requires attribute you could put in the header of the chocolateyInstaller.ps1. Right now it is very much in concept mode. There are far too many questions to be answered.

This is a VERY trivial example, but something like
requires 'Visual Studio' -version '2010'
Each requires would be an AND. So how would you address OR? How would you address different SKUs of the same product? How do you actually figure out if it is installed on the machine?

It would have to be more rugged than the example above and could get ugly with all kinds of fun things (per requires).

It's gets a little more intimate with Windows (and each version) than I think we want to be at this point.


May 12, 2011 at 7:29 PM
Addressing C:\Nuget selection:

Chocolatey is not for libraries as much as it is for applications/tools. One way to think about it is that it's for the compiled stuff. :D

NuGet (or vanilla nuget) is for library packages. And developer focused. Chocolatey NuGet is for application packages. And not necessarily developer focused.

With that chocolatey is a PER machine kind of repository. You install adobereader once (filezilla, fiddler, lockhunter, msysgit, etc, insert actual installed application here). You install lessmsi once (warmup, nhprof, console2, poshgit, sysinternals, etc, insert non-installed tool name here). You want to hit your tools from a command line anywhere on the machine. You don't want to clog up your applications with the tools you use on your machine. These are things that drive a need for chocolatey.

If you are wanting to stay closer to the solution, look at nuget commandline. It will allow you to get things down to anywhere you want.
Coordinator
May 12, 2011 at 7:46 PM

Another way to look at it is that Chocolatey is a machine level package manager built on the NuGet infrastructure. But in many respects, it’s a different product than Nuget for a different scenario.

Rob, do you think it might make sense to have a separate Chocalatey repository using the NuGet Gallery implementation?

May 12, 2011 at 7:59 PM
Phil,
That is something I've been thinking about as well. Wondering when/if it continues to grow should it move off on its own (I think at some point it will likely move). There are many developer focused packages that include tools with them. Chocolatey capitalizes on that as well. Like RavenDB or NUnit. But I have been thinking about how that exit strategy would look if it came to be. Right now it's so young (I haven't even blogged about it yet!), but the initial reception has been tremendous when I've showed folks the tool.

This kind of steered off of the initial discussion, but basically the feature where you can get the aggregate sources in with the command line tool would help facilitate this idea. That way it could hit the chocolatey only packages and the packages that are developer focused as well without more thought on the part of the consumer.

In my mind I keep seeing that gallery's site address as http://chocolatey.nuget.org ;-)
May 12, 2011 at 8:00 PM

I understand your ideas, they are great and solves a big issue. One thing I would like chocolatey to do however is detect that a package is already installed, so in Tools\Init.ps1 or another script I can make sure that all required tools for my factory are installed. I could write code per package to test if it is installed or not, but it seems like a functionality that is better included in chocolatey itself. In that case a list of installed packages should be managed somewhere (in c:\nuget folder) for quick testing. I still can solve the issue by code like:

if(-not (Test-Path -Path c:\nuget\lib\7zip.9.20.0.2)) { Chocolatey install 7zip }

May 12, 2011 at 8:08 PM
I think I see where you are going with this.

The idea here is something like a setup.bat that would ensure a machine is in a state for developing against. Is that the general idea?

I see this implemented like this:
  • Your source has a setup.bat that points to a powershell script.
  • The powershell script detects the presence of chocolatey and installs it if it is not there.
  • The script asks if things are installed at machine level (cisinst 7Zip) and it looks to see if something is installed. (could include version here and source as well)
  • If it is not installed, it runs install for it automatically.
  • and so on.
the script might look like this (pseudocode):
if (cinst == null) {
take steps to download chocolatey
run chocolateyInstall.ps1
}

cisinst 7Zip
cisinst psake
cisinst baretail

This closer to what you mean?
May 12, 2011 at 8:09 PM

Of course that logic could also be in your init.ps1

May 12, 2011 at 8:23 PM

I wonder if you should go too generic with chocolatey, even if you can. There have been more initiatives for Windows (http://www.logilab.org/blogentry/9861) but non of them really took off. I have seen initiatives at big companies where a file share contains a batch script to do unattended installation of applications, but your approach is nicer with respect to the usage of PowerShell, functions for downloading and installing from the internet, versioning support, making apps accessible from the command line and making installation scriptable. I think it is best to keep it small and focused with a repository for .Net development. In the iPhone app store it is almost impossible to find the application you need. 300 Todo managers, but which one is best (rating system basded on amount of downloads?)?

May 12, 2011 at 8:27 PM

That is exactly what I mean! But as far as I can see this functionality (cisinst) is not there yet? Or did I oversee something? 

May 12, 2011 at 8:29 PM
I've been at two companies where I've been heavily involved in the batch scripting for unattended installation. That is where the approach for chocolatey came it. What if we took that concept, and made it global? And thats where the chocolateyinstall.ps1 came from.
May 12, 2011 at 8:30 PM
https://github.com/ferventcoder/nugetpackages/issues/22

Chocolatey needs a mailing list... more things to do. :D
May 12, 2011 at 8:31 PM

Good work:-)

May 12, 2011 at 9:15 PM
Mailing list: http://groups.google.com/group/chocolatey
____
Rob
"Be passionate in all you do"

http://devlicio.us/blogs/rob_reynolds
http://ferventcoder.com
http://twitter.com/ferventcoder


On Thu, May 12, 2011 at 3:30 PM, Rob Reynolds <ferventcoder@gmail.com> wrote:
https://github.com/ferventcoder/nugetpackages/issues/22

Chocolatey needs a mailing list... more things to do. :D