Solution folders fail

Developer
Sep 11, 2010 at 9:06 AM

Solution folders are bad to use for packages since they don't actually map to physical folders on disk. Right now we're faking it by mimicking the structure on disk in the solution. Very early in the project we talked about having a special package project type so we can add the actual files and folders without having to fake it all. I think this needs to be done sooner than later.

Another problem I'm having right now is testing the package manager against source control (TFS2010) . Since the solution folders under the packages aren't mapped to any physical folder, deleting them won't remove them from source control. After removing a package that's under a source control system like TFS you'll be left with empty folders.

Thoughts on this?

Sep 11, 2010 at 1:29 PM
The problem here is the reliance on VS for the source control part.
Is there a reason that NuPack can't simply add it to the disk, and if you are using VS based SCM, add it there as well?
If you are using disk based solution (SVN, Git, Hg), it will just pick it up automatically.

On Sat, Sep 11, 2010 at 11:06 AM, dfowler <notifications@codeplex.com> wrote:

From: dfowler

Solution folders are bad to use for packages since they don't actually map to physical folders on disk. Right now we're faking it by mimicking the structure on disk in the solution. Very early in the project we talked about having a special package project type so we can add the actual files and folders without having to fake it all. I think this needs to be done sooner than later.

Another problem I'm having right now is testing the package manager against source control (TFS2010) . Since the solution folders under the packages aren't mapped to any physical folder, deleting them won't remove them from source control. After removing a package that's under a source control system like TFS you'll be left with empty folders.

Thoughts on this?

Read the full discussion online.

To add a post to this discussion, reply to this email (nupack@discussions.codeplex.com)

To start a new discussion for this project, email nupack@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com


Sep 11, 2010 at 4:08 PM

A custom project type is a option of last resort.  Having a custom project type means that in order to load a solution the project type needs to be installed on the devs system.  For this project to be successful, it has to be non invassive and I think a custom project would not be able to do this, as far aas I know.

I have used other project types to do this before and played with the build configurations, but thoses were hacks.

I am not sure what the right thing to do is, but it needs to leave nupack as an optional install for the devas working on the project.

Sep 11, 2010 at 4:24 PM
Standard C# project with the asms as <content> should work

On Sat, Sep 11, 2010 at 6:09 PM, erichexter <notifications@codeplex.com> wrote:

From: erichexter

A custom project type is a option of last resort.  Having a custom project type means that in order to load a solution the project type needs to be installed on the devs system.  For this project to be successful, it has to be non invassive and I think a custom project would not be able to do this, as far aas I know.

I have used other project types to do this before and played with the build configurations, but thoses were hacks.

I am not sure what the right thing to do is, but it needs to leave nupack as an optional install for the devas working on the project.

Read the full discussion online.

To add a post to this discussion, reply to this email (nupack@discussions.codeplex.com)

To start a new discussion for this project, email nupack@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com


Sep 11, 2010 at 4:54 PM

A standard project needs at least one class file or it with fail a build as fa as I know.

On Sep 11, 2010 10:24 AM, "Ayende" <notifications@codeplex.com> wrote:
> From: Ayende
>
> Standard C# project with the asms as <content> should work
>
> On Sat, Sep 11, 2010 at 6:09 PM, erichexter <notifications@codeplex.com> wrote:
> From: erichexter A custom project type is a option of last resort.  Having a custom project type means that in order to load a solution the project type needs to be installed on the devs system.  For this project to be successful, it has to be non invassive and I think a custom project would not be able to do this, as far aas I know.I have used other project types to do this before and played with the build configurations, but thoses were hacks.I am not sure what the right thing to do is, but it needs to leave nupack as an optional install for the devas working on the project. Read the full discussion online. To add a post to this discussion, reply to this email (nupack@discussions.codeplex.com) To start a new discussion for this project, email nupack@discussions.codeplex.com You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com. Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com
>
>
>

Editor
Sep 11, 2010 at 5:16 PM
Still not clear why this is a big problem.

--
Scott Hanselman - Tiny keyboard, tiny email - Sent from some kind of phone

On Sep 11, 2010, at 1:06 AM, "dfowler" <notifications@codeplex.com> wrote:

From: dfowler

Solution folders are bad to use for packages since they don't actually map to physical folders on disk. Right now we're faking it by mimicking the structure on disk in the solution. Very early in the project we talked about having a special package project type so we can add the actual files and folders without having to fake it all. I think this needs to be done sooner than later.

Another problem I'm having right now is testing the package manager against source control (TFS2010) . Since the solution folders under the packages aren't mapped to any physical folder, deleting them won't remove them from source control. After removing a package that's under a source control system like TFS you'll be left with empty folders.

Thoughts on this?

Sep 11, 2010 at 5:42 PM

I don't understand what the problem is either? You say solution folders are bad because they don't map to physical folders. Why is this a problem? If you delete a solution folder it's reflected in the .sln file which is part of source control (unless you want them to delete the physical folder which by design it doesn't).

Developer
Sep 11, 2010 at 5:48 PM

I'm saying the experience is broken with solution folders and the built in TFS integration which everyone here might not use, but we definitely can't ignore it. 

This is what happens:

  • Create new project that is under source control
  • Add log4net package
  • Check in
  • Remove log4net package
  • Check in

The result of the above actions is files from the log4net package are removed but folders all remain in source control, which is unexpected.

Sep 11, 2010 at 5:49 PM

I may be guessing here, but the only issue should be syncronizing the vs sln file and the actual file system. I think the same problem would exist with a project file.

Developer
Sep 11, 2010 at 5:50 PM

Nope, it's different when the file is in the project, that just works. Solution folders don't.

Sep 11, 2010 at 6:01 PM

A while back, we hand tested using a regular library project and that seemed promising.  If you turn off all the build actions and mark everything as content, it was basically acting as a file holder, with all the right behavior when it comes to source control.

One thing we might lose from Solution folders is the ability to hide it.  In most cases, the solution files are better hidden from the user (too much distraction).  With solution level folders, they can be hidden, but I'm not seeing a way to do this with projects.

In any case, I think this is not high priority enough to try to change it for the initial public release.  It could be disruptive and cause all kind of regressions before we get it right.

Editor
Sep 11, 2010 at 6:02 PM
Seems reasonable that folks have to tidy up after remove. I really don't like the idea of a stray project. Nupack seems ideal for Solution folders. Otherwise they are wholly useless.

--
Scott Hanselman - Tiny keyboard, tiny email - Sent from some kind of phone

On Sep 11, 2010, at 9:49 AM, "dfowler" <notifications@codeplex.com> wrote:

From: dfowler

I'm saying the experience is broken with solution folders and the built in TFS integration which everyone here might not use, but we definitely can't ignore it.

This is what happens:

  • Create new project that is under source control
  • Add log4net package
  • Check in
  • Remove log4net package
  • Check in

The result of the above actions is files from the log4net package are removed but folders all remain in source control, which is unexpected.

Developer
Sep 11, 2010 at 6:06 PM

So what are we saying? Leave it the way it is?

Sep 11, 2010 at 6:08 PM
shanselman wrote:
Seems reasonable that folks have to tidy up after remove. I really don't like the idea of a stray project. Nupack seems ideal for Solution folders. Otherwise they are wholly useless.

Actually, solution folders can be quite useful, but this is not at all the kind of scenarios for which they were designed.  Solution folders are great to gather a selected set of files in a logical structure that may be widely different from the file system.  But they are awful when trying to exactly mimic the underlying file system and stay in sync with it.

Anyway, short term we're staying with them to avoid disrupting things. Later, we re-evaluate our options.

Editor
Sep 11, 2010 at 6:10 PM
You're saying folks would need to delete some folders, right? Why change/mess up the whole experience and solution structure to save an obvious and easy step?

--
Scott Hanselman - Tiny keyboard, tiny email - Sent from some kind of phone

On Sep 11, 2010, at 10:06 AM, "dfowler" <notifications@codeplex.com> wrote:

From: dfowler

So what are we saying? Leave it the way it is?

Developer
Sep 11, 2010 at 6:12 PM

I'm just saying that you get 2 experiences, with the TFS integration it's broken and you have to go tf delete stuff manually. Without source control, everything works as you would expect. 

Editor
Sep 11, 2010 at 6:15 PM
Just TFS or *any* SCC Integration? Visual SVN?

On Sep 11, 2010, at 10:12 AM, "dfowler" <notifications@codeplex.com> wrote:

From: dfowler

I'm just saying that you get 2 experiences, with the TFS integration it's broken and you have to go tf delete stuff manually. Without source control, everything works as you would expect.

Sep 11, 2010 at 6:17 PM
Edited Sep 11, 2010 at 6:18 PM

I wouldn't necessarily say that "Without source control, everything works as you would expect".  It all depends on user expectations.  I think a lot of users will expect that if they delete a package folder in VS, they're actually deleting it for good.  They would then be puzzled to realize that it's still on disk.  That's why solution folders are bad here when trying to exactly match the file system.  It just doesn't work that way.

Developer
Sep 11, 2010 at 6:18 PM

I believe just TFS. I haven't tried anything else other than hg.

Instead of going back and forth on this thread I think we should all go try it and see how it feels (that's what I did before starting the thread). It's easy to argue about something when we haven't tried it ourselves. Everyone pick their favorite source control plugin and see how nupack works against it :).

Developer
Sep 11, 2010 at 6:22 PM
davidebb wrote:

I wouldn't necessarily say that "Without source control, everything works as you would expect".  It all depends on user expectations.  I think a lot of users will expect that if they delete a package folder in VS, they're actually deleting it for good.  They would then be puzzled to realize that it's still on disk.  That's why solution folders are bad here when trying to exactly match the file system.  It just doesn't work that way.

The fact is the files are all gone but folders left in source control (not on disk). So you'll see everything gone and then do a tf get and they'll come back. But like I said, try it out for yourselves.

Developer
Sep 11, 2010 at 6:34 PM

Also it's not really about deleting a solution folder, it's about the what "removing a package with nupack" means. To me it means please remove everything that came with this package, not delete the solution folders in VS and I'll do the ones on disk.

Sep 11, 2010 at 6:50 PM
How do you do the removal? If from Remove-Package, shouldn't the cmdlet handle that?

On Sat, Sep 11, 2010 at 7:48 PM, dfowler <notifications@codeplex.com> wrote:

From: dfowler

I'm saying the experience is broken with solution folders and the built in TFS integration which everyone here might not use, but we definitely can't ignore it. 

This is what happens:

  • Create new project that is under source control
  • Add log4net package
  • Check in
  • Remove log4net package
  • Check in

The result of the above actions is files from the log4net package are removed but folders all remain in source control, which is unexpected.

Read the full discussion online.

To add a post to this discussion, reply to this email (nupack@discussions.codeplex.com)

To start a new discussion for this project, email nupack@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com


Sep 11, 2010 at 6:52 PM
I would rephrase that, without TFS source control, you get the right experience.
SVN, Git, Hg just works.

On Sat, Sep 11, 2010 at 8:12 PM, dfowler <notifications@codeplex.com> wrote:

From: dfowler

I'm just saying that you get 2 experiences, with the TFS integration it's broken and you have to go tf delete stuff manually. Without source control, everything works as you would expect. 

Read the full discussion online.

To add a post to this discussion, reply to this email (nupack@discussions.codeplex.com)

To start a new discussion for this project, email nupack@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com


Sep 11, 2010 at 6:53 PM
I fail to see the problem here.
Remove-Package should:
* Remove the references
* Remove any deps not referenced by something else
* Remove the files from the disk (including folders)
* Remove solution files.

On Sat, Sep 11, 2010 at 8:34 PM, dfowler <notifications@codeplex.com> wrote:

From: dfowler

Also it's not really about deleting a solution folder, it's about the what "removing a package with nupack" means. To me it means please remove everything that came with this package, not delete the solution folders in VS and I'll do the ones on disk.

Read the full discussion online.

To add a post to this discussion, reply to this email (nupack@discussions.codeplex.com)

To start a new discussion for this project, email nupack@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com


Editor
Sep 11, 2010 at 7:08 PM
Removing a package means tidying up my local disk, not cleaning up my source control.

--
Scott Hanselman - Tiny keyboard, tiny email - Sent from some kind of phone

On Sep 11, 2010, at 10:35 AM, "dfowler" <notifications@codeplex.com> wrote:

From: dfowler

Also it's not really about deleting a solution folder, it's about the what "removing a package with nupack" means. To me it means please remove everything that came with this package, not delete the solution folders in VS and I'll do the ones on disk.

Editor
Sep 11, 2010 at 7:08 PM
I agree with Ayende. Does it not do this already?

--
Scott Hanselman - Tiny keyboard, tiny email - Sent from some kind of phone

On Sep 11, 2010, at 10:53 AM, "Ayende" <notifications@codeplex.com> wrote:

From: Ayende

I fail to see the problem here.
Remove-Package should:
* Remove the references
* Remove any deps not referenced by something else
* Remove the files from the disk (including folders)
* Remove solution files.

On Sat, Sep 11, 2010 at 8:34 PM, dfowler <notifications@codeplex.com> wrote:

From: dfowler

Also it's not really about deleting a solution folder, it's about the what "removing a package with nupack" means. To me it means please remove everything that came with this package, not delete the solution folders in VS and I'll do the ones on disk.

Read the full discussion online.

To add a post to this discussion, reply to this email (nupack@discussions.codeplex.com)

To start a new discussion for this project, email nupack@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com


Developer
Sep 11, 2010 at 7:20 PM
Everything I'm talking about is tfs not general source control. I know hg and git will just work. Tfs locks files so if they aren't checked out or unless we remove the read only flag, It'll fail to delete the file from disk if it's under a solution folder. Normally when going through dte on a regular project we are shielded from this since deleting the file from the project will delete it on disk as well and all of the TFS source control things just work. I think I've described what happens earlier I'm this thread the next only way to see what happens is to go try it out on different source control plugins. I'm trying to approach this as a brand new person wanting to try out nupack and I found the behavior to be unexpected and that's why I brought it up.
Sep 11, 2010 at 7:38 PM
We may need to have special code to take care of TFS behavior.
I am pretty sure that my approach would work, but worst case scenario, DTE should also provide a way to access the SCC API, no?

On Sat, Sep 11, 2010 at 9:21 PM, dfowler <notifications@codeplex.com> wrote:

From: dfowler

Everything I'm talking about is tfs not general source control. I know hg and git will just work. Tfs locks files so if they aren't checked out or unless we remove the read only flag, It'll fail to delete the file from disk if it's under a solution folder. Normally when going through dte on a regular project we are shielded from this since deleting the file from the project will delete it on disk as well and all of the TFS source control things just work. I think I've described what happens earlier I'm this thread the next only way to see what happens is to go try it out on different source control plugins. I'm trying to approach this as a brand new person wanting to try out nupack and I found the behavior to be unexpected and that's why I brought it up.

Read the full discussion online.

To add a post to this discussion, reply to this email (nupack@discussions.codeplex.com)

To start a new discussion for this project, email nupack@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com


Sep 11, 2010 at 7:39 PM

Yes, we really need to step back and look at what started this thread.  The behavior with TFS is clearly not quite correct today.  I suggest that everyone tries playing with the product a bit (with TFS or other systems) to get a feel about how it all works.  That'll help get everyone on the same page when discussing these issues :)

Developer
Sep 11, 2010 at 8:10 PM

Trust me I spent some of last night looking for ways to do this cleanly. We check out files using the source control provider registered within dte, and all of this works but the problem i mentioned above still remains. If you have a better idea of what specific API calls we should be making definitely let me know :).

Jan 20, 2011 at 6:21 PM


In Dev11, we’re implementing local workspaces for TFS which will behave very similarly to SVN, HG, etc. Rather than tracking your workspace’s contents on the server, as we do now, we’ll track it by scanning your local disk. This won’t scale to DevDiv-sized enlistments but it will perform just fine for the vast majority of our customers. Server-based workspaces will still be an option, but new workspaces will be local by default.

In a local workspace, you make changes to your local file system and we calculate the changes that need to be committed to the repository when go to check-in. That should alleviate the problems described previously on this thread that are specific to TFS version control. I'll ask the version control team to include NuGet in their testing.

I still like the idea of creating a simple project system to manage the packages. It could use an “Include” to pull in all of the package content in its related folder. It could even implement build and clean targets where, based on how you configured it, it would fetch the latest (or some specified) version of each package on build. It would be pretty slick if you could automatically check for updates to your dependent packages every time you did a “release” build, for example.

For server builds, you could imagine bootstrapping the packages on the first build, then checking for updates (if directed by the configuration) on subsequent, incremental builds. If you get a build failure on a new version, you tweak your configuration to “pin” that package to the known good version. Otherwise, the package content is effectively "invisible" to Visual Studio users who don't go spelunking into their solution directories.

Jan 20, 2011 at 7:00 PM
Thank you for local workspaces.
____
Rob
"Be passionate in all you do"

http://devlicio.us/blogs/rob_reynolds
http://ferventcoder.com
http://twitter.com/ferventcoder
Coordinator
Jan 20, 2011 at 8:37 PM

This is interesting. Could it work with our existing folder structure? So it’d be an “overlay” on top of what we have? That might allow it to degrade gracefully. For example, if someone doesn’t have NuGet installed, they should still be able to build the solution.


Phil

Jan 20, 2011 at 8:58 PM

If I create a solution in VS2010 with a console application and a class library, then unload the class library project, the console application still builds fine without any warnings or errors. Even if I define a solution dependency from the console application to the class library, then unload the class library, the console application still builds successfully without any warnings or errors. You would want some way to get the NuGet project to build first in your overall solution build order and solution dependencies would facilitate that - apparently without precluding the approach from degrading gracefully if the project couldn't be loaded.