Extension SDK support

Topics: Ecosystem
Sep 24, 2012 at 7:57 PM
Edited Sep 24, 2012 at 8:13 PM

I wanted to propose that NuGet support distributing WinRT Extension SDKs.

Rather than rehash many different discussions, I'd like to point to my blog post: http://novotny.org/blog/how-to-use-extension-sdks-per-project to show how it can be done.

Here's the main arguments

  • Extension SDKs currently need an installer. They are installed globally per machine or per user.
  • Installers make using Extension SDKs on build servers difficult as the VSIX package needs to be installed there too
  • Many corporations will not allow installed packages
  • Many corporations require that binary artifacts are version controlled for auditing. Referencing items outside the version control tree is forbidden.
  • In open source projects, developers need to manually find and install all needed VSIX packages.

It is possible to have Extension SDKs "per-project". The key is to override the SDKReferenceDirectoryRoot variable similar to this:

  <PropertyGroup>
    <SDKReferenceDirectoryRoot>$(SolutionDir)\packages;$(SDKReferenceDirectoryRoot)</SDKReferenceDirectoryRoot>
  </PropertyGroup>

The important/difficult bit is that Extension SDKs need a specific folder structure that differs from what NuGet supports so far. Within Packages, the root path would need to be "Microsoft SDKs\Windows\v8.0\ExtensionSDKs", with the Extension SDK folder structure underneath that.

My proposal would be that within a NuPkg, there be an "ExtensionSDK" folder with everything needed there. Then, during NuGet's unpacking, it can extract it to the right location instead of being next to the rest of the folders (net4, sl5, etc).

NuGet would modify the csproj/vbproj file to prepend the custom ExtensionSDK location to the variable as I described above. Then, it'll add the <SDKReference... /> element.

This would be a huge win that would enable Extension SDKs to be more easily distributed and updated, complete with package dependencies.

Sep 24, 2012 at 9:13 PM

LOL.

I'm actually working on exactly this right now, in pretty much exactly this way (in order to support Native libraries in Nuget).  

If I do this right, It's entirely possible that the nuget package file itself could also function as a .VSIX if you changed the extension.

My implementation should work for native libraries--something that apparently the Extension SDKs *could* work for, but non-RT apps apparently don't show any UI for that. Regardless, I'm hoping to make it work correctly regardless of the consuming project type.

 



Garrett Serack
Microsoft Senior Open Source Software Developer  

 

Sep 24, 2012 at 11:05 PM

That's great to hear. If there's anything I can do to help (not that I have a ton of extra time), please let me know.

Sep 25, 2012 at 12:32 AM

Stay Tuned.

I should hopefully have some bits you can test with by sometime next week.

G

Sep 25, 2012 at 8:37 PM

Dammit. 

I was told that VS2012 could *use* Extension SDKs from native projects, but the UI just didn't support them (so you'd have to actually insert the reference to the SDK manually).

So I did that in a native app, and it went looking for it, but didn't do more than poke at the SDKManifest.xml file, it didn't do anything else with it at all.
I manually added the same reference to a WIn8StoreApp, and low and behold, it did what I was expecting.

Which means, that the Extension SDK pattern is useless for everything other than the Win8StoreApps. 

*sigh*

And considering that the pattern is somewhat half-baked, I'm not sure that 'emulating' (by discovering the .props locations and wiring up the  it is the right approach. 

/thinking... 

Sep 25, 2012 at 9:50 PM
Edited Sep 25, 2012 at 9:55 PM

I found that Extension SDKs *do* work for regular managed libraries (though there's no UI)!

Here's the end of a regular .net csproj file:

  
  <ItemGroup>
    <SDKReference Include="XUnit, Version=2.0">
      <Name>XUnit for Managed Projects</Name>
    </SDKReference>
  </ItemGroup>
<PropertyGroup Condition=" '$(VisualStudioVersion)' == '' or '$(VisualStudioVersion)' &lt; '11.0' ">
    <VisualStudioVersion>11.0</VisualStudioVersion>
  </PropertyGroup>
  <PropertyGroup>
    <TargetFrameworkVersion Condition="'$(TargetFrameworkVersion)' == ''">v4.5</TargetFrameworkVersion>
    <TargetPlatformVersion Condition="'$(TargetPlatformVersion)' == ''">8.0</TargetPlatformVersion>
    <ImplicitlyExpandTargetPlatform Condition="'$(ImplicitlyExpandTargetPlatform)' == '' ">true</ImplicitlyExpandTargetPlatform>
    <ImplicitlyExpandTargetFramework Condition="'$(ImplicitlyExpandTargetFramework)' == '' ">true</ImplicitlyExpandTargetFramework>
  </PropertyGroup>
  <Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />
  <PropertyGroup>
    <SDKReferenceDirectoryRoot>$(ProjectDir)\libs;$(SDKReferenceDirectoryRoot)</SDKReferenceDirectoryRoot>
  </PropertyGroup>

I tested it using the Xunit extension SDK I created and it works... :)

I'm not familiar with the native project system though.

 I found that the relevant variables are in the targets normally included for Store apps here:

C:\Program Files (x86)\MSBuild\Microsoft\WindowsXaml\v11.0

The trick is to just get the SDK-related variables without all of the WinRT/Store gunk.

 

Developer
Sep 26, 2012 at 2:58 PM
Edited Sep 26, 2012 at 2:59 PM

Even if we can't get native projects to work, it's still a big bonus to support C++ Windows Store apps.

Sep 26, 2012 at 4:30 PM

The biggest irritant at this point is the excessive folder nesting required to support VS2012's ExtensionSDKs.

On disk extension has to be in a folder that looks like this:

<root>\Windows\v8.0\ExtensionSDKs\<sdkname>\<sdkversion>

So that the <root> folder can be added to the project's <SdkReferenceRoot> element. 

Which means that as we unpack the nupkg files into .\packages it should go into a location like:

...\packages\
      <pkgname-1>\Windows\v8.0\ExtensionSDKs\<sdkname>\<sdkversion>\<actual content goes here>
      <pkgname-2>\Windows\v8.0\ExtensionSDKs\<sdkname>\<sdkversion>\<actual content goes here>
      <pkgname-3>\Windows\v8.0\ExtensionSDKs\<sdkname>\<sdkversion>\<actual content goes here>
 

and then we still have  to add ...\packages\<pkgname-1>;...\packages\<pkgname-2>;...\packages\<pkgname-3> to the <SdkReferenceRoot>, just to get them all included.

I really detest that excessive nesting for no productive purpose. it just increases the complexity in a way that you *know* is gonna come back and bite us in the behind later. What happens with Windows 7, 9, 10 or 11? What about SDKs that work on multiple OSes?.  I'm really not happy about the Extension SDK model at all anymore.  I was thinking that it was implemented in a moderately convenient fashion, but it's just crap.

Sep 26, 2012 at 4:37 PM

My thought was to actually share a single path for the extension SDK's instead of having to manage each one separately. Instead of \packages\<pkgname>\windows\...

I'd have a single \packages\windows\v8.0\ExtensionSDKs folder. Each Extension SDK name has to be unique anyway. I believe it has to match the identity in the SDKManifest.xml, but I'm not sure.

Given that each one has to be unique, I was thinking that NuGet would just extract them all to there. As the folder structure already allows for multiple versions of the same SDK, that's taken care of already.

For packaging, you could embed the VSIX in the NuPkg, then read the metadata from the VSIX to identify the sdkname and sdkversion. That'd determine the extraction path for both adding/removing references.

Then you'd also only need to modify the SDKReferenceDirectoryRoot once and it'd be the same for all projects in the solution. Seems much easier that way.

Sep 26, 2012 at 5:10 PM

That kinda diverges from the nuget model quite a lot. (embedding a whole .vsix) ... really, if you're hoping to track the .vsix with the project anyway, you can just check that in, and unzip it with a single script. Significantly changing the way Nuget works to support a crippled model isn't high on my list of goals.

And we're still left with the incredibly foolish "\Windows\v8.0\" part in the middle. I can't for the life of me figure out why they thought that was a good idea. That's just extremely short-term thinking.

Frankly, we'd all be better off if people producing .vsix packages just produced .nupkg files instead. Encouraging that format is ... counterproductive. The docs are kinda spotty, there isn't any tooling to make it easier, and they don't solve things any better than nuget anyway.

Heck, it probably wouldn't even be terribly difficult to build a converter to make a .nupkg out of a .vsix ExtensionSDK. 

G

Sep 26, 2012 at 5:26 PM

I was just thinking that the project templates make it somewhat easy to produce a VSIX currently. There's no need for that other than having to deal with the directory structure.

That said, each package could have an \esdk\Windows\v8.0\... folder. Then NuGet would copy it to the shared one to make it easier to deal with the csproj files.

I do believe there's a reason why the platform and version moniker are in the directory structure. I don't think it's something that NuGet needs to worry about though. It just needs to handle the concept of the extension sdk root and addind the <SDKReference... />, then let msbuild/vs handle the rest.  I don't know for sure, but I'd think that Phone may be different. It could very well be \WindowsPhone\v8.0\... The logic within MSBuild would do the right thing. That free's NuGet from having to parse out platform identifiers and versions. Just copy the files into \packages\ESDKs\<blah>\<blah> and set the SDKReferenceRoot to include \packages\ESDKs.

Seems pretty easy.

Sep 26, 2012 at 5:44 PM

It's not terribly difficult on the surface, but they didn't solve enough to address *my* scenarios.

I'm *still* going to have to deal with variants of a given library based on

OS {Windows 5/6/7/8, 8RT, Phone (7/8), Xbox, any...others?}
Arch {x86, x64, ARM, Any}
  Runtime { msvcrt, VC7, VC7.1, VC8, VC9, VC10, VC11, GCC}
Binding { static, dynamic, SxS}
Flavor { thread-safe, non-thread-safe, etc...} 

At this point, to do any work to supporting the ExtensionSDK explicitly model is ... just seems like wasting  time supporting an incomplete model.  Like I said, I'd rather have a pkg converter than to bother inserting code into Nuget to support those directly (since there likely won't be that many ExtensionSDKs anyway).

G

Sep 26, 2012 at 5:50 PM

Seems like there's a lot of extension sdk's as it's the only way to reference WinRT libraries that uses C++.

Here's a couple examples:

  • LiveConnect
  • Bing Maps
  • SQLite
  • <some game engine?>
  • etc.
  • XUnit

My original request had to do with those rather than just working as a native project reference mechanism. While I don't disagree that references, especially with three architectures, is painful, I do believe there's still a need to manage the WinRT Extension SDKs as they were intended to be used. I also believe that NuGet is the best place to manage those and it already has the capabilities to do so.

You cannot convert Extension SDKs into regular NuGet packages. They cannot be referenced that way by the project system. There are also other benefits to Extension SDKs, namely that they allow you to have a single "reference" that actually has a collection of libraries in them. It "packagifies" references so you don't need to reference individual libraries anymore.

 

Sep 27, 2012 at 5:03 PM

I met with the Visual Studio folks yesterday, and we worked out some methods to compromise and support additional scenarios while still supporting the ExtensionSDK model.

I'm back on it today experimenting with extending the ExtensionSDK layout and slipping in a bunch of UI.  

 

G

Sep 27, 2012 at 5:08 PM

Awesome, great to hear!

Oct 1, 2012 at 6:54 PM

As a developer of the Microsoft Media Platform Player Framework, which is an Extension SDK, we are investigating if it would be valuable to also provide a NuGet package.

From this discussion, it sounds like it is not yet possible to create a NuGet package that depends on an Extension SDK (i.e. packages a VSIX, installing it if necessary, and then adds the correct references to your project), but that this functionality is currently in development.

Shipping both a VSIX and a NuGet package seems to have little value at the moment, because of headache it introduces in maintaining two very different packages and explaining to users why and how to use one vs. the other.

Am I misunderstanding any of this? Is there still a good reason to ship both? Your thoughts appreciated.

Thanks!

Oct 2, 2012 at 4:24 PM
Edited Oct 2, 2012 at 4:24 PM

I've posted the spec for the changes at http://sdrv.ms/Qx0A17 .

@billyzkid -> hold off on doing anything yet. Once I've added in the ExtensionSDK functionality, it should be trivial to generate the Nuget package from the same files.

I did add a P2 item to make a .VSIX to Nuget converter, since, really that should be trivial as heck.

Once we've got this code in Nuget, you'll probably want to ship both until Nuget supports package installation somewhere other than just the Solution. (Which, I believe is on the roadmap, coming soon) -- Once that's done, you'll really want to drop the VSIX. (since 100% of the ExtensionSDK scenarios will be supported thru Nuget)

 

Oct 2, 2012 at 4:59 PM

Great, thanks! We will hold off on NuGet for now and await the changes you mentioned.

Oct 2, 2012 at 5:43 PM

Updated spec for Nuget Native Packages/ExtensionSDKs : http://sdrv.ms/Qx0A17 (modified from feedback)

I found out that the target location stuff is already in the latest branch, so... that will be included with my changes.  With a little luck, we should see this stuff by the end of the month :D

 

G

 

Oct 12, 2012 at 5:22 PM

What would the rules be if a package contained an ExtensionSDK and a \winrt45 folder?

The package might have both, with the ExtensionSDK stuff there for WinJS consumers but with a better one for .NET. I would think that the mis-named WinRT folder would win (it really should be called NetCore), as that'd be "more specific" and "closer" to the library than a more generic SDK.

Thoughts?

Feb 28, 2013 at 11:09 PM
Hi all, just checking in to see if the extension sdk feature being discussed above was completed?

Thanks!
Developer
Feb 28, 2013 at 11:11 PM
No, we have decided not to do this feature.
Apr 8, 2013 at 5:39 PM
I'm currently trying to work out how to distribute some assemblies that have a dependency on SQLite.

The assemblies are all called: Cirrious.MvvmCross.Plugins.Sqlite.WinRT.dll - but there are 3 separate assemblies - one for x86, one for x64 and one for ARM

Because of this platform specific dependency I need to insert these dependencies as assembly references using code like:
  <Choose>
    <When Condition=" '$(Platform)' == 'ARM' ">
      <ItemGroup>
        <Reference Include="Cirrious.MvvmCross.Plugins.Sqlite.WinRT.dll, Version=1.0.0.0, Culture=neutral, processorArchitecture=MSIL">
          <SpecificVersion>False</SpecificVersion>
          <HintPath>..\..\libs\MvX\WinRT\Release\ARM\Cirrious.MvvmCross.Plugins.Sqlite.WinRT.dll</HintPath>
        </Reference>
      </ItemGroup>
    </When>
  </Choose>
  <Choose>
    <When Condition=" '$(Platform)' == 'x64' ">
      <ItemGroup>
        <Reference Include="Cirrious.MvvmCross.Plugins.Sqlite.WinRT.dll, Version=1.0.0.0, Culture=neutral, processorArchitecture=MSIL">
          <SpecificVersion>False</SpecificVersion>
          <HintPath>..\..\libs\MvX\WinRT\Release\x64\Cirrious.MvvmCross.Plugins.Sqlite.WinRT.dll</HintPath>
        </Reference>
      </ItemGroup>
    </When>
  </Choose>
  <Choose>
    <When Condition=" '$(Platform)' == 'x86' ">
      <ItemGroup>
        <Reference Include="Cirrious.MvvmCross.Plugins.Sqlite.WinRT.dll, Version=1.0.0.0, Culture=neutral, processorArchitecture=MSIL">
          <SpecificVersion>False</SpecificVersion>
          <HintPath>..\..\libs\MvX\WinRT\Release\x86\Cirrious.MvvmCross.Plugins.Sqlite.WinRT.dll</HintPath>
        </Reference>
      </ItemGroup>
    </When>
  </Choose>
I currently guess I'm going to have to write some PowerShell script to do this?

But I'd love to find a way to do it via nuget directly - e.g. I'd love to be able to ship a nuspec with targets for lib/netcore45-x86, lib/netcore45-x64, and lib/netcore45-ARM

Does this sound feasible to anyone? I'm likely to have some time next week (I get bored at conferences) so I could try to code something up then....

Or does this sound like a bad idea?

Stuart
Apr 8, 2013 at 5:52 PM
Hey Stuart,

The ExtensionSDK format turned out to be a ... less-than-ideal solution when redistributing libraries, and terribly unfeasible to implement with NuGet.

Instead, NuGet 2.5 will have the ability for packages to include additional msbuild files into projects (as .props or .targets files), and these are picked up in the package in the /build/<framework> directory.

I actually have a powershell cmdlet that is nearly ready that will let you generate all of this with a whole lot less trouble than you're having to go thru to make the file that you're working on.

I've done a huge amount of refactoring in the last week to support specifically this type of scenario (and now, I'm into the rather lengthy debugging phase. )

The work that I'm primarily doing this for is to support Native C/C++ packages with NuGet, but the tool should really be able to support this as well.

G
Apr 8, 2013 at 5:56 PM
Garrett,

How will that tool work with the other ExtensionSDK features like Release/Debug files, CommonConfiguration and build-time/vs redist files?

For WinRT custom controls, they need to have loose xaml with the related files/images all in there, will this work for those too?
Apr 8, 2013 at 6:00 PM
Thanks fearthecowboy

If you've got a ps file that I can use then I'd be very happy :) Or I can wait for 2.5 to go GA.

I'm planning to demo some nuget next week at the Xamarin conference - showing it working for sqlite for monodroid and monotouch.... so would be nice if it worked with Windows Store too. For the demo I can probably just hide the details - I can even probably just ship x86 and hope no-one notices, but if there's any drop of a real PS file or real Nuget code that I can have a play with then that would be great - as I do plan to ship Mvx as soon as NuGet 2.5 hits live.

Thanks again - great news that there will be something in 2.5 to support these processor-specific assemblies like SQLite, maps, etc

Stuart
Apr 8, 2013 at 6:39 PM
onovotny wrote:
Garrett,

How will that tool work with the other ExtensionSDK features like Release/Debug files, CommonConfiguration and build-time/vs redist files?
The .props and .targets files that get generated support having conditional statements around virtually everything, and the conditions can be on absolutely any property that's built into MSBuild (like $(Platform), $(PlatformToolset), $(Configuration), etc ) as well as any other value that you want to pivot upon. In the c/c++ case this tends to be stuff like Linkage (ie, Static, Dynamic, SxS ...) and sometimes CallingConventions (cdecl, stdcall, etc...)

The packager can merely declare what a particular 'pivot' is , and the possible values, and the generator takes care of all the messy Condition=".." statements and ensuring that the property defaults to a usable value.

We use these features to emit <copy> tasks in AfterBuild targets that ensure that the appropriate files get delivered to the right folder. (In the case of native packages, these rules are automatically inferred based on what the files are (ie, bin files are copied into the $(TargetDir) whereas lib files are automatically linked with the target executable) ... but, this is entirely configurable, and all of the implicit rules can be turned off trivially.

SNEAK PEEK

This is a sample of a .autopkg file (this is pretty much what the zlib script looks like) ... it doesn't show how to do explicit tasks (copyies, etc) but it does show how we treat conditionals.

you can see how the different conditional pivots are declared down in the next file (I'm just showing the defaults in that one.)
@import "version.inc";

nuget {
 // the nuspec file metadata. 
    nuspec {
        id = zlib;
        version : ${package-version};
        title: ZLib compression library;
        authors: { "Jean-loup Gailly", "Mark Adler", "Garrett Serack", "Tim Rogers" };
        owners: Garrett Serack;
        licenseUrl: "http://zlib.net/zlib_license.html";
        projectUrl: "http://github.com/coapp-packages/zlib";
        iconUrl: "http://zlib.net/images/zlib3d-b1.png";
        requireLicenseAcceptance:false;
        summary:A zlib library;
        description: @"A native zlib library.
    zlib homepage:  http://zlib.net";
        releaseNotes: "Release of zlib 1.2.7 libraries.";
        copyright: Copyright 2013;
        tags: { zlib, native, coapp,  };
    };
  
    /* the files that go into the content folders
       (inserted into the nuspec file) */
    files {
        include: { ..\zlib.h, ..\zconf.h };

        docs: {  ..\doc\**\* };
    
        [Win32,v110,dynamic] { /* <--- the stuff in the [] are 'conditions' that we expand out to match things specified in the configurations section */
            lib: { ..\output\vc11\x86\release\lib\zlib1.lib };
            bin: { ..\output\vc11\x86\release\bin\zlib1.dll };
        }

        [x64,v110,dynamic] {
            lib: { ..\output\vc11\x64\release\lib\zlib1.lib };
            bin: { ..\output\vc11\x64\release\bin\zlib1.dll };
        }
        
        [Win32,v110,static] {
            lib: { ..\output\vc11\x86\release\lib\zlibstat.lib };
        }

        [x64,v110,static] {
            lib: { ..\output\vc11\x64\release\lib\zlibstat.lib };
        }

        [Win32,v110,stdcall] {
            lib: { ..\output\vc11\x86\release\lib\zlibwapi.lib };
            bin: { ..\output\vc11\x86\release\bin\zlibwapi.dll };
        }

        [x64,v110,stdcall] {
            lib: { ..\output\vc11\x64\release\lib\zlibwapi.lib };
            bin: { ..\output\vc11\x64\release\bin\zlibwapi.dll };
        }

        [Win32,v100,dynamic] { 
            lib: { ..\output\vc10\x86\release\lib\zlib1.lib };
            bin: { ..\output\vc10\x86\release\bin\zlib1.dll };
        }

        [x64,v100,dynamic] {
            lib: { ..\output\vc10\x64\release\lib\zlib1.lib };
            bin: { ..\output\vc10\x64\release\bin\zlib1.dll };
        }
        
        [Win32,v100,static] {
            lib: { ..\output\vc10\x86\release\lib\zlibstat.lib };
        }

        [x64,v100,static] {
            lib: { ..\output\vc10\x64\release\lib\zlibstat.lib };
        }

        [Win32,v100,stdcall] {
            lib: { ..\output\vc10\x86\release\lib\zlibwapi.lib };
            bin: { ..\output\vc10\x86\release\bin\zlibwapi.dll };
        }

        [x64,v100,stdcall] {
            lib: { ..\output\vc10\x64\release\lib\zlibwapi.lib };
            bin: { ..\output\vc10\x64\release\bin\zlibwapi.dll };
        }
    };

    /* the VC++ .targets file that gets generated and inserted into the ${d_build} folder
    [native] <-- we assume 'native' if not specified. */
    targets {
        // stuff in this node gets put into the zlib  package as a /build/native/zlib.targets file 
        //  things in these nodes can just specify MSBuild objects, or use the simpler aliases that we've provided: 
         
       // this is the same as the next line*/
        ItemDefinitionGroup.ClCompile.AdditionalIncludeDirectories =  += ${pkg_root}${d_include}; 

        Includes += ${pkg_root}${d_include};

        Defines += HAS_ZLIB;
        
        [static]
        Defines += ZLIB_WINAPI;
        
        [stdcall]
        Defines += ZLIB_WINAPI;

    };

    /* the VC++ .props file that gets generated and inserted into the ${d_build} folder */
    /* [native] */
    props { 
        /* stuff in this node gets put into the zlib  package as a /build/native/zlib.props file
           we're going to generate a number of properties anyway. 
           so even if the packager doesn't specify the props object 
           we're going to make this file. */
    };

    [redist]
    targets {
        /* stuff in this node gets put into the zlib.redist package as a /build/native/zlib.redist.targets file */
    }
    
    [redist]
    props {
        /* stuff in this node gets put into the zlib.redist package as a /build/native/zlib.redist.props file */
    }
    
    
    [redist]
    props {
        /* stuff in this node gets put into the zlib.redist package as a /build/native/zlib.redist.props file */
    }
    
    [net40]  /* these conditions determine the framework and package that the file gets generated into (in this case, default package, net40 framework */
    targets {
        /* stuff in this node gets put into the zlib.redist package as a /build/net40/zlib.targets file */
    }
    
}
This is part of the default template that gets applied to every package file (we predefine the conditions, but you can add/change/override them completely)
You generally don't need to do this.
configurations {
    Toolset { 
        key : "PlatformToolset";    // specify a 'key' when it's a built-in property
        choices: { v110, v100, v90, v80, v71, v70, v60, gcc };  
    };
    
    Platform {
        key : "Platform"; 
        choices: { Win32, x64, ARM, AnyCPU };
        Win32.aliases : { x86, win32, ia32, 386 };
        x64.aliases : { x64, amd64, em64t, intel64, x86-64, x64_64 };
        ARM.aliases : { arm, woa };
        AnyCPU.aliases : { anycpu, any };
    };

    Configuration {
        key : "Configuration"; 
        choices: { Release, Debug };
    };

    Linkage { 
        choices : { dynamic, static, ltcg, sxs };  // no 'key' when it's a not built-in property -- this generates a ton of msbuild code behind the scenes.
        description = "Which version of the .lib file to link to this library";

        ltcg.description = "Link Time Compiler Generation";
        dynamic.description = "Dynamic Library (DLL)";
        static.description = "Static";
        sxs.description = "Side-by-Side";
    };

    // Only really applicable to x86
    CallingConvention {
        choices : { cdecl, stdcall, fastcall, thiscall, clrcall };
        description = "Calling convention model to use (for x86 only)";
        cdecl.description = "cdecl";
        stdcall.description = "stdcall (Uncommon)";
        fastcall.description = "fastcall (Rare)";
        thiscall.description = "thiscall (Rare)";
        clrcall.description = "clrcall (Rare)";
    };
}
(Hmm, the colorizer doesn't much like my format :D)

That turns around and generates the msbuild scripts (an example of a generated script: http://fearthecowboy.com/stuff/x-2013-04-08-35840001.html )
For WinRT custom controls, they need to have loose xaml with the related files/images all in there, will this work for those too?
I can't imagine that would be at all problematic... What happens with those files (compiled, copied...?)
Apr 8, 2013 at 6:42 PM
stuartlodge wrote:
Thanks fearthecowboy

If you've got a ps file that I can use then I'd be very happy :) Or I can wait for 2.5 to go GA.
It's not actually a ps script, it's all in c# compiled up to be in a .dll

I don't have anything right-this-minute I'm hoping to post a beta in the next day or two. (I'm pretty much just down to debugging,but I did a ton of code reorganization over the last week...)
I'm planning to demo some nuget next week at the Xamarin conference - showing it working for sqlite for monodroid and monotouch.... so would be nice if it worked with Windows Store too. For the demo I can probably just hide the details - I can even probably just ship x86 and hope no-one notices, but if there's any drop of a real PS file or real Nuget code that I can have a play with then that would be great - as I do plan to ship Mvx as soon as NuGet 2.5 hits live.
I'm workin' as fast as I can to get this stuff done. :D
Apr 8, 2013 at 6:45 PM
fearthecowboy wrote:
Hey Stuart,

The ExtensionSDK format turned out to be a ... less-than-ideal solution when redistributing libraries, and terribly unfeasible to implement with NuGet.

Instead, NuGet 2.5 will have the ability for packages to include additional msbuild files into projects (as .props or .targets files), and these are picked up in the package in the /build/<framework> directory.

I actually have a powershell cmdlet that is nearly ready that will let you generate all of this with a whole lot less trouble than you're having to go thru to make the file that you're working on.

I've done a huge amount of refactoring in the last week to support specifically this type of scenario (and now, I'm into the rather lengthy debugging phase. )

The work that I'm primarily doing this for is to support Native C/C++ packages with NuGet, but the tool should really be able to support this as well.

G
Awesome - one annoying part is handling references vs redist. Many packages will have a dll that needs to be referenced by the project but then the redist folder contains a few other things that need to get copied to the output dir.

In the case of the winrt stuff, the dir structure next to the dll need to be xcopied to the output. It may be more than that though as the AppX mechanism merges resources and stuff...not totally sure how all that works.
Apr 8, 2013 at 7:22 PM
onovotny wrote:
Awesome - one annoying part is handling references vs redist. Many packages will have a dll that needs to be referenced by the project but then the redist folder contains a few other things that need to get copied to the output dir.

In the case of the winrt stuff, the dir structure next to the dll need to be xcopied to the output. It may be more than that though as the AppX mechanism merges resources and stuff...not totally sure how all that works.
Yeah, redist files turned out to be a really, really complicated issue in native -- especially when you need to consume something from the parent of a package.

Needless to say, I think we've got that worked out completely.
Apr 17, 2013 at 10:33 PM
Hi!
I do not usually post comments but this looks awesome! I want to try it! Is it possible to try it?
I cannot understand how it is working or what can be done. It is like first time reading about autoconf or macports. :) But the concept looks nifty and feels very powerful. Especially when the practical stuff happens in msbuild .props and .targets files.
What is that that blueprint technique using c# like that called? Never seen it before.

/Roger
Apr 17, 2013 at 10:48 PM
perrog wrote:
Hi!
I do not usually post comments but this looks awesome! I want to try it! Is it possible to try it?
I cannot understand how it is working or what can be done. It is like first time reading about autoconf or macports. :) But the concept looks nifty and feels very powerful. Especially when the practical stuff happens in msbuild .props and .targets files.
What is that that blueprint technique using c# like that called? Never seen it before.

/Roger
Hey @perrog ...

I was actually going to record a couple of screencasts today (as a precursor for the guy who's producing the documentation and tutorials) but I we came across a couple of issues that required my immediate attention.

If you can wait a day or so, I should be able to post something you can play with along with a screencast that kind-of explains how to get started actually building the package.

Once you see the docs/screencast, you'll see how simple it really is, even if the script seems a bit daunting now.

If you've ever had to build msbuild project files by hand, you know how insanely complex they can be, and the tool removes pretty much all of that complexity completely. Essentially, if you have the right output files, you can build a native package in a few minutes.
What is that that blueprint technique using c# like that called? Never seen it before.
I have no doubt that you've not seen it before :D ... It's a data description language called "Property Sheets" -- it originated as a variant of CSS (Cascading 'Style' Sheets) and grew a few more features borrowed from C#.

Property Sheets are far less complicated and error-prone than XML, and have far more complex behavior than YAML or JSON (which are really just data-encoding mechanisms).

Property Sheets are meant to be hand-edited, and have an extremely forgiving parser. (You'll notice that you generally don't need quotes around things that are strings, and if something can be optional, it pretty much is.

Internally, Property Sheets can map to incredibly complex data models (like, the MSBuild Object Model for example...:) ), without ever having to physically alter the original model. (I have a prototype of an MSBuild app that uses Property Sheets instead of MSBuild xml files... it's far less cumbersome, and really exposes the true beauty of MSBuild ... this tool will be making public notice soon too... )

Anyway, I'm really just finishing with one interesting bug, and then I'm back to doing the screencast (which for intermediate to advanced developers, will fill in enough on how to do everything you need to)...

With luck, tomorrow I'll have something for you.
Apr 17, 2013 at 10:50 PM
Oh, and I have a Syntax-highlighting support for Notepad++ for Autopackage property sheets too...
Apr 18, 2013 at 4:07 PM
@fearthecowboy: Thanks! No, no. No need to hurry. I will read blog posts and get more acquainted with NuGet. Even if it is only a sneak peek and not fully implemented yet, I’m excited.
Yes, MSBuild is powerful but a bit tricky and complex. Difficult to know what is happening and XML make it verbose. I get the feeling it is still much Visual studio internal. Changes in targets between VS releases. Custom item meta-data not display in Visual studio UI. I nowadays try to avoid edit the project file. This is my experience. Maybe I’m wrong, but I’ve had many headaches from this. :-)
Interesting to hear about Property sheet. Is it a new front-end to MSBuild you have been working on that generates .cs|vcx|vbproj files?
Apr 18, 2013 at 6:15 PM
perrog wrote:
Yes, MSBuild is powerful but a bit tricky and complex. Difficult to know what is happening and XML make it verbose. I get the feeling it is still much Visual studio internal. Changes in targets between VS releases. Custom item meta-data not display in Visual studio UI. I nowadays try to avoid edit the project file. This is my experience. Maybe I’m wrong, but I’ve had many headaches from this. :-)
It's a near-nightmare, and the UI doesn't help much for doing anything.
Interesting to hear about Property sheet. Is it a new front-end to MSBuild you have been working on that generates .cs|vcx|vbproj files?
Yes, it is actually. One of the tools we're working on is the ability to take an existing build (using nmake, msbuild, cmake, batch files, whatever) and trace it, and then feed the trace data into the tool to generate new Project files from scratch (that aren't cluttered with a bunch of crap).
Apr 19, 2013 at 4:57 PM
FYI, the first tutorial video is up : https://www.youtube.com/watch?v=l4MAkR13JPA (watch the HD stream!)
Apr 19, 2013 at 9:22 PM
Thanks fearthecowboy

In Dallas heading home to London

Will get this used soon :)

On Friday, 19 April 2013, fearthecowboy wrote:

From: fearthecowboy

FYI, the first tutorial video is up : Building Native Libraries for NuGet with CoApp's Powershell Tools (watch the HD stream!)

Read the full discussion online.

To add a post to this discussion, reply to this email ([email removed])

To start a new discussion for this project, email [email removed]

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe on CodePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at CodePlex.com

Apr 20, 2013 at 3:39 PM
Thanks for the tutorial! I will start experimenting with the autopkg tools. Great video and pedagogy by the way!
Apr 23, 2013 at 11:21 PM
We've got a bit more documentation up :

Look for the pages with New! : http://coapp.org/pages/reference.html

and one more here: http://coapp.org/reference/autopackage-guide.html
Apr 27, 2013 at 2:51 PM
Hi! The URL http://coapp.org/reference/autopackage-guide.html seems to be broken.

I played around a bit and it work like a charm. Even if it native packages adds more complexity, it was easy to create my first own NuGet package.
Running Write-NuGetPackage creates three archives: .nupkg, .redist.nupkg and .symbols.nupkg. I noticed .redist.nupkg always created, even if the package only contains c++ header library files.
I push them to a local feed to make them available from visual studio.
Nice to see different platform names are supported (e.g. Win32 vs. x86.)

How does NuGet tool understand to add the .props and .targets file to the project? Are .vcxproj treated special or a directive in the .nupkg? I'm just curious.

I noticed a link error when my project already have AdditionalDependencies (Project properties > Linker > Input > Additional Dependencies.) To reproduce it, I created a ConsoleApp and WinApp and added my package to both of the projects. The ConsoleApp links just fine, but the WinApp with inherited libraries fails to link. From MSBuild logs I can see how inherited libs are gone and replaced with the package .lib file. Have anyone else experience this?

My suspicions goes to the line adding the lib file in the .targets file:
  <ItemDefinitionGroup Label="x64 and Debug" Condition="'$(Platform)' == 'x64' And '$(Configuration)' == 'Debug'">
    <Link>
      <AdditionalDependencies>$(MSBuildThisFileDirectory)..\..\build\native\lib\x64\Debug\Sphere.lib</AdditionalDependencies>
    </Link>
  </ItemDefinitionGroup>
Apr 27, 2013 at 4:58 PM
Update to the latest version of the tools :
update-coapptools -KillPowershells
will download the latest version and update.

The AdditionalDependencies regression is fixed in the latest release.

As is the bug creating .redist and .symbols when not needed.

NuGet imports .props and .targets files in the /build/<framework> folder of the pkg. (Framework is 'native' in this case).

So, the packager has to do all the actual hard work :D
May 2, 2013 at 12:52 PM
Thanks for all this

I've read through http://coapp.org/tutorials/building-a-package.html and watched the HD stream.

I'm wondering... for my case it should be very simple - I've only got this one SQLite dependency and I'm only targetting WinStore clients - so I think it should be relatively easy to write a .targets file myself - just for this one assembly. So I think the number of pivots I need to make is minimal - just ARM vs x86 vs x64.

Is there anywhere can I find out about the .targets format? e.g. a finished example without using the CoApp scripts

Sorry for being a luddite - but trying to do this myself before I tool up now :)

Stuart
May 2, 2013 at 2:53 PM
stuartlodge wrote:
Thanks for all this

I've read through http://coapp.org/tutorials/building-a-package.html and watched the HD stream.

I'm wondering... for my case it should be very simple - I've only got this one SQLite dependency and I'm only targetting WinStore clients - so I think it should be relatively easy to write a .targets file myself - just for this one assembly. So I think the number of pivots I need to make is minimal - just ARM vs x86 vs x64.

Is there anywhere can I find out about the .targets format? e.g. a finished example without using the CoApp scripts

Sorry for being a luddite - but trying to do this myself before I tool up now :)

Stuart
The .targets file is just an msbuild project file -- http://msdn.microsoft.com/en-us/library/ms171452(v=VS.90).aspx


Not that really tells you what to do -- just lookin' at the format doesn't help much in knowing how to setup the files correctly. :D

If you've got just the one pivot, you can use just that one pivot, and not worry about the rest.

If you want to see the finished example, download one of the native packages, and unzip it (.nupkg files are just zip files) and check out the .targets file.

Check out the native pkgs that we've built: http://j.mp/Zpu5X9

Of course, looking at the .targets from those makes it look far more complicated than it would have to be....

G
May 2, 2013 at 5:20 PM
I've tried most of them....

None of them want to install into a '.NETCore,Version=v4.5', project

Any examples which target a WinStore app with different x86. x64, Arm flavours?

I am sooooo needy - sorry!
May 2, 2013 at 5:53 PM
Is this for a managed project?

That's quite a different story.

NuGet imports .targets and .props files based on the package name and framework.

so the /build/native/foo.targets file only gets imported if you are importing into a native project.

you need to put the targets file into the /build/<framework> folder that you're supporting.
May 2, 2013 at 5:59 PM
yes... it's for a windows store project...

I'm currently trying to work out how to distribute some assemblies that have a dependency on SQLite.

The assemblies are all called: Cirrious.MvvmCross.Plugins.Sqlite.WinRT.dll - but there are 3 separate assemblies - one for x86, one for x64 and one for ARM

Because of this platform specific dependency I need to insert these dependencies as assembly references using code like in the target C# WindowsStore app:
<Choose>
<When Condition=" '$(Platform)' == 'ARM' ">
  <ItemGroup>
    <Reference Include="Cirrious.MvvmCross.Plugins.Sqlite.WinRT.dll, Version=1.0.0.0, Culture=neutral, processorArchitecture=MSIL">
      <SpecificVersion>False</SpecificVersion>
      <HintPath>..\..\libs\MvX\WinRT\Release\ARM\Cirrious.MvvmCross.Plugins.Sqlite.WinRT.dll</HintPath>
    </Reference>
  </ItemGroup>
</When>
</Choose>
<Choose>
<When Condition=" '$(Platform)' == 'x64' ">
  <ItemGroup>
    <Reference Include="Cirrious.MvvmCross.Plugins.Sqlite.WinRT.dll, Version=1.0.0.0, Culture=neutral, processorArchitecture=MSIL">
      <SpecificVersion>False</SpecificVersion>
      <HintPath>..\..\libs\MvX\WinRT\Release\x64\Cirrious.MvvmCross.Plugins.Sqlite.WinRT.dll</HintPath>
    </Reference>
  </ItemGroup>
</When>
</Choose>
<Choose>
<When Condition=" '$(Platform)' == 'x86' ">
  <ItemGroup>
    <Reference Include="Cirrious.MvvmCross.Plugins.Sqlite.WinRT.dll, Version=1.0.0.0, Culture=neutral, processorArchitecture=MSIL">
      <SpecificVersion>False</SpecificVersion>
      <HintPath>..\..\libs\MvX\WinRT\Release\x86\Cirrious.MvvmCross.Plugins.Sqlite.WinRT.dll</HintPath>
    </Reference>
  </ItemGroup>
</When>
</Choose>
Stuart
Jun 25, 2013 at 12:45 AM
Stuart, did you ever get this to work? I need to do the same thing: ship 3 different WinRT dlls (one per architecture) to be consumed by a Windows Store C# app.

Or... if there's an example of a NuGet package out there already that does this, I could easily just crack it open and take a peek at how it's done.

Thanks!
Jun 25, 2013 at 12:49 AM
Hi Tim,

I was able to do this for WP8 projects here:
https://github.com/onovotny/WinRTTimeZones

I needed to switch references for ARM and for x86 to support the device and emulator. I do it via a custom targets file borrowed from the Microsoft.Bcl.Compression implementation, but it can be adapted.

The neat thing it does is "inject" the platform-specific references so that the main referenced dll can remain AnyCpu and avoid warnings.
Apr 17 at 3:04 PM
Here's the powershell I used to insert the SDKReferenceDirectoryRoot tag (I put the ExtensionSDK structure under /tools/)
    #
    # Add SDK Reference Directory root
    #
    #  <PropertyGroup>
    #    <SDKReferenceDirectoryRoot>$(SolutionDir)\packages\[packagename]\tools\ExtensionSDKs\;$(SDKReferenceDirectoryRoot)</SDKReferenceDirectoryRoot>
    #  </PropertyGroup>
    $group = $msbuild.Xml.AddPropertyGroup();
    $sdkroot = $msbuild.GetProperty("SDKReferenceDirectoryRoot");
    if( $sdkroot.IsImported ) #doesn't exist in project - create new node
    {
    $group = $msbuild.Xml.AddPropertyGroup();
    $group.AddProperty("SDKReferenceDirectoryRoot", $toolsPath + "\ExtensionSDKs\;`$(SDKReferenceDirectoryRoot)");
    # remove and re-add to ensure it goes on the bottom
    $msbuild.Xml.RemoveChild($group);
    $msbuild.Xml.AppendChild($group);
    }
    else #append to node
    {
        sdkroot.Xml.Value = $toolsPath + "\ExtensionSDKs\;" + sdkroot.Xml.Value;
    }
The other step was to add a .targets file that references the SDK. Seems to work like a charm
Apr 17 at 4:13 PM
Edited Apr 17 at 5:31 PM
...also added this issue (please vote!): https://nuget.codeplex.com/workitem/4119