Command line parsing

Sep 7, 2010 at 4:18 PM

I saw in th NuPack project there was some TODO driven programming about dealing with the command line. I've also struggled with this and went hunting for solutions (I'm building a command line tool right now and really love the simplicity of Ruby's OptionParser, alas there's no .NET equivelant and probably can't be due to the dynamic nature of Ruby (although I'm considering creating an NOptionParser library to mimic it).

Anyways, here are some options I found. Looking for some feedback on your thoughts or do we just RYO and put some simple code into the system to do this. (PS isn't there some internal MSFT project that deals with this already we can abscond the source code from?)

C# Command Line Parser (CmdLine)
http://cmdline.codeplex.com/
Simple implementation but limited to string and int options. GPLv2. Ugh

NDesk.Options
http://ndesk.org/Options
One of the better ones mentioned out there and has a lot of options. Would just be a new dll (NDesk.Options.dll) that would be included with the command line tool. MIT/X11 license. Also has a Mono implementation found here: http://anonsvn.mono-project.com/source/trunk/mcs/class/Mono.Options/Mono.Options/Options.cs

CommonLibrary.NET
http://commonlibrarynet.codeplex.com/
Has an Arguments parser that seems pretty feature right but don't want to drag the entire library in for that. LGPL. Ugh. (BTW some pretty cool utils in this thing if you're looking for stuff)

BizArk Framework
http://bizark.codeplex.com/
Has a command-line parser detailed here: http://bizark.codeplex.com/wikipage?title=Command-line%20Parsing&referringTitle=Home which is pretty simple but again, part of a bigger framework we don't want to bring in. MS-PL license (does that allow partial copying?)

Genghis
http://www.sellsbrothers.com/Posts/Details/12655
Another big ass framework but has a command line parser (by Chris Sells). RYO license.

There were others I found out there but they were either too old (I ignored anything that hasn't been updated since 2007) or too complex (one had a gianormous single class to handle them).

PS I hate the fact that developers have to solve this problem in 2010. It should be something in the framework. Not that it's going to change anything, that's just me ranting.

Coordinator
Sep 7, 2010 at 4:31 PM

Let's define what we need out of a command line parser first, then figure out which of the existing libraries, if any, meet those needs. What style of command line flags do we want?

  npack /flag:value /flag2

  npack --verbose-flag flag value -vf another value

???

 

Sep 7, 2010 at 4:39 PM

Sorry, putting solutions on the table without requirements. My bad.

A few ideas on requirements:

  • I prefer the dash style style over the /property:value but maybe that's just personal preference
  • Flags/optons should be specified in any order
  • Each option needs it's own help/syntax, (optional) default value, and exception handling so it's self-sufficient
  • Options should include a short version (-v), long version (--version), no version (for options that can be turned off --[no]-default_library) and value version (--libdir="test dir"). For V1 of the product maybe just the short and value version can suffice.
  • Adding options should be fairly simple and robotic. I shouldn't have to think much about adding a new option (add option to section of code, add method to handle it, done)
  • (nice to have) get the program to spit out it's options so we can just drop it onto a wiki page for documentation (and users don't have to go hunting on a website to get help for the program)
Coordinator
Sep 7, 2010 at 4:41 PM

Ok, which of these libraries support all this? :) I found there are more libraries on SourceForge that you didn't list. Is this a comprehensive list?

Developer
Sep 7, 2010 at 4:43 PM

Totally agree with your rant, BCL should ship with its own command line parse.

We looked at some of those options that you listed and discovered that we had an internally implemented command line parser (CLP). Our current working scenario has a single argument to deal with (nupack <manifest file>) which makes dealing with arguments and help pretty trivial. In fact the latest code from the console app does not contain any of the hacked up regex parsing (and the TODO message is out-of-date J). Our original scenario consisted of a way to build a nupack file without a manifest file (nupack.exe /id:foo /version:1.1 /out:abc.nupack <content files>) which necessitated a decent CLP.

Cheers,

Pranav

Sep 7, 2010 at 4:44 PM

http://devlicious.com/blogs/rob_reynolds/archive/2009/11/22/command-line-parsing-with-mono-options.aspx (NDesk.Options) - Take a look at how sweet the setup is:

OptionSet option_set = new OptionSet()   
    .Add("?|help|h", "Prints out the options.", option => help = option != null)   
    .Add("d=|db=|database=|databasename=",   
       "REQUIRED: DatabaseName - The database you want to create/migrate.",   
       option => configuration.DatabaseName = option) 

 It is a single cs file that you can include in source and has the attributions in the file. So there is no need for another DLL.

Notice how it supports all of those styles for command line flags Phil: <tt>-flag</tt>, <tt>--flag</tt>, <tt>/flag</tt>

  • Parameters of the form: <tt>-flag</tt>, <tt>--flag</tt>, <tt>/flag</tt>, <tt>-flag=value</tt>, <tt>--flag=value</tt>, <tt>/flag=value</tt>, <tt>-flag:value</tt>, <tt>--flag:value</tt>, <tt>/flag:value</tt>, <tt>-flag value</tt>, <tt>--flag value</tt>, <tt>/flag value</tt>.
  • "boolean" parameters of the form: <tt>-flag</tt>, <tt>--flag</tt>, and <tt>/flag</tt>. Boolean parameters can have a `<tt>+</tt>' or `<tt>-</tt>' appended to explicitly enable or disable the flag (in the same fashion as <tt>mcs -debug+</tt>). For boolean callbacks, the provided value is non-<tt>null</tt> for enabled, and <tt>null</tt> for disabled.
  • "value" parameters with a required value (append `<tt>=</tt>' to the option name) or an optional value (append `<tt>:</tt>' to the option name). The option value can either be in the current option (<tt>--opt=value</tt>) or in the following parameter (<tt>--opt value</tt>). The actual value is provided as the parameter to the callback delegate, unless it's (1) optional and (2) missing, in which case <tt>null</tt> is passed.
  • Parameters of the form: <tt>-flag</tt>, <tt>--flag</tt>, <tt>/flag</tt>, <tt>-flag=value</tt>, <tt>--flag=value</tt>, <tt>/flag=value</tt>, <tt>-flag:value</tt>, <tt>--flag:value</tt>, <tt>/flag:value</tt>, <tt>-flag value</tt>, <tt>--flag value</tt>, <tt>/flag value</tt>.
  • "boolean" parameters of the form: <tt>-flag</tt>, <tt>--flag</tt>, and <tt>/flag</tt>. Boolean parameters can have a `<tt>+</tt>' or `<tt>-</tt>' appended to explicitly enable or disable the flag (in the same fashion as <tt>mcs -debug+</tt>). For boolean callbacks, the provided value is non-<tt>null</tt> for enabled, and <tt>null</tt> for disabled.
  • "value" parameters with a required value (append `<tt>=</tt>' to the option name) or an optional value (append `<tt>:</tt>' to the option name). The option value can either be in the current option (<tt>--opt=value</tt>) or in the following parameter (<tt>--opt value</tt>). The actual value is provided as the parameter to the callback delegate, unless it's (1) optional and (2) missing, in which case <tt>null</tt> is passed.

    The vote of the community is with NDesk.Options as well: http://stackoverflow.com/questions/491595/best-way-to-parse-command-line-arguments-in-c (Notice it is the top two answers with a total of 25 votes).

  • Sep 7, 2010 at 4:44 PM

    I don't know and probably none completely. These are just my thoughts but it's not the one to say what all the must-have vs. nice-to-have features are.

    No, this is not a comprehensive list. There are dozens of others out there. This list is just the ones that I found to be a) somewhat feature rich b) mature (enough to use) c) currently developed or active in some form (i.e. not abandoned).

    Sep 7, 2010 at 4:45 PM

    Well that didn't copy in nicely. I think the gist is there though...

    Developer
    Sep 7, 2010 at 4:47 PM

    I think we want a mercurial style command line tool. i.e 

    nupack [command] arguments
    nupack help [command]

    There is a command line parser I know of that works this way (it's actually used on codeplex for the svn bridge) and it's written by a softie (Brad Wilson). We should see if we can use this.

    Sep 7, 2010 at 4:48 PM

    @ferventcoder: Yeah, I included NDesk.Options from that SO question as well. It looks like it might handle most if not everything we want. Maybe just do a quick spike to see if it fits?

    If we can include it in as a single file would that be cool from a MSFT licensing perspective? I'm not sure the implications of "copy-n-paste" code from other projects into this one?

    Coordinator
    Sep 7, 2010 at 4:49 PM

    Yeah, I like that. It's very similar to the gem syntax as well. gem [command] arguments. I think this also fits with what Bill wanted. Let's take a look at Brad's command line parser. Can you provide details on it?

    Coordinator
    Sep 7, 2010 at 4:54 PM

    Bil, did you just volunteer to spike on it? :)

    We can probably copy-n-paste the code just fine as long as we retain the copyright notice. As usual, if we choose this one, I'll check with our lawyers. But I'm 99.999% sure of this one.

    Sep 7, 2010 at 4:55 PM

    @haacked: I can spike NDesk.Options sure. Can you get me the code for the other one and I'll do that for comparison.

    Sep 7, 2010 at 4:56 PM

    The Magnum guys were writing one that implements the "[command] arguments" that scans for items that implement a particular command. To add a new command you just add another class that implements the argument. The implementation I saw made me want to flip over to use it instead of NDesk.Options for RoundhousE.  I haven't had a chance to spike the magnum one yet though.

    So you can imagine the maintenance is very small. You add a class and you are done.

    Dru?

    Developer
    Sep 7, 2010 at 4:58 PM

    The code is acutally hosted on codeplex :) 

    http://codeplexclient.codeplex.com/SourceControl/changeset/view/52373#40625

    Sep 7, 2010 at 5:01 PM

    @dfowler: Neat, that looks like the stuff Rob was talking about (create a new class for each command). Kind of like this. Will take a look this morning and report back on some results/comparisons.

    Sep 7, 2010 at 5:01 PM

    NDesk itself doesn't do "exe [command] options" only "exe [options]" unless I'm missing something. I do the "[command]" part myself.  So if we are looking at something that does "exe [command] options" we might want to take a look at Brad's or the one that Chris Patterson/Dru wrote.  I'm for whichever one has the lowest amount of mainenance (assuming both work out of the box).

    NDesk does meet all of the other requirements though.

    Sep 7, 2010 at 5:03 PM

    Could we put together (in this thread or a wiki page) a list of some of the command line options we're thinking of implmenting so I can just build some test cases around them? Thanks.

    Sep 7, 2010 at 6:29 PM
    you can find the command object style approach in the old 'nu' which is located here:
    http://github.com/drusellers/nu/tree/oldnet
    http://github.com/drusellers/nu/tree/oldnet/src/nu.core/Commands/

    -d

    On Tue, Sep 7, 2010 at 11:03 AM, bsimser <notifications@codeplex.com> wrote:

    From: bsimser

    Could we put together (in this thread or a wiki page) a list of some of the command line options we're thinking of implmenting so I can just build some test cases around them? Thanks.

    Read the full discussion online.

    To add a post to this discussion, reply to this email (nupack@discussions.codeplex.com)

    To start a new discussion for this project, email nupack@discussions.codeplex.com

    You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

    Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com


    Sep 7, 2010 at 7:51 PM

    Spent some time with the code from the CodePlex Client on command line parsing. It's interesting and I'm spending the rest of the day mucking about with it and other options. There's two show stoppers in using the code as-is (well one really). It depends on TfsLibrary stuff (it's CodePlex code but an abstract interface overtop of TFS). The other hump is it introduces a DependencyContainer which is a CodePlex codebase abstraction over an IoC.

    It also drags in a bunch of other things like helpers or parsers and attributes, but it's all good stuff. It's just more than one .cs file.

    In any case, the code itself is curious and uses a similar approach that Dru pointed out in the older nu codebase (using a Command class with an Execute method). I think that approach is solid so you just create new classes for each command line option you add.

    Going down the NDesk.Options path to see what that looks like but I think in the end we might return to a hybrid RYO using the CodePlex code as a base (with all the TFS and CodePlex stuff stripped out).

    Will make for a nice separate package when it's done.

    I'm all over this like a fat kid on a smartie (and writing a blog post on it as I go).

    Developer
    Sep 7, 2010 at 7:56 PM

    Sweet! I actually have a trimmed down version of that code that removes all of the stuff you just mentioned and keeps the bare bones needed. I also have a modified help printer and I think some other bug fixes. Should I sent that over?

    Sep 7, 2010 at 7:58 PM

    Oh cool, if you already have that stuff feel free to send me a zip or create a release package (don't want to jam it into the codebase just yet). Saves me from having to do it. Thanks.

    Developer
    Sep 7, 2010 at 8:05 PM

    Heh, I actually used MEF to load the commands. That might not be a bad idea.

    Sep 7, 2010 at 9:02 PM

    @dfowler: does your copy remove the dependency on Windows.Forms though (the Application class).

    Developer
    Sep 7, 2010 at 9:58 PM

    Yea, I have a stand alone console app. I'll zip it up.

    Coordinator
    Sep 19, 2010 at 12:55 AM

    Hey, so what ever happened with the command line parsing? Did we resolve this or do we need to log an issue to follow up and close on this?

    Sep 19, 2010 at 12:59 AM

    David had something and I had a fork going. I'll check to see if there's somehing logged and if not add it. Thanks.

    Sep 20, 2010 at 4:14 AM
    This discussion has been copied to a work item. Click here to go to the work item and continue the discussion.