program_options strips off escaped quotes in some situations

[apoxy]

boost::program_options strips off escaped quotes from command line strings like -a "\"quoted value\"".  A command line string like -a "\"quoted value\" unquoted_value" does not have its escaped quotes striped out.

I am using program_options in Windows to parse the command line of my application.  One of the requirements is that I be able to pass in the command line of a different application such as the following:
myapp -c "\"c:\App versions\App v1\App.exe\" -a \"A Value\" -b Another_Value"

In this case, program_options will correctly parse out the value of the parameter c to be:
"c:\App versions\App v1\App.exe" -a "A Value" -b Another_Value

However, if I pass in the following:
myapp -c "\"c:\App versions\App v1\App.exe\""

program_options parses c as:
c:\App versions\App v1\App.exe

Note that the escaped quotes were stripped off instead of preserved like you would expect.  This is a problem because my application immediately turns around and calls ::CreateProcess using the value of the c parameter.   ::CreateProcess' documented behavior is that it executes the first match it can find of c:\App.exe, c:\App versions\App.exe, and c:\App versions\App v1\App.exe.  The only way you can stop this behavior is by enclosing the file path in quotes.

The code I'm using to get the options is pasted below.  Please email me with any questions.  Thanks!

--------------------

pair<string, bool> CCmdLineParser::GetParam(const string& sParamName)
{
	Run(); // Make sure we're parsed
	if (!Impl_.bOptionsDescribed && !Impl_.bPositionalOptionsDescribed)
		return GetUnregisteredParam_(sParamName);
	return Impl_.RegisteredOptions.count(sParamName)
		? make_pair(Impl_.RegisteredOptions[sParamName].as<string>(), true) : make_pair("", false);
}

void CCmdLineParser::Run(void)
{
	if (Impl_.bInitialized)
		return;
	Impl_.bInitialized = true;

	// Create parser
	command_line_parser Parser(split_winmain(AfxGetApp()->m_lpCmdLine));
	bool bUnregistered = !Impl_.bOptionsDescribed && !Impl_.bPositionalOptionsDescribed;

	// Setup parser
	Parser.options(Impl_.OptionsDesc); // Always required (even if empty) because command_line_parser::run() asserts it
	if (Impl_.bPositionalOptionsDescribed)
		Parser.positional(Impl_.PositionalOptionsDesc);
	if (bUnregistered)
		Parser.allow_unregistered();

	// Parse
	parsed_options ParsedOptions = Parser.run();

	// Retrieve
	// We can't call store on something with unregistered options.  It throws an exception even if you specifically
	// call allow_unregistered().  This is a known bug.  It is fixed in the boost CVS tree, but not released yet.
	// What all of this basically means if that we can't mix registered and unregistered options (yet).
	if (bUnregistered)
		Impl_.UnregisteredOptions = collect_unrecognized(ParsedOptions.options, include_positional);
	else {
		store(ParsedOptions, Impl_.RegisteredOptions);
		// notify() is required for automatic value storage and default values
		notify(Impl_.RegisteredOptions);
	}
}

[gardnermj@…]

I can confirm this on Linux, and add my vote for it to be fixed as well. My application is a type of search engine, and this behavior breaks phrase searches when the entire query is a phrase.

I believe this behavior is new as of boost 1.33, as I didn't observe it with version 1.32.

[anonymous]

I notice a similar problem (in a similar application). Not that in order to preserve quotes, I need to add an extra (third) level of quotes. This is in boost 1.34. First example is single quote, double quote. Second example is single quote, double quote, double quote.

$ ./search --query='"a test"'
Query: a test

$ ./search --query='""a test""'
Query: "a test"

[s.ochsenknecht@…]

patch

[s.ochsenknecht@…]

There is some code which is removing quotes like '"' and if they occur and the begin and at the of the option value. Maybe there is a good reason for this? I don't know (maybe because of config files?).

Anyhow, I attached a patch which removes this code and keeps the value as it come from the command line (expect the quotas which were removed by the shell).

[Vladimir Prus]

The code in question dates back to 2004-05-14, and I no longer have any idea why it's here. Let's try to back it out.

[Vladimir Prus]

(In [57519]) Don't strip quotes from values.

Fixes #850. Patch from Sascha Ochsenknecht.

[thomas.jarosch@…]

Hello,

I just updated from boost version 1.39.0 to boost 1.44.0 (on linux) and this change broke the config file parsers.

Imagine a simple config file like this:

WorkDir= "/some/where"

The new code keeps the quotes on the directory name string and so the open() call for the directory fails.

So I could change my code to manually strip the quotes or remove the quotes in the config file (which get generated automatically).

Though there's one case where this won't work as good as before. Imagine a config line like this:

WorkDir = "/some/where "

This can't be express with the changed code as: WorkDir = /some/where

-> It will strip all the spaces, so you can't have config file values ending on spaces anymore.

So the code was broken for the CLI parser but needed for the config file parser (as suspected above...).

Any chance we can restore the behavior for the config file parser only?

Cheers, Thomas

[Olaf van der Spek <olafvdspek@…>]

Hi Vladimir,

Any updates?