Using Python to parse complex arguments to shell script

You could potentially take advantage of associative arrays in bash to help obtain your goal.

declare -A opts=($(getopts.py $@))
cd ${opts[dir]}
complex_function ${opts[append]}  ${opts[overwrite]} ${opts[recurse]} \
                 ${opts[verbose]} ${opts[args]}

To make this work, getopts.py should be a python script that parses and sanitizes your arguments. It should print a string like the following:

[dir]=/tmp
[append]=foo
[overwrite]=bar
[recurse]=baz
[verbose]=fizzbuzz
[args]="a b c d"

You could set aside values for checking that the options were able to be properly parsed and sanitized as well.

Returned from getopts.py:

[__error__]=true

Added to bash script:

if ${opts[__error__]}; then
    exit 1
fi

If you would rather work with the exit code from getopts.py, you could play with eval:

getopts=$(getopts.py $@) || exit 1
eval declare -A opts=($getopts)

Alternatively:

getopts=$(getopts.py $@)
if [[ $? -ne 0 ]]; then
    exit 1;
fi
eval declare -A opts=($getopts)

Edit: I haven't used it (yet), but if I were posting this answer today I would probably recommend https://github.com/docopt/docopts instead of a custom approach like the one described below.

---

I've put together a short Python script that does most of what I want. I'm not convinced it's production quality yet (notably error handling is lacking), but it's better than nothing. I'd welcome any feedback.

It takes advantage of the set builtin to re-assign the positional arguments, allowing the remainder of the script to still handle them as desired.

bashparse.py

#!/usr/bin/env python

import optparse, sys
from pipes import quote

'''
Uses Python's optparse library to simplify command argument parsing.

Takes in a set of optparse arguments, separated by newlines, followed by command line arguments, as argv[2] and argv[3:]
and outputs a series of bash commands to populate associated variables.
'''

class _ThrowParser(optparse.OptionParser):
    def error(self, msg):
        """Overrides optparse's default error handling
        and instead raises an exception which will be caught upstream
        """
        raise optparse.OptParseError(msg)

def gen_parser(usage, opts_ls):
    '''Takes a list of strings which can be used as the parameters to optparse's add_option function.
    Returns a parser object able to parse those options
    '''
    parser = _ThrowParser(usage=usage)
    for opts in opts_ls:
        if opts:
            # yes, I know it's evil, but it's easy
            eval('parser.add_option(%s)' % opts)
    return parser

def print_bash(opts, args):
    '''Takes the result of optparse and outputs commands to update a shell'''
    for opt, val in opts.items():
        if val:
            print('%s=%s' % (opt, quote(val)))
    print("set -- %s" % " ".join(quote(a) for a in args))

if __name__ == "__main__":
    if len(sys.argv) < 2:
        sys.stderr.write("Needs at least a usage string and a set of options to parse")
        sys.exit(2)
    parser = gen_parser(sys.argv[1], sys.argv[2].split('\n'))

    (opts, args) = parser.parse_args(sys.argv[3:])
    print_bash(opts.__dict__, args)

Example usage:

#!/bin/bash

usage="[-f FILENAME] [-t|--truncate] [ARGS...]"
opts='
"-f"
"-t", "--truncate",action="store_true"
'

echo "$(./bashparse.py "$usage" "$opts" "$@")"
eval "$(./bashparse.py "$usage" "$opts" "$@")"

echo
echo OUTPUT

echo $f
echo $@
echo $0 $2

Which, if run as: ./run.sh one -f 'a_filename.txt' "two' still two" three outputs the following (notice that the internal positional variables are still correct):

f=a_filename.txt
set -- one 'two'"'"' still two' three

OUTPUT
a_filename.txt
one two' still two three
./run.sh two' still two

Disregarding the debugging output, you're looking at approximately four lines to construct a powerful argument parser. Thoughts?

Having the very same needs, I ended up writing an optparse-inspired parser for bash (which actually uses python internally); you can find it here:

https://github.com/carlobaldassi/bash_optparse

See the README at the bottom for a quick explanation. You may want to check out a simple example at:

https://github.com/carlobaldassi/bash_optparse/blob/master/doc/example_script_simple

From my experience, it's quite robust (I'm super-paranoid), feature-rich, etc., and I'm using it heavily in my scripts. I hope it may be useful to others. Feedback/contributions welcome.