From: W. Trevor King Date: Thu, 31 Dec 2009 20:54:12 +0000 (-0500) Subject: Merged be.restructure, major internal reorganization. X-Git-Tag: 1.0.0~59^2~52 X-Git-Url: http://git.tremily.us/?a=commitdiff_plain;h=b0b5341c4045dd27cfbb3e2585cb2614ed9ad903;p=be.git Merged be.restructure, major internal reorganization. Added a bunch of classes to make the commands, user interfaces, and storage backends more abstract and distinct. This should make it much easier to extend and maintain BE. Features: * Directory restructured: becommands/ -> libbe/commands submods sorted by functionality. * Lots of new classes: Option, Argument, Command InputOutput, StorageCallbacks, UserInterface Storage * Consolidated ID handling in libbe.util.id * Transitioned VCS backends for Python-based VCSs from subprocess calss to internal python calls. Plus the user-visible changes: * New bugdir/bug/comment ID format replaces old bug:comment format. * Deprecated support for `be diff` on Arch and Darcs <= 2.3.1. A new backend abstraction (Storage) makes the former implementation ungainly. * Improved command completion. * Removed commands close, open, email_bugs, * Flipped some arguments `be assign BUG-ID [ASSIGNEE]` -> `be status ASSIGNED BUG-ID ...` `be severity BUG-ID SEVERITY` -> `be severity SEVERITY BUG-ID ...` `be status BUG-ID STATUS` -> `be status STATUS BUG-ID ...` In the merge: * Added 'commit' to list of pagerless commands. * Updated doc/README.dev See #bea86499-824e-4e77-b085-2d581fa9ccab/1100c966-9671-4bc6-8b68-6d408a910da1# for a discussion of why the changes were made and some of the difficulties en-route. --- b0b5341c4045dd27cfbb3e2585cb2614ed9ad903 diff --cc doc/README.dev index 0000000,2a09463..a2b8d30 mode 000000,100644..100644 --- a/doc/README.dev +++ b/doc/README.dev @@@ -1,0 -1,96 +1,64 @@@ + Extending BE + ============ + -To write a plugin, you simply create a new file in the becommands -directory. Take a look at one of the simpler plugins (e.g. open.py) ++Adding commands ++--------------- ++ ++To write a plugin, you simply create a new file in the libbe/commands/ ++directory. Take a look at one of the simpler plugins (e.g. remove.py) + for an example of how that looks, and to start getting a feel for the + libbe interface. + -To fit into the current framework, your extension module should -provide the following elements: - __desc__ - A short string describing the purpose of your plugin - execute(args, manipulate_encodings=True, restrict_file_access=False, - dir=".") - The entry function for your plugin. args is everything from - sys.argv after the name of your plugin (e.g. for the command - `be open abc', args=['abc']). - - manipulate_encodings should be passed through to any calls to - bugdir.BugDir(). See the BugDir documentation for details. - - If restrict_file_access==True, you should call - cmdutil.restrict_file_access(bugdir, path) - before attempting to read or write a file. See the - restrict_file_access documentation for details. - - dir is a directory inside the repository of interest. - - Note: be supports command-completion. To avoid raising errors you - need to deal with possible '--complete' options and arguments. - See the 'Command completion' section below for more information. - help() - Return the string to be output by `be help ', - `be --help', etc. - -While that's all that's strictly necessary, many plugins (all the -current ones) use libbe.cmdutil.CmdOptionParser to provide a -consistent interface - get_parser() - Return an instance of CmdOptionParser(""). You can - alter the parser (e.g. add some more options) before returning it. - -Again, you can just browse around in becommands to get a feel for things. ++See libbe/commands/base.py for the definition of the important classes ++Option, Argument, Command, InputOutput, StorageCallbacks, and ++UserInterface classes. You'll be subclassing Command for your ++command, but all those classes will be important. ++ ++ ++Command completion ++ ++BE implements a general framework to make it easy to support command ++completion for arbitrary plugins. In order to support this system, ++any of your completable Argument() instances (in your commands ++.options or .args) should be initialized with some valid ++completion_callback function. Some common cases are defined in ++libbe.command.util. If you need more flexibility, see ++libbe.command.list's "--sort" option for an example of extensions via ++libbe.command.util.Completer, or write a custom completion function ++from scratch. ++ ++ ++Adding user interfaces ++---------------------- ++ ++Take a look at libbe/ui/command_line.py for an example. Basically ++you'll need to setup a UserInterface instance for running commands. ++More details to come after I write an HTML ui... + + + Testing -------- ++======= + -Run any doctests in your plugin with - be$ python test.py ++Run any tests in your module with ++ be$ python test.py + for example - be$ python test.py merge ++ be$ python test.py libbe.command.merge + ++For a definition of "any tests", see test.py's add_module_tests() ++function. + -Command completion ------------------- - -BE implements a general framework to make it easy to support command -completion for arbitrary plugins. In order to support this system, -all becommands should properly handle the '--complete' commandline -argument, returning a list of possible completions. For example - $ be --commands - lists options accepted by be and the names of all available becommands. - $ be list --commands - lists options accepted by becommand/list - $ be list --status --commands - lists arguments accepted by the becommand/list --status option - $ be show -- --commands - lists possible vals for the first positional argument of becommand/show -This is a lot of information, but command-line completion is really -convenient for the user. See becommand/list.py and becommand/show.py -for example implementations. The basic idea is to raise - cmdutil.GetCompletions(['list','of','possible','completions']) -once you've determined what that list should be. - -However, command completion is not critical. The first priority is to -implement the target functionality, with fancy shell sugar coming -later. In recognition of this, cmdutil provides the default_complete -function which ensures that if '--complete' is any one of the -arguments, options, or option-arguments, GetCompletions will be raised -with and empty list. + + Profiling + ========= + + Find out which 20 calls take the most cumulative time (time of + execution + childrens' times). + + $ python -m cProfile -o profile be [command] [args] + $ python -c "import pstats; p=pstats.Stats('profile'); p.sort_stats('cumulative').print_stats(20)" + + It's often useful to toss a + import sys, traceback + print >> sys.stderr, '-'*60, '\n', '\n'.join(traceback.format_stack()[-10:]) + into expensive functions (e.g. libbe.util.subproc.invoke()), if you're + not sure why they're being called. diff --cc libbe/command/close.py index 026c605,0000000..026c605 mode 100644,000000..100644 --- a/libbe/command/close.py +++ b/libbe/command/close.py diff --cc libbe/command/email_bugs.py index f6641e3,0000000..f6641e3 mode 100644,000000..100644 --- a/libbe/command/email_bugs.py +++ b/libbe/command/email_bugs.py diff --cc libbe/command/open.py index a6fe48d,0000000..a6fe48d mode 100644,000000..100644 --- a/libbe/command/open.py +++ b/libbe/command/open.py diff --cc libbe/ui/command_line.py index 0000000,1c7399d..7f74782 mode 000000,100755..100644 --- a/libbe/ui/command_line.py +++ b/libbe/ui/command_line.py @@@ -1,0 -1,317 +1,317 @@@ + # Copyright (C) 2005-2009 Aaron Bentley and Panometrics, Inc. + # Chris Ball + # Gianluca Montecchi + # Oleg Romanyshyn + # W. Trevor King + # + # This program is free software; you can redistribute it and/or modify + # it under the terms of the GNU General Public License as published by + # the Free Software Foundation; either version 2 of the License, or + # (at your option) any later version. + # + # This program is distributed in the hope that it will be useful, + # but WITHOUT ANY WARRANTY; without even the implied warranty of + # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + # GNU General Public License for more details. + # + # You should have received a copy of the GNU General Public License along + # with this program; if not, write to the Free Software Foundation, Inc., + # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + + """ + A command line interface to Bugs Everywhere. + """ + + import optparse + import os + import sys + + import libbe + import libbe.bugdir + import libbe.command + import libbe.command.util + import libbe.version + import libbe.ui.util.pager + + if libbe.TESTING == True: + import doctest + + class CallbackExit (Exception): + pass + + class CmdOptionParser(optparse.OptionParser): + def __init__(self, command): + self.command = command + optparse.OptionParser.__init__(self) + self.remove_option('-h') + self.disable_interspersed_args() + self._option_by_name = {} + for option in self.command.options: + self._add_option(option) + self.set_usage(command.usage()) + + + def _add_option(self, option): + option.validate() + self._option_by_name[option.name] = option + long_opt = '--%s' % option.name + if option.short_name != None: + short_opt = '-%s' % option.short_name + assert '_' not in option.name, \ + 'Non-reconstructable option name %s' % option.name + kwargs = {'dest':option.name.replace('-', '_'), + 'help':option.help} + if option.arg == None: # a callback option + kwargs['action'] = 'callback' + kwargs['callback'] = self.callback + elif option.arg.type == 'bool': + kwargs['action'] = 'store_true' + kwargs['metavar'] = None + kwargs['default'] = False + else: + kwargs['type'] = option.arg.type + kwargs['action'] = 'store' + kwargs['metavar'] = option.arg.metavar + kwargs['default'] = option.arg.default + if option.short_name != None: + opt = optparse.Option(short_opt, long_opt, **kwargs) + else: + opt = optparse.Option(long_opt, **kwargs) + opt._option = option + self.add_option(opt) + + def parse_args(self, args=None, values=None): + args = self._get_args(args) + options,parsed_args = optparse.OptionParser.parse_args( + self, args=args, values=values) + options = options.__dict__ + for name,value in options.items(): + if '_' in name: # reconstruct original option name + options[name.replace('_', '-')] = options.pop(name) + for name,value in options.items(): + if value == '--complete': + argument = None + option = self._option_by_name[name] + if option.arg != None: + argument = option.arg + fragment = None + indices = [i for i,arg in enumerate(args) + if arg == '--complete'] + for i in indices: + assert i > 0 # this --complete is an option value + if args[i-1] in ['--%s' % o.name + for o in self.command.options]: + name = args[i-1][2:] + if name == option.name: + break + elif option.short_name != None \ + and args[i-1].startswith('-') \ + and args[i-1].endswith(option.short_name): + break + if i+1 < len(args): + fragment = args[i+1] + self.complete(argument, fragment) + for i,arg in enumerate(parsed_args): + if arg == '--complete': + if i > 0 and self.command.name == 'be': + break # let this pass through for the command parser to handle + elif i < len(self.command.args): + argument = self.command.args[i] + elif len(self.command.args) == 0: + break # command doesn't take arguments + else: + argument = self.command.args[-1] + if argument.repeatable == False: + raise libbe.command.UserError('Too many arguments') + fragment = None + if i < len(parsed_args) - 1: + fragment = parsed_args[i+1] + self.complete(argument, fragment) + if len(parsed_args) > len(self.command.args) \ + and self.command.args[-1].repeatable == False: + raise libbe.command.UserError('Too many arguments') + for arg in self.command.args[len(parsed_args):]: + if arg.optional == False: + raise libbe.command.UserError( + 'Missing required argument %s' % arg.metavar) + return (options, parsed_args) + + def callback(self, option, opt, value, parser): + command_option = option._option + if command_option.name == 'complete': + argument = None + fragment = None + if len(parser.rargs) > 0: + fragment = parser.rargs[0] + self.complete(argument, fragment) + else: + print >> self.command.stdout, command_option.callback( + self.command, command_option, value) + raise CallbackExit + + def complete(self, argument=None, fragment=None): + comps = self.command.complete(argument, fragment) + if fragment != None: + comps = [c for c in comps if c.startswith(fragment)] + if len(comps) > 0: + print >> self.command.stdout, '\n'.join(comps) + raise CallbackExit + + class BE (libbe.command.Command): + """Class for parsing the command line arguments for `be`. + This class does not contain a useful _run() method. Call this + module's main() function instead. + + >>> ui = libbe.command.UserInterface() + >>> ui.io.stdout = sys.stdout + >>> be = BE(ui=ui) + >>> ui.io.setup_command(be) + >>> p = CmdOptionParser(be) + >>> p.exit_after_callback = False + >>> try: + ... options,args = p.parse_args(['--help']) # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE + ... except CallbackExit: + ... pass + usage: be [options] [COMMAND [command-options] [COMMAND-ARGS ...]] + + Options: + -h, --help Print a help message. + + --complete Print a list of possible completions. + + --version Print version string. + ... + >>> try: + ... options,args = p.parse_args(['--complete']) # doctest: +ELLIPSIS + ... except CallbackExit: + ... print ' got callback' + --help + --complete + --version + ... + subscribe + tag + target + got callback + """ + name = 'be' + + def __init__(self, *args, **kwargs): + libbe.command.Command.__init__(self, *args, **kwargs) + self.options.extend([ + libbe.command.Option(name='version', + help='Print version string.', + callback=self.version), + libbe.command.Option(name='full-version', + help='Print full version information.', + callback=self.full_version), + libbe.command.Option(name='repo', short_name='r', + help='Select BE repository (see `be help repo`) rather ' + 'than the current directory.', + arg=libbe.command.Argument( + name='repo', metavar='REPO', default='.', + completion_callback=libbe.command.util.complete_path)), + libbe.command.Option(name='paginate', + help='Pipe all output into less (or if set, $PAGER).'), + libbe.command.Option(name='no-pager', + help='Do not pipe git output into a pager.'), + ]) + self.args.extend([ + libbe.command.Argument( + name='command', optional=False, + completion_callback=libbe.command.util.complete_command), + libbe.command.Argument( + name='args', optional=True, repeatable=True) + ]) + + def usage(self): + return 'usage: be [options] [COMMAND [command-options] [COMMAND-ARGS ...]]' + + def _long_help(self): + cmdlist = [] + for name in libbe.command.commands(): + Class = libbe.command.get_command_class(command_name=name) + assert hasattr(Class, '__doc__') and Class.__doc__ != None, \ + 'Command class %s missing docstring' % Class + cmdlist.append((name, Class.__doc__.splitlines()[0])) + cmdlist.sort() + longest_cmd_len = max([len(name) for name,desc in cmdlist]) + ret = ['Bugs Everywhere - Distributed bug tracking', + '', 'Supported commands'] + for name, desc in cmdlist: + numExtraSpaces = longest_cmd_len-len(name) + ret.append('be %s%*s %s' % (name, numExtraSpaces, '', desc)) + ret.extend(['', 'Run', ' be help [command]', 'for more information.']) + return '\n'.join(ret) + + def version(self, *args): + return libbe.version.version(verbose=False) + + def full_version(self, *args): + return libbe.version.version(verbose=True) + + def dispatch(ui, command, args): + parser = CmdOptionParser(command) + try: + options,args = parser.parse_args(args) + ret = ui.run(command, options, args) + except CallbackExit: + return 0 + except libbe.command.UserError, e: + print >> ui.io.stdout, 'ERROR:\n', e + return 1 + except libbe.storage.ConnectionError, e: + print >> ui.io.stdout, 'Connection Error:\n', e + return 1 + except (libbe.util.id.MultipleIDMatches, libbe.util.id.NoIDMatches, + libbe.util.id.InvalidIDStructure), e: + print >> ui.io.stdout, 'Invalid id:\n', e + return 1 + finally: + command.cleanup() + return ret + + def main(): + io = libbe.command.StdInputOutput() + ui = libbe.command.UserInterface(io) + ui.restrict_file_access = False + ui.storage_callbacks = None + be = BE(ui=ui) + ui.setup_command(be) + + parser = CmdOptionParser(be) + try: + options,args = parser.parse_args() + except CallbackExit: + return 0 + except libbe.command.UserError, e: + print >> ui.io.stdout, 'ERROR:\n', e + return 1 + + command_name = args.pop(0) + try: + Class = libbe.command.get_command_class(command_name=command_name) + except libbe.command.UnknownCommand, e: + print >> ui.io.stdout, e + return 1 + + ui.storage_callbacks = libbe.command.StorageCallbacks(options['repo']) + command = Class(ui=ui) + ui.setup_command(command) + - if command.name in ['comment']: ++ if command.name in ['comment', 'commit']: + paginate = 'never' + else: + paginate = 'auto' + if options['paginate'] == True: + paginate = 'always' + if options['no-pager'] == True: + paginate = 'never' + libbe.ui.util.pager.run_pager(paginate) + + ret = dispatch(ui, command, args) + ui.cleanup() + return ret + + if __name__ == '__main__': + sys.exit(main())