Bash completion library

From FVue
Jump to: navigation, search

NOTE: Bash-completion-lib will be merged with bash-completion. Because a lot of my attention goes to bash-completion right now, development on bash-completion-lib has slowed down.

I'm trying to turn the bash-completion file of Ian Macdonald (http://www.caliban.org/bash/) into separate files - a bash completion library.

What are the benefits of doing so?

Loading the 260K file is nearing one second to load every time I login. Splitting up the completions into separate files - loaded just-in-time - is much faster, typically 5 times as fast. Splitting up the 10,000 lines into separate files per completion command also eases maintenance.

Contents

How does it work?

  • All completions run through a central function comp_load. The first time completion is invoked for a specific command, comp_load will load the specific command completion function, run it and install it for subsequent invocations. For example:
$ complete -p alias              # Show how `alias' is completed
complete -F comp_load alias      # First completion uses `comp_load'
$ alias <TAB>                    # Complete on `alias'
$ alias ^C                       # Interrupt by pressing CTRL-C
$ complete -p alias              # Show how `alias' is completed
complete -F _alias alias         # Subsequent completions use `_alias'

Bash_completion_lib loads 10 times faster than bash_completion-20060301 (with caching enabled). With caching disabled, bash_completion_lib loads 5 times faster. For more information, see: #Features

Install

  • unpack the file: tar -xvzf <file>
  • include bash_completion_lib from within your ~/.bashrc:
. bash_completion_lib-1.3.1/bash_completion_lib
  • remove a possible cache file:
$ rm ~/.bash_completion_lib.d/cache~
  • reload .bashrc:
$ . ~/.bashrc

Download

See also:

Subversion repository

This project is hosted at http://code.google.com/p/bash-completion-lib/

Roadmap

                     GOALS                                         REALISATION
     ------------------------------------------   -----------------------------------------------
(1)  Improve bash-completion-20060301:            bash_completion_lib-1.3.1:
 |   - improve initial loading speed              - loads 10x faster
 |   - improve completions                        - contains fixes of 20060301-debian4
 |                                                - contains fixes of 20060301-ubuntu1
 |   - split completions into separate files      - split into 543 files for 609 completions
 |   - add testsuite                              - contains a DejaGnu testsuite with >1300 tests
 |
 |
(2)  Patch bash, so that real dynamic loading     - bash-4.0 contains a `compopt' builtin
 |   becomes possible:                            - Fedora has patched bash-3 with a catch-all 
 |   - add `complete-not-found' handler to bash     programmable completion handler
 |   - add `compset' bash builtin
 |
 |
(3)  Improve bash-completion to do real           --
     dynamic loading

Features

Cache

Bash-completion-lib has a fast algorithm to install completions. During startup, all completions in the ./complete* subdirectories are converted to complete comp_load ... commands. By caching the conversion result, bash-completion-lib reduces this fast startup time even more by 50% (0.20 sec -> 0.10 sec).

Bash-completion-lib automatically sets COMP_CACHE (if not already set) to the following cache file:

COMP_CACHE=$HOME/.bash_completion_lib.d/cache~

After having installed a new version of bash-completion-lib, the cache will automatically be cleared. Bash-completion-lib does this by comparing two environment variables (COMP_VERSION and COMP_CACHE_VERSION) each time it loads the cache file.

To disable caching, set COMP_CACHE to be empty:

COMP_CACHE=

Care must be taken to empty the cache file after updates have been made to the ./completions directory. The cache can be emptied with this bash command:

> $COMP_CACHE

NOTE: The cache need only be reset, if completions have been added, moved or deleted, or when the arguments to `complete' have changed. Other modifications like an improved completion function are automatically loaded - remember: it's only the `complete' installments which are cached. E.g.:

  1. You moved ./completions/*/cd to another directory.
    The cache must be cleared.
  2. You improved the _cd completion within ./completions/*/cd.
    You don't have to clear the cache.

Optional restrict by extension

An environment setting COMP_RESTRICT_BY_EXTENSION lets you decide whether restricted completion is enabled, e.g. whether "acroread TAB" will only show files with extension `.pdf'. Default is false (0).

Environment variables

COMP_CACHE
If `COMP_CACHE' is defined and points to a writable file, completions will be read from this file. Completions are stored in this file after the first run of `bash_completion_lib'. Default is `~/.bash_completion_lib~'.
COMP_CACHE_VERSION
Version of bash_completion_lib while cache was generated.
Every time bash_completion_lib loads the cache, the value of `COMP_CACHE_VERSION' is compared with the value of `COMP_VERSION'. If the versions differ, the cache is regenerated.
COMP_DIR
Directory of `bash_completion_lib'. This variable is automatically set.
COMP_INSTALL
True (1, default) if real completion must be installed after a first completion. If set to False (0), the default completion handler `comp_load' will not be replaced with the actual completion handler after the first completion.
COMP_LIB
Array version of `COMP_PATH'.
Not used anymore because an array cannot be exported to a bash subshell. Use `COMP_PATH' instead. Deprecated as of bash_completion_lib >= 1.2.6.
COMP_LOAD_DEINIT
True (1, default) if `comp_load' must deinitialize itself after having installed the actual completion handler. Deinitializing means removing variables and functions which are only useful to `comp_load'. If set to False (0), the variables and functions remain defined.
COMP_PATH
String variable containing directories to search for completions, separated by a colon (:). By default `COMP_PATH' contains these directories:
  • $HOME/.bash_completion_lib.d/completions
  • $COMP_DIR/completions
Bash-completion-lib uses the first directory in which a completion is found.
COMP_RESTRICT_BY_EXTENSION
True (1) if restricted completion is enabled, False (0, default) if not. Example: if enabled, `acroread <TAB>' will only show files with a .pdf extension.
COMP_VERSION
Version of bash_completion_lib.
If this variable is non-empty, bash_completion_lib will return without loading any completions. This prevents bash_completion_lib from doing too much work if bash_completion_lib is sourced for a second time within the same shell.
If the value of `COMP_VERSION' differs from the value of `COMP_CACHE_VERSION', the cache will be regenerated.

Automated testing

The download contains an additional test suite. The test suite requires dejagnu, tcl, expect and diff to be installed.

To run the tests:

$ cd test
$ ./runCompletionLib
$ ./runInstall

Considerations

  • The function `have' which checked if a given command is present on the system, has been removed: the run-time loading of completion now prevents bulky functions in memory if we don't need them.
  • The minimum requirements for the completion package is bash-3. No efforts has been made to keep the completions compatible with bash-2.
  • To prevent name collisions, most of the completion install functions are prefixed with `comp_'.

Include

Within bash_completion_lib, the function `comp_load_init' resides in the file `include/comp_load_init', which is loaded the first time a completion is invoked - typically when TAB is pressed. To prevent `comp_load_init' from growing ever larger as completions keep getting added to bash-completion, I decided to move all shared helper functions to an `include' subdirectory. Completions can include shared helper functions with the function-call `comp_include [FILE]...'. This prevents the bash environment from becoming cluttered with unused shared bash-completion helper functions.

One might argue that common functions like `_filedir' should always be available without a call to `comp_include', but I think that as development continues (e.g. bash-4), helper functions come and go and including them specific, eases maintenance.


Known problems

  • See #Implemented workarounds
  • While adding completions from the original bash_completion-20060301-4 (Debian), TABs have changed to spaces (expandtab in vim)
  • Completions for installpkg, upgradepkg and explodepkg are left out
  • Completion for display is defined twice in original bash completion (Debian 20060301-4)

Extending bash_completion_lib

Cdots

Suppose you want to extend bash completions with additonal completions, e.g. the cdots commands (.. ... .... etc.). And you want to install them local so they don't interfere with a future upgrade of bash_completion_lib:

  • download cdots and unpack: tar -xvzf cdots-1.2.1.tar.gz
  • see INSTALL for how to install cdots itself
  • execute the lines below to install cdots bash completion locally for the current user:
bcl add cdots-completion.sh cdots -o 'filenames nospace' \
--alias '.. ... .... ..... ...... ....... ........'

Cdp

Cdproj (cdp) is an easy command to change directory to a predefined set of 'project' directories.

You can add cdp-completion to bash_completion_lib by following these steps:

  • download cdp and unpack: tar -xvzf cdp-version.tar.gz
  • create directory: mkdir -p ~/.bash_completion_lib.d/completions/'complete -o filenames -o nospace'
  • copy cdproj-completion to cdp: cp cdp-completion.inc.sh ~/.bash_completion_lib.d/completions/'complete -o filenames -o nospace'/cdp
  • modify file cdp to contain only the _cdp function – remove the complete command from the end of the file

Implemented workarounds

Problem 1: No `catch-all' complete

Bash has no option to set a `catch-all' completion. A `catch-all' completion could in theory select the right completion run-time. Note that this would only work if problem 2 would be solved as well.

Workaround 1

Bash_completion_split installs the comp_load function for every possible completion.

Solution 1: Add `complete-not-found' handler to bash

Adding bash `complete-not-found' handler.

Ubuntu has this bash extension: command-not-found handler. A similar `complete-not-found' handler would make it possible to install no completions at first, but make the `complete-not-found' handler install the right completion for a command when this command is completed for the first time.

Problem 2: Completion can't access completion meta-options

The bash-completion meta-options cannot be accessed by a completion function. It concerns the `-o COMP-OPTION' options:

  • bashdefault
  • default
  • dirnames
  • filenames
  • nospace

Because these options cannot be modified from within a completion function, and have to be set during installation rather then during invocation, it is impossible to create a generic run-time completion installer.

Being not able to access the completion meta-options, also means you can't fine-grain completions. For example, take scp-completion. When completing scp -F you want to set complete -o filenames in order to escape spaces in directory names. But when completing scp host:path you don't want to set complete -o filenames because it escapes the last space that scp-completion puts at the end of a filename.

Workaround 2

As a workaround - to be able to install a large group of completions at once - `bash_completion_lib' groups completions into directories named after their meta-options:

  • complete -o default
  • complete -o default -o filenames
  • complete -o default -o nospace
  • complete -o dirnames
  • complete -o filenames
  • complete -o filenames -o nospace
  • complete -o nospace

The other directories:

  • complete-cd
  • complete-linux
  • complete-nohostcomplete
  • complete-restrict-by-extension
  • complete-tar

contain a completion initialization file `.complete' which sets meta-options at run-time depending on the user configuration.

Solution 2: Add `compset' bash builtin

Re: Temporarily change completion options
Newsgroup thread that suggests introducing a new compgen/complete builtin which would solve the problem

This `compset' should then be able to - from within a completion - access all the readline completion settings.

Alternatives

Debian package

Debian has taken over maintenance of the original bash_completion.

Alioth: Bash Completion
Bug list
Bash-completion-commits
Mailing list detailing Debian updates to bash_completion
Overview of bash-completion source package
Debian package "bash-completion" source homepage

bash_dyncompletion

Debian Bug report logs - #467231 bash_completion is big and loads slowly; load-by-need proposed
Converter script `bash_dyncompletion' which replaces bash_completion functions with placeholders. These placeholders then do a load-by-need of the actual bash completions functions from the generated $BASH_DYNCOMPLETION_DIR.

Pros

  • the dynamic completion can easily be generated from the original source

Cons

  • the placeholder functions still need all to be defined in bash, consuming resources

See also

Bash Completion on Debian

Bash Completion (semi-) Official Homepage
Homepage
Bash-completion-devel -- Coordination of bash-completion's development
The Debian bash-completion package contains many improvements to bash-completion-20060301. This mailing list coordinates the development.
Overview of bash-completion source package
Debian bash-completion source package homepage
Bugs in package bash-completion
Debian bash-completion buglist
http://wiki.debian.org/Teams/BashCompletion

Catch-all programmable completion handler on Fedora

Bug 475229 - Implement catch-all programmable completion handler
Behdad Esfahbod is writing a bash catch-all completion handler
New feature: Implement catch-all programmable completion handler
The catch-all feature will not be implemented before bash-4.1 according to Chet Ramey on gnu.bash.bug.

Miscellaneous

Working more productively with bash 2.x/3.x
Homepage of the original bash-completion-20060301

Reference

Function/file invocation

   user       
  action            function                 file
-----------   ---------------------   --------------------   
login bash    comp_setup              bash_completion_lib
	      |-- comp_setup_init     include/comp_setup
	      |-- comp_setup_deinit   include/comp_setup

complete      comp_load               bash_completion_lib
(press TAB)   |-- comp_load_init      include/comp_load
	      |-- comp_install        include/comp_install
              |-- comp_load_deinit    include/comp_load

completionrc

This file is loaded the first time a completion is invoked - typically when TAB is pressed

script/generate

Developer script to generate files for a new completion. Usage:

$ script/generate COMMAND FUNCTION COMPLETE_ARGS

Example:

$ script/generate make _make '-o filenames'

script/search_absent_include

Developer script to search for a file/include which is used in a completion but isn't explicitly included via `#comp_include'. The script returns a list of quoted filenames. Typical use:

$ eval vi -- $(script/search_absent_include NAME)

Functions

comp_include

The function `comp_include' resides in the file `completionrc', which is loaded the first time a completion is invoked - typically when TAB is pressed.

Completions can include shared helper functions with the function-call `comp_include [FILE]...'. This prevents the bash environment from becoming cluttered with unused shared bash-completion helper functions.

Test suite

The following test suites are available:

        command                          test objectives
-------------------   -----------------------------------------------------------
runCompletionLib      Tab completions using `bash_completion_lib'
runCompletionOrig     Tab completions using `bash_completion-20060301'
runCompletionDebian   Tab completions using `bash_completion-20060301-4' (Debian)
runInstall            Install `bash_completion_lib'

To run a subset of the tests, e.g. all tests starting with `s':

bash_completion_lib/test/$ ./runCompletionLib $(cd lib/completions && ls s*)

Journal

20080612

RFH: bash-completion -- programmable completion for the bash shell
Request for Help from the Debian Maintainer: Luk Claes. LIsted myself.

20090114

Bash-4.0-rc1 available for FTP
From the release notes:
This document details the changes between this version, bash-4.0-alpha,
and the previous version, bash-3.2-release. 

3.  New Features in Bash 

o.  There is a new `compopt' builtin that allows completion functions to modify
    completion options for existing completions or the completion currently
    being executed.

Comments

blog comments powered by Disqus

Personal tools
Google