Bash completion library
NOTE: Bash-completion-lib will be merged with bash-completion. Because a lot of my attention goes to bash-completion right now, development on bash-completion-lib has slowed down.
I'm trying to turn the bash-completion file of Ian Macdonald (http://www.caliban.org/bash/) into separate files - a bash completion library.
What are the benefits of doing so?
Loading the 260K file is nearing one second to load every time I login. Splitting up the completions into separate files - loaded just-in-time - is much faster, typically 5 times as fast. Splitting up the 10,000 lines into separate files per completion command also eases maintenance.
How does it work?
- All completions run through a central function comp_load. The first time completion is invoked for a specific command, comp_load will load the specific command completion function, run it and install it for subsequent invocations. For example:
$ complete -p alias # Show how `alias' is completed complete -F comp_load alias # First completion uses `comp_load' $ alias <TAB> # Complete on `alias' $ alias ^C # Interrupt by pressing CTRL-C $ complete -p alias # Show how `alias' is completed complete -F _alias alias # Subsequent completions use `_alias'
Bash_completion_lib loads 10 times faster than bash_completion-20060301 (with caching enabled). With caching disabled, bash_completion_lib loads 5 times faster. For more information, see: #Features
- unpack the file:
tar -xvzf <file>
- include bash_completion_lib from within your ~/.bashrc:
- remove a possible cache file:
$ rm ~/.bash_completion_lib.d/cache~
- reload .bashrc:
$ . ~/.bashrc
- bash-completion-lib_1.3.1-1_all.deb (Debian/GNU Linux or Ubuntu package)
- bash_completion_lib-1.3.1.tar.gz (binary), bash_completion_lib-1.3.1.tar.gz.asc (signature)
This project is hosted at http://code.google.com/p/bash-completion-lib/
GOALS REALISATION ------------------------------------------ ----------------------------------------------- (1) Improve bash-completion-20060301: bash_completion_lib-1.3.1: | - improve initial loading speed - loads 10x faster | - improve completions - contains fixes of 20060301-debian4 | - contains fixes of 20060301-ubuntu1 | - split completions into separate files - split into 543 files for 609 completions | - add testsuite - contains a DejaGnu testsuite with >1300 tests | | (2) Patch bash, so that real dynamic loading - bash-4.0 contains a `compopt' builtin | becomes possible: - Fedora has patched bash-3 with a catch-all | - add `complete-not-found' handler to bash programmable completion handler | - add `compset' bash builtin | | (3) Improve bash-completion to do real -- dynamic loading
Bash-completion-lib has a fast algorithm to install completions. During
startup, all completions in the ./complete* subdirectories are converted to
complete comp_load ... commands. By caching the conversion result,
bash-completion-lib reduces this fast startup time even more by 50% (0.20 sec -> 0.10 sec).
Bash-completion-lib automatically sets COMP_CACHE (if not already set) to the following cache file:
After having installed a new version of bash-completion-lib, the cache will automatically be cleared. Bash-completion-lib does this by comparing two environment variables (COMP_VERSION and COMP_CACHE_VERSION) each time it loads the cache file.
To disable caching, set COMP_CACHE to be empty:
Care must be taken to empty the cache file after updates have been made to the ./completions directory. The cache can be emptied with this bash command:
NOTE: The cache need only be reset, if completions have been added, moved or deleted, or when the arguments to `complete' have changed. Other modifications like an improved completion function are automatically loaded - remember: it's only the `complete' installments which are cached. E.g.:
- You moved ./completions/*/cd to another directory.
The cache must be cleared.
- You improved the _cd completion within ./completions/*/cd.
You don't have to clear the cache.
Optional restrict by extension
An environment setting COMP_RESTRICT_BY_EXTENSION lets you decide whether restricted completion is enabled, e.g. whether "acroread TAB" will only show files with extension `.pdf'. Default is false (0).
- If `COMP_CACHE' is defined and points to a writable file, completions will be read from this file. Completions are stored in this file after the first run of `bash_completion_lib'. Default is `~/.bash_completion_lib~'.
- Version of bash_completion_lib while cache was generated.
Every time bash_completion_lib loads the cache, the value of `COMP_CACHE_VERSION' is compared with the value of `COMP_VERSION'. If the versions differ, the cache is regenerated.
- Directory of `bash_completion_lib'. This variable is automatically set.
- True (1, default) if real completion must be installed after a first completion. If set to False (0), the default completion handler `comp_load' will not be replaced with the actual completion handler after the first completion.
- Array version of `COMP_PATH'.
- Not used anymore because an array cannot be exported to a bash subshell. Use `COMP_PATH' instead. Deprecated as of bash_completion_lib >= 1.2.6.
- True (1, default) if `comp_load' must deinitialize itself after having installed the actual completion handler. Deinitializing means removing variables and functions which are only useful to `comp_load'. If set to False (0), the variables and functions remain defined.
- String variable containing directories to search for completions, separated by a colon (:). By default `COMP_PATH' contains these directories:
- Bash-completion-lib uses the first directory in which a completion is found.
- True (1) if restricted completion is enabled, False (0, default) if not. Example: if enabled, `acroread <TAB>' will only show files with a .pdf extension.
- Version of bash_completion_lib.
If this variable is non-empty, bash_completion_lib will return without loading any completions. This prevents bash_completion_lib from doing too much work if bash_completion_lib is sourced for a second time within the same shell.
If the value of `COMP_VERSION' differs from the value of `COMP_CACHE_VERSION', the cache will be regenerated.
The download contains an additional test suite. The test suite requires dejagnu, tcl, expect and diff to be installed.
To run the tests:
$ cd test $ ./runCompletionLib $ ./runInstall
- The function `have' which checked if a given command is present on the system, has been removed: the run-time loading of completion now prevents bulky functions in memory if we don't need them.
- The minimum requirements for the completion package is bash-3. No efforts has been made to keep the completions compatible with bash-2.
- To prevent name collisions, most of the completion install functions are prefixed with `comp_'.
Within bash_completion_lib, the function `comp_load_init' resides in the file `include/comp_load_init', which is loaded the first time a completion is invoked - typically when TAB is pressed. To prevent `comp_load_init' from growing ever larger as completions keep getting added to bash-completion, I decided to move all shared helper functions to an `include' subdirectory. Completions can include shared helper functions with the function-call `comp_include [FILE]...'. This prevents the bash environment from becoming cluttered with unused shared bash-completion helper functions.
One might argue that common functions like `_filedir' should always be available without a call to `comp_include', but I think that as development continues (e.g. bash-4), helper functions come and go and including them specific, eases maintenance.
- See #Implemented workarounds
- While adding completions from the original bash_completion-20060301-4 (Debian), TABs have changed to spaces (expandtab in vim)
- Completions for installpkg, upgradepkg and explodepkg are left out
- Completion for display is defined twice in original bash completion (Debian 20060301-4)
Suppose you want to extend bash completions with additonal completions, e.g. the cdots commands (.. ... .... etc.). And you want to install them local so they don't interfere with a future upgrade of bash_completion_lib:
- download cdots and unpack:
tar -xvzf cdots-1.2.1.tar.gz
- see INSTALL for how to install cdots itself
- execute the lines below to install cdots bash completion locally for the current user:
bcl add cdots-completion.sh cdots -o 'filenames nospace' \ --alias '.. ... .... ..... ...... ....... ........'
Cdproj (cdp) is an easy command to change directory to a predefined set of 'project' directories.
You can add cdp-completion to bash_completion_lib by following these steps:
- download cdp and unpack:
tar -xvzf cdp-version.tar.gz
- create directory:
mkdir -p ~/.bash_completion_lib.d/completions/'complete -o filenames -o nospace'
- copy cdproj-completion to cdp:
cp cdp-completion.inc.sh ~/.bash_completion_lib.d/completions/'complete -o filenames -o nospace'/cdp
- modify file cdp to contain only the _cdp function – remove the complete command from the end of the file
Problem 1: No `catch-all' complete
Bash has no option to set a `catch-all' completion. A `catch-all' completion could in theory select the right completion run-time. Note that this would only work if problem 2 would be solved as well.
Bash_completion_split installs the comp_load function for every possible completion.
Solution 1: Add `complete-not-found' handler to bash
Adding bash `complete-not-found' handler.
Ubuntu has this bash extension: command-not-found handler. A similar `complete-not-found' handler would make it possible to install no completions at first, but make the `complete-not-found' handler install the right completion for a command when this command is completed for the first time.
Problem 2: Completion can't access completion meta-options
The bash-completion meta-options cannot be accessed by a completion function. It concerns the `-o COMP-OPTION' options:
Because these options cannot be modified from within a completion function, and have to be set during installation rather then during invocation, it is impossible to create a generic run-time completion installer.
Being not able to access the completion meta-options, also means you can't fine-grain completions. For example, take scp-completion. When completing scp -F you want to set complete -o filenames in order to escape spaces in directory names. But when completing scp host:path you don't want to set complete -o filenames because it escapes the last space that scp-completion puts at the end of a filename.
As a workaround - to be able to install a large group of completions at once - `bash_completion_lib' groups completions into directories named after their meta-options:
- complete -o default
- complete -o default -o filenames
- complete -o default -o nospace
- complete -o dirnames
- complete -o filenames
- complete -o filenames -o nospace
- complete -o nospace
The other directories:
contain a completion initialization file `.complete' which sets meta-options at run-time depending on the user configuration.
Solution 2: Add `compset' bash builtin
- Re: Temporarily change completion options
- Newsgroup thread that suggests introducing a new compgen/complete builtin which would solve the problem
This `compset' should then be able to - from within a completion - access all the readline completion settings.
Debian has taken over maintenance of the original bash_completion.
- Alioth: Bash Completion
- Bug list
- Mailing list detailing Debian updates to bash_completion
- Overview of bash-completion source package
- Debian package "bash-completion" source homepage
- Debian Bug report logs - #467231 bash_completion is big and loads slowly; load-by-need proposed
- Converter script `bash_dyncompletion' which replaces bash_completion functions with placeholders. These placeholders then do a load-by-need of the actual bash completions functions from the generated $BASH_DYNCOMPLETION_DIR.
- the dynamic completion can easily be generated from the original source
- the placeholder functions still need all to be defined in bash, consuming resources
Bash Completion on Debian
- Bash-completion-devel -- Coordination of bash-completion's development
- The Debian bash-completion package contains many improvements to bash-completion-20060301. This mailing list coordinates the development.
- Overview of bash-completion source package
- Debian bash-completion source package homepage
- Bugs in package bash-completion
- Debian bash-completion buglist
Catch-all programmable completion handler on Fedora
- Bug 475229 - Implement catch-all programmable completion handler
- Behdad Esfahbod is writing a bash catch-all completion handler
- New feature: Implement catch-all programmable completion handler
- The catch-all feature will not be implemented before bash-4.1 according to Chet Ramey on gnu.bash.bug.
- Working more productively with bash 2.x/3.x
- Homepage of the original bash-completion-20060301
user action function file ----------- --------------------- -------------------- login bash comp_setup bash_completion_lib |-- comp_setup_init include/comp_setup |-- comp_setup_deinit include/comp_setup complete comp_load bash_completion_lib (press TAB) |-- comp_load_init include/comp_load |-- comp_install include/comp_install |-- comp_load_deinit include/comp_load
This file is loaded the first time a completion is invoked - typically when TAB is pressed
Developer script to generate files for a new completion. Usage:
$ script/generate COMMAND FUNCTION COMPLETE_ARGS
$ script/generate make _make '-o filenames'
Developer script to search for a file/include which is used in a completion but isn't explicitly included via `#comp_include'. The script returns a list of quoted filenames. Typical use:
$ eval vi -- $(script/search_absent_include NAME)
The function `comp_include' resides in the file `completionrc', which is loaded the first time a completion is invoked - typically when TAB is pressed.
Completions can include shared helper functions with the function-call `comp_include [FILE]...'. This prevents the bash environment from becoming cluttered with unused shared bash-completion helper functions.
The following test suites are available:
command test objectives ------------------- ----------------------------------------------------------- runCompletionLib Tab completions using `bash_completion_lib' runCompletionOrig Tab completions using `bash_completion-20060301' runCompletionDebian Tab completions using `bash_completion-20060301-4' (Debian) runInstall Install `bash_completion_lib'
To run a subset of the tests, e.g. all tests starting with `s':
bash_completion_lib/test/$ ./runCompletionLib $(cd lib/completions && ls s*)
- RFH: bash-completion -- programmable completion for the bash shell
- Request for Help from the Debian Maintainer: Luk Claes. LIsted myself.
- Bash-4.0-rc1 available for FTP
- From the release notes:
This document details the changes between this version, bash-4.0-alpha, and the previous version, bash-3.2-release. 3. New Features in Bash o. There is a new `compopt' builtin that allows completion functions to modify completion options for existing completions or the completion currently being executed.