Bash completion library
NOTE: Bash-completion-lib will be merged with bash-completion. Because a lot of my attention goes to bash-completion right now, development on bash-completion-lib has slowed down.
I'm trying to turn the bash-completion file of Ian Macdonald (http://www.caliban.org/bash/) into separate files - a bash completion library.
What are the benefits of doing so?
Loading the 260K file is nearing one second to load every time I login. Splitting up the completions into separate files - loaded just-in-time - is much faster, typically 5 times as fast. Splitting up the 10,000 lines into separate files per completion command also eases maintenance.
Contents
How does it work?
- All completions run through a central function comp_load. The first time completion is invoked for a specific command, comp_load will load the specific command completion function, run it and install it for subsequent invocations. For example:
$ complete -p alias # Show how `alias' is completed complete -F comp_load alias # First completion uses `comp_load' $ alias <TAB> # Complete on `alias' $ alias ^C # Interrupt by pressing CTRL-C $ complete -p alias # Show how `alias' is completed complete -F _alias alias # Subsequent completions use `_alias'
Bash_completion_lib loads 10 times faster than bash_completion-20060301 (with caching enabled). With caching disabled, bash_completion_lib loads 5 times faster. For more information, see: #Features
Install
- unpack the file:
tar -xvzf <file> - include bash_completion_lib from within your ~/.bashrc:
. bash_completion_lib-1.3.1/bash_completion_lib
- remove a possible cache file:
$ rm ~/.bash_completion_lib.d/cache~
- reload .bashrc:
$ . ~/.bashrc
Download
- bash-completion-lib_1.3.1-1_all.deb (Debian/GNU Linux or Ubuntu package)
- bash_completion_lib-1.3.1.tar.gz (binary), bash_completion_lib-1.3.1.tar.gz.asc (signature)
See also:
Subversion repository
This project is hosted at http://code.google.com/p/bash-completion-lib/
Roadmap
GOALS REALISATION
------------------------------------------ -----------------------------------------------
(1) Improve bash-completion-20060301: bash_completion_lib-1.3.1:
| - improve initial loading speed - loads 10x faster
| - improve completions - contains fixes of 20060301-debian4
| - contains fixes of 20060301-ubuntu1
| - split completions into separate files - split into 543 files for 609 completions
| - add testsuite - contains a DejaGnu testsuite with >1300 tests
|
|
(2) Patch bash, so that real dynamic loading - bash-4.0 contains a `compopt' builtin
| becomes possible: - Fedora has patched bash-3 with a catch-all
| - add `complete-not-found' handler to bash programmable completion handler
| - add `compset' bash builtin
|
|
(3) Improve bash-completion to do real --
dynamic loading
Features
Cache
Bash-completion-lib has a fast algorithm to install completions. During
startup, all completions in the ./complete* subdirectories are converted to
complete comp_load ... commands. By caching the conversion result,
bash-completion-lib reduces this fast startup time even more by 50% (0.20 sec -> 0.10 sec).
Bash-completion-lib automatically sets COMP_CACHE (if not already set) to the following cache file:
COMP_CACHE=$HOME/.bash_completion_lib.d/cache~
After having installed a new version of bash-completion-lib, the cache will automatically be cleared. Bash-completion-lib does this by comparing two environment variables (COMP_VERSION and COMP_CACHE_VERSION) each time it loads the cache file.
To disable caching, set COMP_CACHE to be empty:
COMP_CACHE=
Care must be taken to empty the cache file after updates have been made to the ./completions directory. The cache can be emptied with this bash command:
> $COMP_CACHE
NOTE: The cache need only be reset, if completions have been added, moved or deleted, or when the arguments to `complete' have changed. Other modifications like an improved completion function are automatically loaded - remember: it's only the `complete' installments which are cached. E.g.:
- You moved ./completions/*/cd to another directory.
The cache must be cleared. - You improved the _cd completion within ./completions/*/cd.
You don't have to clear the cache.
Optional restrict by extension
An environment setting COMP_RESTRICT_BY_EXTENSION lets you decide whether restricted completion is enabled, e.g. whether "acroread TAB" will only show files with extension `.pdf'. Default is false (0).
Environment variables
- COMP_CACHE
- If `COMP_CACHE' is defined and points to a writable file, completions will be read from this file. Completions are stored in this file after the first run of `bash_completion_lib'. Default is `~/.bash_completion_lib~'.
- COMP_CACHE_VERSION
- Version of bash_completion_lib while cache was generated.
Every time bash_completion_lib loads the cache, the value of `COMP_CACHE_VERSION' is compared with the value of `COMP_VERSION'. If the versions differ, the cache is regenerated.
- COMP_DIR
- Directory of `bash_completion_lib'. This variable is automatically set.
- COMP_INSTALL
- True (1, default) if real completion must be installed after a first completion. If set to False (0), the default completion handler `comp_load' will not be replaced with the actual completion handler after the first completion.
- COMP_LIB
- Array version of `COMP_PATH'.
- Not used anymore because an array cannot be exported to a bash subshell. Use `COMP_PATH' instead. Deprecated as of bash_completion_lib >= 1.2.6.
- COMP_LOAD_DEINIT
- True (1, default) if `comp_load' must deinitialize itself after having installed the actual completion handler. Deinitializing means removing variables and functions which are only useful to `comp_load'. If set to False (0), the variables and functions remain defined.
- COMP_PATH
- String variable containing directories to search for completions, separated by a colon (:). By default `COMP_PATH' contains these directories:
- $HOME/.bash_completion_lib.d/completions
- $COMP_DIR/completions
- Bash-completion-lib uses the first directory in which a completion is found.
- COMP_RESTRICT_BY_EXTENSION
- True (1) if restricted completion is enabled, False (0, default) if not. Example: if enabled, `acroread <TAB>' will only show files with a .pdf extension.
- COMP_VERSION
- Version of bash_completion_lib.
If this variable is non-empty, bash_completion_lib will return without loading any completions. This prevents bash_completion_lib from doing too much work if bash_completion_lib is sourced for a second time within the same shell.
If the value of `COMP_VERSION' differs from the value of `COMP_CACHE_VERSION', the cache will be regenerated.
Automated testing
The download contains an additional test suite. The test suite requires dejagnu, tcl, expect and diff to be installed.
To run the tests:
$ cd test $ ./runCompletionLib $ ./runInstall
Considerations
- The function `have' which checked if a given command is present on the system, has been removed: the run-time loading of completion now prevents bulky functions in memory if we don't need them.
- The minimum requirements for the completion package is bash-3. No efforts has been made to keep the completions compatible with bash-2.
- To prevent name collisions, most of the completion install functions are prefixed with `comp_'.
Include
Within bash_completion_lib, the function `comp_load_init' resides in the file `include/comp_load_init', which is loaded the first time a completion is invoked - typically when TAB is pressed. To prevent `comp_load_init' from growing ever larger as completions keep getting added to bash-completion, I decided to move all shared helper functions to an `include' subdirectory. Completions can include shared helper functions with the function-call `comp_include [FILE]...'. This prevents the bash environment from becoming cluttered with unused shared bash-completion helper functions.
One might argue that common functions like `_filedir' should always be available without a call to `comp_include', but I think that as development continues (e.g. bash-4), helper functions come and go and including them specific, eases maintenance.
Known problems
- See #Implemented workarounds
- While adding completions from the original bash_completion-20060301-4 (Debian), TABs have changed to spaces (expandtab in vim)
- Completions for installpkg, upgradepkg and explodepkg are left out
- Completion for display is defined twice in original bash completion (Debian 20060301-4)
Extending bash_completion_lib
Cdots
Suppose you want to extend bash completions with additonal completions, e.g. the cdots commands (.. ... .... etc.). And you want to install them local so they don't interfere with a future upgrade of bash_completion_lib:
- download cdots and unpack:
tar -xvzf cdots-1.2.1.tar.gz - see INSTALL for how to install cdots itself
- execute the lines below to install cdots bash completion locally for the current user:
bcl add cdots-completion.sh cdots -o 'filenames nospace' \ --alias '.. ... .... ..... ...... ....... ........'
Cdp
Cdproj (cdp) is an easy command to change directory to a predefined set of 'project' directories.
You can add cdp-completion to bash_completion_lib by following these steps:
- download cdp and unpack:
tar -xvzf cdp-version.tar.gz - create directory:
mkdir -p ~/.bash_completion_lib.d/completions/'complete -o filenames -o nospace' - copy cdproj-completion to cdp:
cp cdp-completion.inc.sh ~/.bash_completion_lib.d/completions/'complete -o filenames -o nospace'/cdp - modify file cdp to contain only the _cdp function – remove the complete command from the end of the file
Implemented workarounds
Problem 1: No `catch-all' complete
Bash has no option to set a `catch-all' completion. A `catch-all' completion could in theory select the right completion run-time. Note that this would only work if problem 2 would be solved as well.
Workaround 1
Bash_completion_split installs the comp_load function for every possible completion.
Solution 1: Add `complete-not-found' handler to bash
Adding bash `complete-not-found' handler.
Ubuntu has this bash extension: command-not-found handler. A similar `complete-not-found' handler would make it possible to install no completions at first, but make the `complete-not-found' handler install the right completion for a command when this command is completed for the first time.
Problem 2: Completion can't access completion meta-options
The bash-completion meta-options cannot be accessed by a completion function. It concerns the `-o COMP-OPTION' options:
- bashdefault
- default
- dirnames
- filenames
- nospace
Because these options cannot be modified from within a completion function, and have to be set during installation rather then during invocation, it is impossible to create a generic run-time completion installer.
Being not able to access the completion meta-options, also means you can't fine-grain completions. For example, take scp-completion. When completing scp -F you want to set complete -o filenames in order to escape spaces in directory names. But when completing scp host:path you don't want to set complete -o filenames because it escapes the last space that scp-completion puts at the end of a filename.
Workaround 2
As a workaround - to be able to install a large group of completions at once - `bash_completion_lib' groups completions into directories named after their meta-options:
- complete -o default
- complete -o default -o filenames
- complete -o default -o nospace
- complete -o dirnames
- complete -o filenames
- complete -o filenames -o nospace
- complete -o nospace
The other directories:
- complete-cd
- complete-linux
- complete-nohostcomplete
- complete-restrict-by-extension
- complete-tar
contain a completion initialization file `.complete' which sets meta-options at run-time depending on the user configuration.
Solution 2: Add `compset' bash builtin
- Re: Temporarily change completion options
- Newsgroup thread that suggests introducing a new compgen/complete builtin which would solve the problem
This `compset' should then be able to - from within a completion - access all the readline completion settings.
Alternatives
Debian package
Debian has taken over maintenance of the original bash_completion.
- Alioth: Bash Completion
- Bug list
- Bash-completion-commits
- Mailing list detailing Debian updates to bash_completion
- Overview of bash-completion source package
- Debian package "bash-completion" source homepage
bash_dyncompletion
- Debian Bug report logs - #467231 bash_completion is big and loads slowly; load-by-need proposed
- Converter script `bash_dyncompletion' which replaces bash_completion functions with placeholders. These placeholders then do a load-by-need of the actual bash completions functions from the generated $BASH_DYNCOMPLETION_DIR.
Pros
- the dynamic completion can easily be generated from the original source
Cons
- the placeholder functions still need all to be defined in bash, consuming resources
See also
Bash Completion on Debian
- Bash-completion-devel -- Coordination of bash-completion's development
- The Debian bash-completion package contains many improvements to bash-completion-20060301. This mailing list coordinates the development.
- Overview of bash-completion source package
- Debian bash-completion source package homepage
- Bugs in package bash-completion
- Debian bash-completion buglist
Catch-all programmable completion handler on Fedora
- Bug 475229 - Implement catch-all programmable completion handler
- Behdad Esfahbod is writing a bash catch-all completion handler
- New feature: Implement catch-all programmable completion handler
- The catch-all feature will not be implemented before bash-4.1 according to Chet Ramey on gnu.bash.bug.
Miscellaneous
- Working more productively with bash 2.x/3.x
- Homepage of the original bash-completion-20060301
Reference
Function/file invocation
user
action function file
----------- --------------------- --------------------
login bash comp_setup bash_completion_lib
|-- comp_setup_init include/comp_setup
|-- comp_setup_deinit include/comp_setup
complete comp_load bash_completion_lib
(press TAB) |-- comp_load_init include/comp_load
|-- comp_install include/comp_install
|-- comp_load_deinit include/comp_load
completionrc
This file is loaded the first time a completion is invoked - typically when TAB is pressed
script/generate
Developer script to generate files for a new completion. Usage:
$ script/generate COMMAND FUNCTION COMPLETE_ARGS
Example:
$ script/generate make _make '-o filenames'
script/search_absent_include
Developer script to search for a file/include which is used in a completion but isn't explicitly included via `#comp_include'. The script returns a list of quoted filenames. Typical use:
$ eval vi -- $(script/search_absent_include NAME)
Functions
comp_include
The function `comp_include' resides in the file `completionrc', which is loaded the first time a completion is invoked - typically when TAB is pressed.
Completions can include shared helper functions with the function-call `comp_include [FILE]...'. This prevents the bash environment from becoming cluttered with unused shared bash-completion helper functions.
Test suite
The following test suites are available:
command test objectives
------------------- -----------------------------------------------------------
runCompletionLib Tab completions using `bash_completion_lib'
runCompletionOrig Tab completions using `bash_completion-20060301'
runCompletionDebian Tab completions using `bash_completion-20060301-4' (Debian)
runInstall Install `bash_completion_lib'
To run a subset of the tests, e.g. all tests starting with `s':
bash_completion_lib/test/$ ./runCompletionLib $(cd lib/completions && ls s*)
Journal
20080612
- RFH: bash-completion -- programmable completion for the bash shell
- Request for Help from the Debian Maintainer: Luk Claes. LIsted myself.
20090114
- Bash-4.0-rc1 available for FTP
- From the release notes:
This document details the changes between this version, bash-4.0-alpha,
and the previous version, bash-3.2-release.
3. New Features in Bash
o. There is a new `compopt' builtin that allows completion functions to modify
completion options for existing completions or the completion currently
being executed.
