bash(1)

Expansion

Generator

# generate sequence from n to m
+{n..m}
+# generate sequence from n to m step by s
+{n..m..s}
+
+# expand cartesian product
+{a,b}{c,d}
+

Parameter

# default value
+bar=${foo:-some_val}  # if $foo set, then bar=$foo else bar=some_val
+
+# alternate value
+bar=${foo:+bla $foo}  # if $foo set, then bar="bla $foo" else bar=""
+
+# check param set
+bar=${foo:?msg}  # if $foo set, then bar=$foo else exit and print msg
+
+# indirect
+FOO=foo
+BAR=FOO
+bar=${!BAR}  # deref value of BAR -> bar=$FOO
+
+# prefix
+${foo#prefix}  # remove prefix when expanding $foo
+# suffix
+${foo%suffix}  # remove suffix when expanding $foo
+
+# substitute
+${foo/pattern/string}  # replace pattern with string when expanding foo
+# pattern starts with
+# '/'   replace all occurences of pattern
+# '#'   pattern match at beginning
+# '%'   pattern match at end
+
+# set programmatically with priintf builtin
+printf -v "VAR1" "abc"
+NAME=VAR2
+printf -v "$NAME" "%s" "def"
+

+
Note: prefix/suffix/pattern are expanded as pathnames.
+

Pathname

*           match any string
+?           match any single char
+\\          match backslash
+[abc]       match any char of 'a' 'b' 'c'
+[a-z]       match any char between 'a' - 'z'
+[^ab]       negate, match all not 'a' 'b'
+[:class:]   match any char in class, available:
+              alnum,alpha,ascii,blank,cntrl,digit,graph,lower,
+              print,punct,space,upper,word,xdigit
+

With extglob shell option enabled it is possible to have more powerful +patterns. In the following pattern-list is one ore more patterns separated +by | char.

?(pattern-list)   matches zero or one occurrence of the given patterns
+*(pattern-list)   matches zero or more occurrences of the given patterns
++(pattern-list)   matches one or more occurrences of the given patterns
+@(pattern-list)   matches one of the given patterns
+!(pattern-list)   matches anything except one of the given patterns
+

+
Note: shopt -s extglob/shopt -u extglob to enable/disable extglob +option.
+

I/O redirection

+
Note: The trick with bash I/O redirection is to interpret from left-to-right.
+

# stdout & stderr to file
+command >file 2>&1
+# equivalent
+command &>file
+
+# stderr to stdout & stdout to file
+command 2>&1 >file
+

+
The article Bash One-Liners Explained, Part III: All about +redirections +contains some nice visualization to explain bash redirections.
+

Explanation

j>&i
+

Duplicate fd i to fd j, making j a copy of i. See dup2(2).

Example:

command 2>&1 >file
+

duplicate fd 1 to fd 2, effectively redirecting stderr to stdout
redirect stdout to file

Process substitution (ref)

Process substitution allows to redirect the stdout of multiple processes at +once.

vim -d <(grep foo bar) <(grep foo moose)
+

Command grouping

Execute commands in a group with or without subshell. Can be used to easily +redirect stdout/stderr of all commands in the group into one file.

# Group commands without subshell.
+v=abc ; { v=foo; echo $v; } ; echo $v
+# foo
+# foo
+
+# Group commands with subshell.
+v=abc ; ( v=foo; echo $v; ) ; echo $v
+# foo
+# abc
+

Argument parsing with `getopts`

The getopts builtin uses following global variables:

OPTARG, value of last option argument
OPTIND, index of the next argument to process (user must reset)
OPTERR, display errors if set to 1

getopts <optstring> <param> [<args>]
+

<optstring> specifies the names of supported options, eg f:c +
- f: means -f option with an argument
- c means -c option without an argument
+
<param> specifies a variable name which getopts fills with the last parsed option argument
<args> optionally specify argument string to parse, by default getopts parses $@

Example

#!/bin/bash
+function parse_args() {
+    while getopts "f:c" PARAM; do
+        case $PARAM in
+            f) echo "GOT -f $OPTARG";;
+            c) echo "GOT -c";;
+            *) echo "ERR: print usage"; exit 1;;
+        esac
+    done
+    # users responsibility to reset OPTIND
+    OPTIND=1
+}
+
+parse_args -f xxx -c
+parse_args -f yyy
+

Regular Expressions

Bash supports regular expression matching with the binary operator =~. +The match results can be accessed via the $BASH_REMATCH variable:

${BASH_REMATCH[0]} contains the full match
${BASH_REMATCH[1]} contains match of the first capture group

INPUT='title foo : 1234'
+REGEX='^title (.+) : ([0-9]+)$'
+if [[ $INPUT =~ $REGEX ]]; then
+    echo "${BASH_REMATCH[0]}"    # title foo : 1234
+    echo "${BASH_REMATCH[1]}"    # foo
+    echo "${BASH_REMATCH[2]}"    # 1234
+fi
+

+
Caution: When specifying a regex in the [[ ]] block directly, quotes will be treated as part of the pattern. +[[ $INPUT =~ "foo" ]] will match against "foo" not foo!
+

Completion

The complete builtin is used to interact with the completion system.

complete                    # print currently installed completion handler
+complete -F <func> <cmd>    # install <func> as completion handler for <cmd>
+complete -r <cmd>           # uninstall completion handler for <cmd>
+

Variables available in completion functions:

# in
+$1              # <cmd>
+$2              # current word
+$3              # privous word
+
+COMP_WORDS      # array with current command line words
+COMP_CWORD      # index into COMP_WORDS with current cursor position
+
+# out
+COMPREPLY       # array with possible completions
+

The compgen builtin is used to generate possible matches by comparing word +against words generated by option.

compgen <option> <word>
+
+# usefule options:
+# -W <list>    specify list of possible completions
+# -d           generate list with dirs
+# -f           generate list with files
+# -u           generate list with users
+# -e           generate list with exported variables
+
+# compare "f" against words "foo" "foobar" "bar" and generate matches
+compgen -W "foo foobar bar" "f"
+
+# compare "hom" against file/dir names and generate matches
+compgen -d -f "hom"
+

Example

Skeleton to copy/paste for writing simple completions.

Assume a program foo with the following interface:

foo -c green|red|blue -s low|high -f <file> -h
+

The completion handler could be implemented as follows:

function _foo() {
+    local curr=$2
+    local prev=$3
+
+    local opts="-c -s -f -h"
+    case $prev in
+        -c) COMPREPLY=( $(compgen -W "green red blue" -- $curr) );;
+        -s) COMPREPLY=( $(compgen -W "low high" -- $curr) );;
+        -f) COMPREPLY=( $(compgen -f -- $curr) );;
+        *)  COMPREPLY=( $(compgen -W "$opts" -- $curr) );;
+    esac
+}
+
+complete -F _foo foo
+

+ +

zsh(1)

Keybindings

Change input mode:

bindkey -v              change to vi keymap
+bindkey -e              change to emacs keymap
+

Define key-mappings:

bindkey                 list mappings in current keymap
+bindkey in-str cmd      create mapping for `in-str` to `cmd`
+bindkey -r in-str       remove binding for `in-str`
+
+# C-v <key>             dump <key> code, which can be used in `in-str`
+# zle -l                list all functions for keybindings
+# man zshzle(1)         STANDARD WIDGETS: get description of functions
+

Access edit buffer in zle widget:

$BUFFER       # Entire edit buffer content
+$LBUFFER      # Edit buffer content left to cursor
+$RBUFFER      # Edit buffer content right to cursor
+
+# create zle widget which adds text right of the cursor
+function add-text() {
+    RBUFFER="some text $RBUFFER"
+}
+zle -N add-text
+
+bindkey "^p" add-text
+

Parameter

Default value:

# default value
+echo ${foo:-defval}   # defval
+foo=bar
+echo ${foo:-defval}   # bar
+

Alternative value:

echo ${foo:+altval}   # ''
+foo=bar
+echo ${foo:+altval}   # altval
+

Check variable set, error if not set:

echo ${foo:?msg}      # print `msg` and return errno `1`
+foo=bar
+echo ${foo:?msg}      # bar
+

Sub-string ${var:offset:length}:

foo=abcdef
+echo ${foo:1:3}       # bcd
+

Trim prefix ${var#prefix}:

foo=bar.baz
+echo ${foo#bar}       # .baz
+

Trim suffix ${var%suffix}:

foo=bar.baz
+echo ${foo%.baz}      # bar
+

Substitute pattern ${var/pattern/replace}:

foo=aabbccbbdd
+echo ${foo/bb/XX}    # aaXXccbbdd
+echo ${foo//bb/XX}   # aaXXccXXdd
+# replace prefix
+echo ${foo/#bb/XX}   # aabbccbbdd
+echo ${foo/#aa/XX}   # XXbbccbbdd
+# replace suffix
+echo ${foo/%bb/XX}   # aabbccbbdd
+echo ${foo/%dd/XX}   # aabbccbbXX
+

+
Note: prefix/suffix/pattern are expanded as pathnames.
+

Variables

# Variable with local scope
+local var=val
+
+# Read-only variable
+readonly var=bal
+

Indexed arrays:

arr=(aa bb cc dd)
+echo $arr[1]           # aa
+echo $arr[-1]          # dd
+
+arr+=(ee)
+echo $arr[-1]          # ee
+
+echo $arr[1,3]         # aa bb cc
+

Associative arrays:

typeset -A arr
+arr[x]='aa'
+arr[y]='bb'
+echo $arr[x]           # aa
+

Tied arrays:

typeset -T VEC vec=(1 2 3) '|'
+
+echo $vec              # 1 2 3
+echo $VEC              # 1|2|3
+

Unique arrays (set):

typeset -U vec=(1 2 3)
+
+echo $vec             # 1 2 3
+vec+=(1 2 4)
+echo $vec             # 1 2 3 4
+

Expansion Flags

Join array to string j:sep::

foo=(1 2 3 4)
+echo ${(j:-:)foo}     # 1-2-3-4
+echo ${(j:\n:)foo}    # join with new lines
+

Split string to array s:sep:

foo='1-2-3-4'
+bar=(${(s:-:)foo})    # capture as array
+echo $bar             # 1 2 3 4
+echo $bar[2]          # 2
+

Upper/Lower case string:

foo=aaBB
+echo ${(L)foo}        # aabb
+echo ${(U)foo}        # AABB
+

Key/values in associative arrays:

typeset -A vec; vec[a]='aa'; vec[b]='bb'
+
+echo ${(k)vec}        # a b
+echo ${(v)vec}        # aa bb
+echo ${(kv)vec}       # a aa b bb
+
+# Iterate over key value pairs.
+for k v in ${(kv)vec)}; do ...; done
+

I/O redirections

See bash - I/O redirection

Process substitution

Process substitution allows to redirect the stdout of multiple processes at +once.

vim -d <(grep foo bar) <(grep foo moose)
+

Argument parsing with `zparseopts`

zparseopts [-D] [-E] [-A assoc] specs
+

Arguments are copied into the associative array assoc according to specs. +Each spec is described by an entry as opt[:][=array].

opt is the option without the - char. Passing -f is matched against f +opt, --long is matched against -long.
Using : means the option will take an argument.
The optional =array specifies an alternate storage container where this +option should be stored.

+
Documentation can be found in man zshmodules.
+

Example

#!/bin/zsh
+function test() {
+    zparseopts -D -E -A opts f=flag o: -long:
+    echo "flag $flag"
+    echo "o    $opts[-o]"
+    echo "long $opts[--long]"
+    echo "pos  $1"
+}
+
+test -f -o OPTION --long LONG_OPT POSITIONAL
+
+# Outputs:
+#   flag -f
+#   o    OPTION
+#   long LONG_OPT
+#   pos  POSITIONAL
+

Regular Expressions

Zsh supports regular expression matching with the binary operator =~. +The match results can be accessed via the $MATCH variable and +$match indexed array:

$MATCH contains the full match
$match[1] contains match of the first capture group

INPUT='title foo : 1234'
+REGEX='^title (.+) : ([0-9]+)$'
+if [[ $INPUT =~ $REGEX ]]; then
+    echo "$MATCH"       # title foo : 1234
+    echo "$match[1]"    # foo
+    echo "$match[2]"    # 1234
+fi
+

Completion

Installation

Completion functions are provided via files and need to be placed in a location +covered by $fpath. By convention the completion files are names as _<CMD>.

A completion skeleton for the command foo, stored in _foo

#compdef _foo foo
+
+function _foo() {
+    ...
+}
+

Alternatively one can install a completion function explicitly by calling compdef <FUNC> <CMD>.

Completion Variables

Following variables are available in Completion functions:

$words              # array with command line in words
+$#words             # number words
+$CURRENT            # index into $words for cursor position
+$words[CURRENT-1]   # previous word (relative to cursor position)
+

Completion Functions

_describe simple completion, just words + description
_arguments sophisticated completion, allow to specify actions

Completion with `_describe`

_describe MSG COMP
+

MSG simple string with header message
COMP array of completions where each entry is "opt:description"

function _foo() {
+    local -a opts
+    opts=('bla:desc for bla' 'blu:desc for blu')
+    _describe 'foo-msg' opts
+}
+compdef _foo foo
+
+foo <TAB><TAB>
+ -- foo-msg --
+bla  -- desc for bla
+blu  -- desc for blu
+

Completion with `_arguments`

_arguments SPEC [SPEC...]
+

where SPEC can have one of the following forms:

OPT[DESC]:MSG:ACTION for option flags
N:MSG:ACTION for positional arguments

Available actions

(op1 op2)   list possible matches
+->VAL       set $state=VAL and continue, `$state` can be checked later in switch case
+FUNC        call func to generate matches
+{STR}       evaluate `STR` to generate matches
+

Example

Skeleton to copy/paste for writing simple completions.

Assume a program foo with the following interface:

foo -c green|red|blue -s low|high -f <file> -d <dir> -h
+

The completion handler could be implemented as follows in a file called _foo:

#compdef _foo foo
+
+function _foo_color() {
+    local colors=()
+    colors+=('green:green color')
+    colors+=('red:red color')
+    colors+=('blue:blue color')
+    _describe "color" colors
+}
+
+function _foo() {
+    _arguments                              \
+        "-c[define color]:color:->s_color"  \
+        "-s[select sound]:sound:(low high)" \
+        "-f[select file]:file:_files"       \
+        "-d[select dir]:dir:_files -/"      \
+        "-h[help]"
+
+    case $state in
+        s_color) _foo_color;;
+    esac
+}
+

Example with optional arguments

For this example we assume that the command foo takes at least three optional +arguments such as

foo arg1 arg2 arg3 [argN..]
+

function _foo() {
+    _arguments              \
+        "1:opt 1:(a b c)"   \
+        ":opt next:(d e f)" \
+        "*:opt all:(u v w)"
+}
+

Explanation:

1:MSG:ACTION sets completion for the first optional argument
:MSG:ACTION sets completion for the next optional argument
*:MSG:ACTION sets completion for the optional argument where none of the +previous rules apply, so in our example for arg3, argN...

+
_files is a zsh builtin utility function to complete files/dirs see
+
+
zsh completion functions
+
zsh completion utility functions
+
+

+ +

Notes

Process substitution (ref)

Notes

Notes

Notes