Advanced Bash-Scripting Guide: An in-depth exploration of the art of shell scripting
PrevChapter 9. Variables RevisitedNext

9.3. Parameter Substitution

Manipulating and/or expanding variables

${parameter}

Same as $parameter, i.e., value of the variable parameter. In certain contexts, only the less ambiguous ${parameter} form works.

May be used for concatenating variables with strings.

   1 your_id=${USER}-on-${HOSTNAME}
   2 echo "$your_id"
   3 #
   4 echo "Old \$PATH = $PATH"
   5 PATH=${PATH}:/opt/bin  #Add /opt/bin to $PATH for duration of script.
   6 echo "New \$PATH = $PATH"

${parameter-default}, ${parameter:-default}

If parameter not set, use default.

   1 echo ${username-`whoami`}
   2 # Echoes the result of `whoami`, if variable $username is still unset.

Note

${parameter-default} and ${parameter:-default} are almost equivalent. The extra : makes a difference only when parameter has been declared, but is null. 

   1 #!/bin/bash
   2 # param-sub.sh
   3 
   4 #  Whether a variable has been declared
   5 #+ affects triggering of the default option
   6 #+ even if the variable is null.
   7 
   8 username0=
   9 echo "username0 has been declared, but is set to null."
  10 echo "username0 = ${username0-`whoami`}"
  11 # Will not echo.
  12 
  13 echo
  14 
  15 echo username1 has not been declared.
  16 echo "username1 = ${username1-`whoami`}"
  17 # Will echo.
  18 
  19 username2=
  20 echo "username2 has been declared, but is set to null."
  21 echo "username2 = ${username2:-`whoami`}"
  22 #                            ^
  23 # Will echo because of :- rather than just - in condition test.
  24 # Compare to first instance, above.
  25 
  26 
  27 #
  28 
  29 # Once again:
  30 
  31 variable=
  32 # variable has been declared, but is set to null.
  33 
  34 echo "${variable-0}"    # (no output)
  35 echo "${variable:-1}"   # 1
  36 #               ^
  37 
  38 unset variable
  39 
  40 echo "${variable-2}"    # 2
  41 echo "${variable:-3}"   # 3
  42 
  43 exit 0

The default parameter construct finds use in providing "missing" command-line arguments in scripts.

   1 DEFAULT_FILENAME=generic.data
   2 filename=${1:-$DEFAULT_FILENAME}
   3 #  If not otherwise specified, the following command block operates
   4 #+ on the file "generic.data".
   5 #
   6 #  Commands follow.

See also Example 3-4, Example 28-2, and Example A-6.

Compare this method with using an and list to supply a default command-line argument.

${parameter=default}, ${parameter:=default}

If parameter not set, set it to default.

Both forms nearly equivalent. The : makes a difference only when $parameter has been declared and is null, [1] as above. 

   1 echo ${username=`whoami`}
   2 # Variable "username" is now set to `whoami`.

${parameter+alt_value}, ${parameter:+alt_value}

If parameter set, use alt_value, else use null string.

Both forms nearly equivalent. The : makes a difference only when parameter has been declared and is null, see below.

   1 echo "###### \${parameter+alt_value} ########"
   2 echo
   3 
   4 a=${param1+xyz}
   5 echo "a = $a"      # a =
   6 
   7 param2=
   8 a=${param2+xyz}
   9 echo "a = $a"      # a = xyz
  10 
  11 param3=123
  12 a=${param3+xyz}
  13 echo "a = $a"      # a = xyz
  14 
  15 echo
  16 echo "###### \${parameter:+alt_value} ########"
  17 echo
  18 
  19 a=${param4:+xyz}
  20 echo "a = $a"      # a =
  21 
  22 param5=
  23 a=${param5:+xyz}
  24 echo "a = $a"      # a =
  25 # Different result from   a=${param5+xyz}
  26 
  27 param6=123
  28 a=${param6:+xyz}
  29 echo "a = $a"      # a = xyz

${parameter?err_msg}, ${parameter:?err_msg}

If parameter set, use it, else print err_msg.

Both forms nearly equivalent. The : makes a difference only when parameter has been declared and is null, as above.

Example 9-15. Using parameter substitution and error messages

   1 #!/bin/bash
   2 
   3 #  Check some of the system's environmental variables.
   4 #  This is good preventative maintenance.
   5 #  If, for example, $USER, the name of the person at the console, is not set,
   6 #+ the machine will not recognize you.
   7 
   8 : ${HOSTNAME?} ${USER?} ${HOME?} ${MAIL?}
   9   echo
  10   echo "Name of the machine is $HOSTNAME."
  11   echo "You are $USER."
  12   echo "Your home directory is $HOME."
  13   echo "Your mail INBOX is located in $MAIL."
  14   echo
  15   echo "If you are reading this message,"
  16   echo "critical environmental variables have been set."
  17   echo
  18   echo
  19 
  20 # ------------------------------------------------------
  21 
  22 #  The ${variablename?} construction can also check
  23 #+ for variables set within the script.
  24 
  25 ThisVariable=Value-of-ThisVariable
  26 #  Note, by the way, that string variables may be set
  27 #+ to characters disallowed in their names.
  28 : ${ThisVariable?}
  29 echo "Value of ThisVariable is $ThisVariable".
  30 echo
  31 echo
  32 
  33 
  34 : ${ZZXy23AB?"ZZXy23AB has not been set."}
  35 #  If ZZXy23AB has not been set,
  36 #+ then the script terminates with an error message.
  37 
  38 # You can specify the error message.
  39 # : ${variablename?"ERROR MESSAGE"}
  40 
  41 
  42 # Same result with:    dummy_variable=${ZZXy23AB?}
  43 #                      dummy_variable=${ZZXy23AB?"ZXy23AB has not been set."}
  44 #
  45 #                      echo ${ZZXy23AB?} >/dev/null
  46 
  47 #  Compare these methods of checking whether a variable has been set
  48 #+ with "set -u" . . .
  49 
  50 
  51 
  52 echo "You will not see this message, because script already terminated."
  53 
  54 HERE=0
  55 exit $HERE   # Will NOT exit here.
  56 
  57 # In fact, this script will return an exit status (echo $?) of 1.

Example 9-16. Parameter substitution and "usage" messages

   1 #!/bin/bash
   2 # usage-message.sh
   3 
   4 : ${1?"Usage: $0 ARGUMENT"}
   5 #  Script exits here if command-line parameter absent,
   6 #+ with following error message.
   7 #    usage-message.sh: 1: Usage: usage-message.sh ARGUMENT
   8 
   9 echo "These two lines echo only if command-line parameter given."
  10 echo "command line parameter = \"$1\""
  11 
  12 exit 0  # Will exit here only if command-line parameter present.
  13 
  14 # Check the exit status, both with and without command-line parameter.
  15 # If command-line parameter present, then "$?" is 0.
  16 # If not, then "$?" is 1.

Parameter substitution and/or expansion. The following expressions are the complement to the match in expr string operations (see Example 15-9). These particular ones are used mostly in parsing file path names.

Variable length / Substring removal

${#var}

String length (number of characters in $var). For an array, ${#array} is the length of the first element in the array.

Note

Exceptions:

  • ${#*} and ${#@} give the number of positional parameters.

  • For an array, ${#array[*]} and ${#array[@]} give the number of elements in the array.

Example 9-17. Length of a variable

   1 #!/bin/bash
   2 # length.sh
   3 
   4 E_NO_ARGS=65
   5 
   6 if [ $# -eq 0 ]  # Must have command-line args to demo script.
   7 then
   8   echo "Please invoke this script with one or more command-line arguments."
   9   exit $E_NO_ARGS
  10 fi  
  11 
  12 var01=abcdEFGH28ij
  13 echo "var01 = ${var01}"
  14 echo "Length of var01 = ${#var01}"
  15 # Now, let's try embedding a space.
  16 var02="abcd EFGH28ij"
  17 echo "var02 = ${var02}"
  18 echo "Length of var02 = ${#var02}"
  19 
  20 echo "Number of command-line arguments passed to script = ${#@}"
  21 echo "Number of command-line arguments passed to script = ${#*}"
  22 
  23 exit 0
${var#Pattern}, ${var##Pattern}

Remove from $var the shortest/longest part of $Pattern that matches the front end of $var.

A usage illustration from Example A-7: 
   1 # Function from "days-between.sh" example.
   2 # Strips leading zero(s) from argument passed.
   3 
   4 strip_leading_zero () #  Strip possible leading zero(s)
   5 {                     #+ from argument passed.
   6   return=${1#0}       #  The "1" refers to "$1" -- passed arg.
   7 }                     #  The "0" is what to remove from "$1" -- strips zeros.

Manfred Schwarb's more elaborate variation of the above: 
   1 strip_leading_zero2 () # Strip possible leading zero(s), since otherwise
   2 {                      # Bash will interpret such numbers as octal values.
   3   shopt -s extglob     # Turn on extended globbing.
   4   local val=${1##+(0)} # Use local variable, longest matching series of 0's.
   5   shopt -u extglob     # Turn off extended globbing.
   6   _strip_leading_zero2=${val:-0}
   7                        # If input was 0, return 0 instead of "".
   8 }

Another usage illustration: 
   1 echo `basename $PWD`        # Basename of current working directory.
   2 echo "${PWD##*/}"           # Basename of current working directory.
   3 echo
   4 echo `basename $0`          # Name of script.
   5 echo $0                     # Name of script.
   6 echo "${0##*/}"             # Name of script.
   7 echo
   8 filename=test.data
   9 echo "${filename##*.}"      # data
  10                             # Extension of filename.

${var%Pattern}, ${var%%Pattern}

Remove from $var the shortest/longest part of $Pattern that matches the back end of $var.

Version 2 of Bash added additional options.

Example 9-18. Pattern matching in parameter substitution

   1 #!/bin/bash
   2 # patt-matching.sh
   3 
   4 # Pattern matching  using the # ## % %% parameter substitution operators.
   5 
   6 var1=abcd12345abc6789
   7 pattern1=a*c  # * (wild card) matches everything between a - c.
   8 
   9 echo
  10 echo "var1 = $var1"           # abcd12345abc6789
  11 echo "var1 = ${var1}"         # abcd12345abc6789
  12                               # (alternate form)
  13 echo "Number of characters in ${var1} = ${#var1}"
  14 echo
  15 
  16 echo "pattern1 = $pattern1"   # a*c  (everything between 'a' and 'c')
  17 echo "--------------"
  18 echo '${var1#$pattern1}  =' "${var1#$pattern1}"    #         d12345abc6789
  19 # Shortest possible match, strips out first 3 characters  abcd12345abc6789
  20 #                                     ^^^^^               |-|
  21 echo '${var1##$pattern1} =' "${var1##$pattern1}"   #                  6789      
  22 # Longest possible match, strips out first 12 characters  abcd12345abc6789
  23 #                                    ^^^^^                |----------|
  24 
  25 echo; echo; echo
  26 
  27 pattern2=b*9            # everything between 'b' and '9'
  28 echo "var1 = $var1"     # Still  abcd12345abc6789
  29 echo
  30 echo "pattern2 = $pattern2"
  31 echo "--------------"
  32 echo '${var1%pattern2}  =' "${var1%$pattern2}"     #     abcd12345a
  33 # Shortest possible match, strips out last 6 characters  abcd12345abc6789
  34 #                                     ^^^^                         |----|
  35 echo '${var1%%pattern2} =' "${var1%%$pattern2}"    #     a
  36 # Longest possible match, strips out last 12 characters  abcd12345abc6789
  37 #                                    ^^^^                 |-------------|
  38 
  39 # Remember, # and ## work from the left end (beginning) of string,
  40 #           % and %% work from the right end.
  41 
  42 echo
  43 
  44 exit 0

Example 9-19. Renaming file extensions:

   1 #!/bin/bash
   2 # rfe.sh: Renaming file extensions.
   3 #
   4 #         rfe old_extension new_extension
   5 #
   6 # Example:
   7 # To rename all *.gif files in working directory to *.jpg,
   8 #          rfe gif jpg
   9 
  10 
  11 E_BADARGS=65
  12 
  13 case $# in
  14   0|1)             # The vertical bar means "or" in this context.
  15   echo "Usage: `basename $0` old_file_suffix new_file_suffix"
  16   exit $E_BADARGS  # If 0 or 1 arg, then bail out.
  17   ;;
  18 esac
  19 
  20 
  21 for filename in *.$1
  22 # Traverse list of files ending with 1st argument.
  23 do
  24   mv $filename ${filename%$1}$2
  25   #  Strip off part of filename matching 1st argument,
  26   #+ then append 2nd argument.
  27 done
  28 
  29 exit 0

Variable expansion / Substring replacement

These constructs have been adopted from ksh.

${var:pos}

Variable var expanded, starting from offset pos.

${var:pos:len}

Expansion to a max of len characters of variable var, from offset pos. See Example A-14 for an example of the creative use of this operator.

${var/Pattern/Replacement}

First match of Pattern, within var replaced with Replacement.

If Replacement is omitted, then the first match of Pattern is replaced by nothing, that is, deleted.

${var//Pattern/Replacement}

Global replacement. All matches of Pattern, within var replaced with Replacement.

As above, if Replacement is omitted, then all occurrences of Pattern are replaced by nothing, that is, deleted.

Example 9-20. Using pattern matching to parse arbitrary strings

   1 #!/bin/bash
   2 
   3 var1=abcd-1234-defg
   4 echo "var1 = $var1"
   5 
   6 t=${var1#*-*}
   7 echo "var1 (with everything, up to and including first - stripped out) = $t"
   8 #  t=${var1#*-}  works just the same,
   9 #+ since # matches the shortest string,
  10 #+ and * matches everything preceding, including an empty string.
  11 # (Thanks, Stephane Chazelas, for pointing this out.)
  12 
  13 t=${var1##*-*}
  14 echo "If var1 contains a \"-\", returns empty string...   var1 = $t"
  15 
  16 
  17 t=${var1%*-*}
  18 echo "var1 (with everything from the last - on stripped out) = $t"
  19 
  20 echo
  21 
  22 # -------------------------------------------
  23 path_name=/home/bozo/ideas/thoughts.for.today
  24 # -------------------------------------------
  25 echo "path_name = $path_name"
  26 t=${path_name##/*/}
  27 echo "path_name, stripped of prefixes = $t"
  28 # Same effect as   t=`basename $path_name` in this particular case.
  29 #  t=${path_name%/}; t=${t##*/}   is a more general solution,
  30 #+ but still fails sometimes.
  31 #  If $path_name ends with a newline, then `basename $path_name` will not work,
  32 #+ but the above expression will.
  33 # (Thanks, S.C.)
  34 
  35 t=${path_name%/*.*}
  36 # Same effect as   t=`dirname $path_name`
  37 echo "path_name, stripped of suffixes = $t"
  38 # These will fail in some cases, such as "../", "/foo////", # "foo/", "/".
  39 #  Removing suffixes, especially when the basename has no suffix,
  40 #+ but the dirname does, also complicates matters.
  41 # (Thanks, S.C.)
  42 
  43 echo
  44 
  45 t=${path_name:11}
  46 echo "$path_name, with first 11 chars stripped off = $t"
  47 t=${path_name:11:5}
  48 echo "$path_name, with first 11 chars stripped off, length 5 = $t"
  49 
  50 echo
  51 
  52 t=${path_name/bozo/clown}
  53 echo "$path_name with \"bozo\" replaced  by \"clown\" = $t"
  54 t=${path_name/today/}
  55 echo "$path_name with \"today\" deleted = $t"
  56 t=${path_name//o/O}
  57 echo "$path_name with all o's capitalized = $t"
  58 t=${path_name//o/}
  59 echo "$path_name with all o's deleted = $t"
  60 
  61 exit 0
${var/#Pattern/Replacement}

If prefix of var matches Pattern, then substitute Replacement for Pattern.

${var/%Pattern/Replacement}

If suffix of var matches Pattern, then substitute Replacement for Pattern.

Example 9-21. Matching patterns at prefix or suffix of string

   1 #!/bin/bash
   2 # var-match.sh:
   3 # Demo of pattern replacement at prefix / suffix of string.
   4 
   5 v0=abc1234zip1234abc    # Original variable.
   6 echo "v0 = $v0"         # abc1234zip1234abc
   7 echo
   8 
   9 # Match at prefix (beginning) of string.
  10 v1=${v0/#abc/ABCDEF}    # abc1234zip1234abc
  11                         # |-|
  12 echo "v1 = $v1"         # ABCDEF1234zip1234abc
  13                         # |----|
  14 
  15 # Match at suffix (end) of string.
  16 v2=${v0/%abc/ABCDEF}    # abc1234zip123abc
  17                         #              |-|
  18 echo "v2 = $v2"         # abc1234zip1234ABCDEF
  19                         #               |----|
  20 
  21 echo
  22 
  23 #  ----------------------------------------------------
  24 #  Must match at beginning / end of string,
  25 #+ otherwise no replacement results.
  26 #  ----------------------------------------------------
  27 v3=${v0/#123/000}       # Matches, but not at beginning.
  28 echo "v3 = $v3"         # abc1234zip1234abc
  29                         # NO REPLACEMENT.
  30 v4=${v0/%123/000}       # Matches, but not at end.
  31 echo "v4 = $v4"         # abc1234zip1234abc
  32                         # NO REPLACEMENT.
  33 
  34 exit 0
${!varprefix*}, ${!varprefix@}

Matches all previously declared variables beginning with varprefix. 
   1 xyz23=whatever
   2 xyz24=
   3 
   4 a=${!xyz*}      # Expands to names of declared variables beginning with "xyz".
   5 echo "a = $a"   # a = xyz23 xyz24
   6 a=${!xyz@}      # Same as above.
   7 echo "a = $a"   # a = xyz23 xyz24
   8 
   9 # Bash, version 2.04, adds this feature.

Notes

[1]

If $parameter is null in a non-interactive script, it will terminate with a 127 exit status (the Bash error code for "command not found").

PrevHomeNext
Manipulating StringsUpTyping variables: declare or typeset