Arrays

Advanced Bash-Scripting Guide:
Prev |  | Next

# Chapter 27. Arrays

Newer versions of Bash support one-dimensional arrays. Array elements may be initialized with the `variable[xx]` notation. Alternatively, a script may introduce the entire array by an explicit `declare -a variable` statement. To dereference (retrieve the contents of) an array element, use curly bracket notation, that is, `${element[xx]}`.

Example 27-1. Simple array usage

    #!/bin/bash

    area[11]=23
    area[13]=37
    area[51]=UFOs

    #  Array members need not be consecutive or contiguous.

    #  Some members of the array can be left uninitialized.
    #  Gaps in the array are okay.
    #  In fact, arrays with sparse data ("sparse arrays")
    #+ are useful in spreadsheet-processing software.

    echo -n "area[11] = "
    echo ${area[11]}    #  {curly brackets} needed.

    echo -n "area[13] = "
    echo ${area[13]}

    echo "Contents of area[51] are ${area[51]}."

    # Contents of uninitialized array variable print blank (null variable).
    echo -n "area[43] = "
    echo ${area[43]}
    echo "(area[43] unassigned)"

    echo

    # Sum of two array variables assigned to third
    area[5]=`expr ${area[11]} + ${area[13]}`
    echo "area[5] = area[11] + area[13]"
    echo -n "area[5] = "
    echo ${area[5]}

    area[6]=`expr ${area[11]} + ${area[51]}`
    echo "area[6] = area[11] + area[51]"
    echo -n "area[6] = "
    echo ${area[6]}
    # This fails because adding an integer to a string is not permitted.

    echo; echo; echo

    # -----------------------------------------------------------------
    # Another array, "area2".
    # Another way of assigning array variables...
    # array_name=( XXX YYY ZZZ ... )

    area2=( zero one two three four )

    echo -n "area2[0] = "
    echo ${area2[0]}
    # Aha, zero-based indexing (first element of array is [0], not [1]).

    echo -n "area2[1] = "
    echo ${area2[1]}    # [1] is second element of array.
    # -----------------------------------------------------------------

    echo; echo; echo

    # -----------------------------------------------
    # Yet another array, "area3".
    # Yet another way of assigning array variables...
    # array_name=([xx]=XXX [yy]=YYY ...)

    area3=([17]=seventeen [24]=twenty-four)

    echo -n "area3[17] = "
    echo ${area3[17]}

    echo -n "area3[24] = "
    echo ${area3[24]}
    # -----------------------------------------------

    exit 0

As we have seen, a convenient way of initializing an entire array is the `array=( element1 element2 ... elementN )` notation.

    base64_charset=( {A..Z} {a..z} {0..9} + / = )
                   #  Using extended brace expansion
                   #+ to initialize the elements of the array.
                   #  Excerpted from vladz's "base64.sh" script
                   #+ in the "Contributed Scripts" appendix.

Example 27-2. Formatting a poem

    #!/bin/bash
    # poem.sh: Pretty-prints one of the ABS Guide author's favorite poems.

    # Lines of the poem (single stanza).
    Line[1]="I do not know which to prefer,"
    Line[2]="The beauty of inflections"
    Line[3]="Or the beauty of innuendoes,"
    Line[4]="The blackbird whistling"
    Line[5]="Or just after."
    # Note that quoting permits embedding whitespace.

    # Attribution.
    Attrib[1]=" Wallace Stevens"
    Attrib[2]="\"Thirteen Ways of Looking at a Blackbird\""
    # This poem is in the Public Domain (copyright expired).

    echo

    tput bold   # Bold print.

    for index in 1 2 3 4 5    # Five lines.
    do
      printf "     %s\n" "${Line[index]}"
    done

    for index in 1 2          # Two attribution lines.
    do
      printf "          %s\n" "${Attrib[index]}"
    done

    tput sgr0   # Reset terminal.
                # See 'tput' docs.

    echo

    exit 0

    # Exercise:
    # --------
    # Modify this script to pretty-print a poem from a text data file.

Array variables have a syntax all their own, and even standard Bash commands and operators have special options adapted for array use.

Example 27-3. Various array operations

    #!/bin/bash
    # array-ops.sh: More fun with arrays.

    array=( zero one two three four five )
    # Element 0   1   2    3     4    5

    echo ${array[0]}       #  zero
    echo ${array:0}        #  zero
                           #  Parameter expansion of first element,
                           #+ starting at position # 0 (1st character).
    echo ${array:1}        #  ero
                           #  Parameter expansion of first element,
                           #+ starting at position # 1 (2nd character).

    echo "--------------"

    echo ${#array[0]}      #  4
                           #  Length of first element of array.
    echo ${#array}         #  4
                           #  Length of first element of array.
                           #  (Alternate notation)

    echo ${#array[1]}      #  3
                           #  Length of second element of array.
                           #  Arrays in Bash have zero-based indexing.

    echo ${#array[*]}      #  6
                           #  Number of elements in array.
    echo ${#array[@]}      #  6
                           #  Number of elements in array.

    echo "--------------"

    array2=( [0]="first element" [1]="second element" [3]="fourth element" )
    #            ^     ^       ^     ^      ^       ^     ^      ^       ^
    # Quoting permits embedding whitespace within individual array elements.

    echo ${array2[0]}      # first element
    echo ${array2[1]}      # second element
    echo ${array2[2]}      #
                           # Skipped in initialization, and therefore null.
    echo ${array2[3]}      # fourth element
    echo ${#array2[0]}     # 13    (length of first element)
    echo ${#array2[*]}     # 3     (number of elements in array)

    exit

Many of the standard string operations work on arrays.

Example 27-4. String operations on arrays

    #!/bin/bash
    # array-strops.sh: String operations on arrays.

    # Script by Michael Zick.
    # Used in ABS Guide with permission.
    # Fixups: 05 May 08, 04 Aug 08.

    #  In general, any string operation using the ${name ... } notation
    #+ can be applied to all string elements in an array,
    #+ with the ${name[@] ... } or ${name[*] ...} notation.

    arrayZ=( one two three four five five )

    echo

    # Trailing Substring Extraction
    echo ${arrayZ[@]:0}     # one two three four five five
    #                ^        All elements.

    echo ${arrayZ[@]:1}     # two three four five five
    #                ^        All elements following element[0].

    echo ${arrayZ[@]:1:2}   # two three
    #                  ^      Only the two elements after element[0].

    echo "---------"

    # Substring Removal

    # Removes shortest match from front of string(s).

    echo ${arrayZ[@]#f*r}   # one two three five five
    #               ^       # Applied to all elements of the array.
                            # Matches "four" and removes it.

    # Longest match from front of string(s)
    echo ${arrayZ[@]##t*e}  # one two four five five
    #               ^^      # Applied to all elements of the array.
                            # Matches "three" and removes it.

    # Shortest match from back of string(s)
    echo ${arrayZ[@]%h*e}   # one two t four five five
    #               ^       # Applied to all elements of the array.
                            # Matches "hree" and removes it.

    # Longest match from back of string(s)
    echo ${arrayZ[@]%%t*e}  # one two four five five
    #               ^^      # Applied to all elements of the array.
                            # Matches "three" and removes it.

    echo "----------------------"

    # Substring Replacement

    # Replace first occurrence of substring with replacement.
    echo ${arrayZ[@]/fiv/XYZ}   # one two three four XYZe XYZe
    #               ^           # Applied to all elements of the array.

    # Replace all occurrences of substring.
    echo ${arrayZ[@]//iv/YY}    # one two three four fYYe fYYe
                                # Applied to all elements of the array.

    # Delete all occurrences of substring.
    # Not specifing a replacement defaults to 'delete' ...
    echo ${arrayZ[@]//fi/}      # one two three four ve ve
    #               ^^          # Applied to all elements of the array.

    # Replace front-end occurrences of substring.
    echo ${arrayZ[@]/#fi/XY}    # one two three four XYve XYve
    #                ^          # Applied to all elements of the array.

    # Replace back-end occurrences of substring.
    echo ${arrayZ[@]/%ve/ZZ}    # one two three four fiZZ fiZZ
    #                ^          # Applied to all elements of the array.

    echo ${arrayZ[@]/%o/XX}     # one twXX three four five five
    #                ^          # Why?

    echo "-----------------------------"

    replacement() {
        echo -n "!!!"
    }

    echo ${arrayZ[@]/%e/$(replacement)}
    #                ^  ^^^^^^^^^^^^^^
    # on!!! two thre!!! four fiv!!! fiv!!!
    # The stdout of replacement() is the replacement string.
    # Q.E.D: The replacement action is, in effect, an 'assignment.'

    echo "------------------------------------"

    #  Accessing the "for-each":
    echo ${arrayZ[@]//*/$(replacement optional_arguments)}
    #                ^^ ^^^^^^^^^^^^^
    # !!! !!! !!! !!! !!! !!!

    #  Now, if Bash would only pass the matched string
    #+ to the function being called . . .

    echo

    exit 0

    #  Before reaching for a Big Hammer -- Perl, Python, or all the rest --
    #  recall:
    #    $( ... ) is command substitution.
    #    A function runs as a sub-process.
    #    A function writes its output (if echo-ed) to stdout.
    #    Assignment, in conjunction with "echo" and command substitution,
    #+   can read a function's stdout.
    #    The name[@] notation specifies (the equivalent of) a "for-each"
    #+   operation.
    #  Bash is more powerful than you think!

Command substitution can construct the individual elements of an array.

Example 27-5. Loading the contents of a script into an array

    #!/bin/bash
    # script-array.sh: Loads this script into an array.
    # Inspired by an e-mail from Chris Martin (thanks!).

    script_contents=( $(cat "$0") )  #  Stores contents of this script ($0)
                                     #+ in an array.

    for element in $(seq 0 $((${#script_contents[@]} - 1)))
      do                #  ${#script_contents[@]}
                        #+ gives number of elements in the array.
                        #
                        #  Question:
                        #  Why is  seq 0  necessary?
                        #  Try changing it to seq 1.
      echo -n "${script_contents[$element]}"
                        # List each field of this script on a single line.
    # echo -n "${script_contents[element]}" also works because of ${ ... }.
      echo -n " -- "    # Use " -- " as a field separator.
    done

    echo

    exit 0

    # Exercise:
    # --------
    #  Modify this script so it lists itself
    #+ in its original format,
    #+ complete with whitespace, line breaks, etc.

In an array context, some Bash builtins have a slightly altered meaning. For example, unset deletes array elements, or even an entire array.

Example 27-6. Some special properties of arrays

    #!/bin/bash

    declare -a colors
    #  All subsequent commands in this script will treat
    #+ the variable "colors" as an array.

    echo "Enter your favorite colors (separated from each other by a space)."

    read -a colors    # Enter at least 3 colors to demonstrate features below.
    #  Special option to 'read' command,
    #+ allowing assignment of elements in an array.

    echo

    element_count=${#colors[@]}
    # Special syntax to extract number of elements in array.
    #     element_count=${#colors[*]} works also.
    #
    #  The "@" variable allows word splitting within quotes
    #+ (extracts variables separated by whitespace).
    #
    #  This corresponds to the behavior of "$@" and "$*"
    #+ in positional parameters.

    index=0

    while [ "$index" -lt "$element_count" ]
    do    # List all the elements in the array.
      echo ${colors[$index]}
      #    ${colors[index]} also works because it's within ${ ... } brackets.
      let "index = $index + 1"
      # Or:
      #    ((index++))
    done
    # Each array element listed on a separate line.
    # If this is not desired, use  echo -n "${colors[$index]} "
    #
    # Doing it with a "for" loop instead:
    #   for i in "${colors[@]}"
    #   do
    #     echo "$i"
    #   done
    # (Thanks, S.C.)

    echo

    # Again, list all the elements in the array, but using a more elegant method.
      echo ${colors[@]}          # echo ${colors[*]} also works.

    echo

    # The "unset" command deletes elements of an array, or entire array.
    unset colors[1]              # Remove 2nd element of array.
                                 # Same effect as   colors[1]=
    echo  ${colors[@]}           # List array again, missing 2nd element.

    unset colors                 # Delete entire array.
                                 #  unset colors[*] and
                                 #+ unset colors[@] also work.
    echo; echo -n "Colors gone."
    echo ${colors[@]}            # List array again, now empty.

    exit 0

As seen in the previous example, either ${array_name[@]} or ${array_name[*]} refers to all the elements of the array. Similarly, to get a count of the number of elements in an array, use either ${#array_name[@]} or ${#array_name[*]}. ${#array_name} is the length (number of characters) of ${array_name[0]}, the first element of the array.

Example 27-7. Of empty arrays and empty elements

    #!/bin/bash
    # empty-array.sh

    #  Thanks to Stephane Chazelas for the original example,
    #+ and to Michael Zick and Omair Eshkenazi, for extending it.
    #  And to Nathan Coulter for clarifications and corrections.

    # An empty array is not the same as an array with empty elements.

      array0=( first second third )
      array1=( '' )   # "array1" consists of one empty element.
      array2=( )      # No elements . . . "array2" is empty.
      array3=(   )    # What about this array?

    echo
    ListArray()
    {
    echo
    echo "Elements in array0:  ${array0[@]}"
    echo "Elements in array1:  ${array1[@]}"
    echo "Elements in array2:  ${array2[@]}"
    echo "Elements in array3:  ${array3[@]}"
    echo
    echo "Length of first element in array0 = ${#array0}"
    echo "Length of first element in array1 = ${#array1}"
    echo "Length of first element in array2 = ${#array2}"
    echo "Length of first element in array3 = ${#array3}"
    echo
    echo "Number of elements in array0 = ${#array0[*]}"  # 3
    echo "Number of elements in array1 = ${#array1[*]}"  # 1  (Surprise!)
    echo "Number of elements in array2 = ${#array2[*]}"  # 0
    echo "Number of elements in array3 = ${#array3[*]}"  # 0
    }

    # ===================================================================

    ListArray

    # Try extending those arrays.

    # Adding an element to an array.
    array0=( "${array0[@]}" "new1" )
    array1=( "${array1[@]}" "new1" )
    array2=( "${array2[@]}" "new1" )
    array3=( "${array3[@]}" "new1" )

    ListArray

    # or
    array0[${#array0[*]}]="new2"
    array1[${#array1[*]}]="new2"
    array2[${#array2[*]}]="new2"
    array3[${#array3[*]}]="new2"

    ListArray

    # When extended as above, arrays are 'stacks' ...
    # Above is the 'push' ...
    # The stack 'height' is:
    height=${#array2[@]}
    echo
    echo "Stack height for array2 = $height"

    # The 'pop' is:
    unset array2[${#array2[@]}-1]   #  Arrays are zero-based,
    height=${#array2[@]}            #+ which means first element has index 0.
    echo
    echo "POP"
    echo "New stack height for array2 = $height"

    ListArray

    # List only 2nd and 3rd elements of array0.
    from=1		    # Zero-based numbering.
    to=2
    array3=( ${array0[@]:1:2} )
    echo
    echo "Elements in array3:  ${array3[@]}"

    # Works like a string (array of characters).
    # Try some other "string" forms.

    # Replacement:
    array4=( ${array0[@]/second/2nd} )
    echo
    echo "Elements in array4:  ${array4[@]}"

    # Replace all matching wildcarded string.
    array5=( ${array0[@]//new?/old} )
    echo
    echo "Elements in array5:  ${array5[@]}"

    # Just when you are getting the feel for this . . .
    array6=( ${array0[@]#*new} )
    echo # This one might surprise you.
    echo "Elements in array6:  ${array6[@]}"

    array7=( ${array0[@]#new1} )
    echo # After array6 this should not be a surprise.
    echo "Elements in array7:  ${array7[@]}"

    # Which looks a lot like . . .
    array8=( ${array0[@]/new1/} )
    echo
    echo "Elements in array8:  ${array8[@]}"

    #  So what can one say about this?

    #  The string operations are performed on
    #+ each of the elements in var[@] in succession.
    #  Therefore : Bash supports string vector operations.
    #  If the result is a zero length string,
    #+ that element disappears in the resulting assignment.
    #  However, if the expansion is in quotes, the null elements remain.

    #  Michael Zick:    Question, are those strings hard or soft quotes?
    #  Nathan Coulter:  There is no such thing as "soft quotes."
    #!    What's really happening is that
    #!+   the pattern matching happens after
    #!+   all the other expansions of [word]
    #!+   in cases like ${parameter#word}.

    zap='new*'
    array9=( ${array0[@]/$zap/} )
    echo
    echo "Number of elements in array9:  ${#array9[@]}"
    array9=( "${array0[@]/$zap/}" )
    echo "Elements in array9:  ${array9[@]}"
    # This time the null elements remain.
    echo "Number of elements in array9:  ${#array9[@]}"

    # Just when you thought you were still in Kansas . . .
    array10=( ${array0[@]#$zap} )
    echo
    echo "Elements in array10:  ${array10[@]}"
    # But, the asterisk in zap won't be interpreted if quoted.
    array10=( ${array0[@]#"$zap"} )
    echo
    echo "Elements in array10:  ${array10[@]}"
    # Well, maybe we _are_ still in Kansas . . .
    # (Revisions to above code block by Nathan Coulter.)

    #  Compare array7 with array10.
    #  Compare array8 with array9.

    #  Reiterating: No such thing as soft quotes!
    #  Nathan Coulter explains:
    #  Pattern matching of 'word' in ${parameter#word} is done after
    #+ parameter expansion and *before* quote removal.
    #  In the normal case, pattern matching is done *after* quote removal.

    exit

The relationship of ${array_name[@]} and ${array_name[*]} is analogous to that between $@ and $*. This powerful array notation has a number of uses.

    # Copying an array.
    array2=( "${array1[@]}" )
    # or
    array2="${array1[@]}"
    #
    #  However, this fails with "sparse" arrays,
    #+ arrays with holes (missing elements) in them,
    #+ as Jochen DeSmet points out.
    # ------------------------------------------
      array1[0]=0
    # array1[1] not assigned
      array1[2]=2
      array2=( "${array1[@]}" )       # Copy it?

    echo ${array2[0]}      # 0
    echo ${array2[2]}      # (null), should be 2
    # ------------------------------------------

    # Adding an element to an array.
    array=( "${array[@]}" "new element" )
    # or
    array[${#array[*]}]="new element"

    # Thanks, S.C.

| The array=( element1 element2 ... elementN ) initialization operation, with the help of command substitution, makes it possible to load the contents of a text file into an array.
    #!/bin/bash

    filename=sample_file

    #            cat sample_file
    #
    #            1 a b c
    #            2 d e fg

    declare -a array1

    array1=( `cat "$filename"`)                #  Loads contents
    #         List file to stdout              #+ of $filename into array1.
    #
    #  array1=( `cat "$filename" | tr '\n' ' '`)
    #                            change linefeeds in file to spaces.
    #  Not necessary because Bash does word splitting,
    #+ changing linefeeds to spaces.

    echo ${array1[@]}            # List the array.
    #                              1 a b c 2 d e fg
    #
    #  Each whitespace-separated "word" in the file
    #+ has been assigned to an element of the array.

    element_count=${#array1[*]}
    echo $element_count          # 8

Clever scripting makes it possible to add array operations.

Example 27-8. Initializing arrays

    #! /bin/bash
    # array-assign.bash

    #  Array operations are Bash-specific,
    #+ hence the ".bash" in the script name.

    # Copyright (c) Michael S. Zick, 2003, All rights reserved.
    # License: Unrestricted reuse in any form, for any purpose.
    # Version: $ID$
    #
    # Clarification and additional comments by William Park.

    #  Based on an example provided by Stephane Chazelas
    #+ which appeared in an earlier version of the
    #+ Advanced Bash Scripting Guide.

    # Output format of the 'times' command:
    # User CPU <space> System CPU
    # User CPU of dead children <space> System CPU of dead children

    #  Bash has two versions of assigning all elements of an array
    #+ to a new array variable.
    #  Both drop 'null reference' elements
    #+ in Bash versions 2.04 and later.
    #  An additional array assignment that maintains the relationship of
    #+ [subscript]=value for arrays may be added to newer versions.

    #  Constructs a large array using an internal command,
    #+ but anything creating an array of several thousand elements
    #+ will do just fine.

    declare -a bigOne=( /dev/* )  # All the files in /dev . . .
    echo
    echo 'Conditions: Unquoted, default IFS, All-Elements-Of'
    echo "Number of elements in array is ${#bigOne[@]}"

    # set -vx

    echo
    echo '- - testing: =( ${array[@]} ) - -'
    times
    declare -a bigTwo=( ${bigOne[@]} )
    # Note parens:    ^              ^
    times

    echo
    echo '- - testing: =${array[@]} - -'
    times
    declare -a bigThree=${bigOne[@]}
    # No parentheses this time.
    times

    #  Comparing the numbers shows that the second form, pointed out
    #+ by Stephane Chazelas, is faster.
    #
    #  As William Park explains:
    #+ The bigTwo array assigned element by element (because of parentheses),
    #+ whereas bigThree assigned as a single string.
    #  So, in essence, you have:
    #                   bigTwo=( [0]="..." [1]="..." [2]="..." ... )
    #                   bigThree=( [0]="... ... ..." )
    #
    #  Verify this by:  echo ${bigTwo[0]}
    #                   echo ${bigThree[0]}

    #  I will continue to use the first form in my example descriptions
    #+ because I think it is a better illustration of what is happening.

    #  The reusable portions of my examples will actual contain
    #+ the second form where appropriate because of the speedup.

    # MSZ: Sorry about that earlier oversight folks.

    #  Note:
    #  ----
    #  The "declare -a" statements in lines 32 and 44
    #+ are not strictly necessary, since it is implicit
    #+ in the  Array=( ... )  assignment form.
    #  However, eliminating these declarations slows down
    #+ the execution of the following sections of the script.
    #  Try it, and see.

    exit 0

| Adding a superfluous declare -a statement to an array declaration may speed up execution of subsequent operations on the array.

Example 27-9. Copying and concatenating arrays

    #! /bin/bash
    # CopyArray.sh
    #
    # This script written by Michael Zick.
    # Used here with permission.

    #  How-To "Pass by Name & Return by Name"
    #+ or "Building your own assignment statement".

    CpArray_Mac() {

    # Assignment Command Statement Builder

        echo -n 'eval '
        echo -n "$2"                    # Destination name
        echo -n '=( ${'
        echo -n "$1"                    # Source name
        echo -n '[@]} )'

    # That could all be a single command.
    # Matter of style only.
    }

    declare -f CopyArray                # Function "Pointer"
    CopyArray=CpArray_Mac               # Statement Builder

    Hype()
    {

    # Hype the array named $1.
    # (Splice it together with array containing "Really Rocks".)
    # Return in array named $2.

        local -a TMP
        local -a hype=( Really Rocks )

        $($CopyArray $1 TMP)
        TMP=( ${TMP[@]} ${hype[@]} )
        $($CopyArray TMP $2)
    }

    declare -a before=( Advanced Bash Scripting )
    declare -a after

    echo "Array Before = ${before[@]}"

    Hype before after

    echo "Array After = ${after[@]}"

    # Too much hype?

    echo "What ${after[@]:3:2}?"

    declare -a modest=( ${after[@]:2:1} ${after[@]:3:2} )
    #                    ---- substring extraction ----

    echo "Array Modest = ${modest[@]}"

    # What happened to 'before' ?

    echo "Array Before = ${before[@]}"

    exit 0

Example 27-10. More on concatenating arrays

    #! /bin/bash
    # array-append.bash

    # Copyright (c) Michael S. Zick, 2003, All rights reserved.
    # License: Unrestricted reuse in any form, for any purpose.
    # Version: $ID$
    #
    # Slightly modified in formatting by M.C.

    # Array operations are Bash-specific.
    # Legacy UNIX /bin/sh lacks equivalents.

    #  Pipe the output of this script to 'more'
    #+ so it doesn't scroll off the terminal.
    #  Or, redirect output to a file.

    declare -a array1=( zero1 one1 two1 )
    # Subscript packed.
    declare -a array2=( [0]=zero2 [2]=two2 [3]=three2 )
    # Subscript sparse -- [1] is not defined.

    echo
    echo '- Confirm that the array is really subscript sparse. -'
    echo "Number of elements: 4"        # Hard-coded for illustration.
    for (( i = 0 ; i < 4 ; i++ ))
    do
        echo "Element [$i]: ${array2[$i]}"
    done
    # See also the more general code example in basics-reviewed.bash.

    declare -a dest

    # Combine (append) two arrays into a third array.
    echo
    echo 'Conditions: Unquoted, default IFS, All-Elements-Of operator'
    echo '- Undefined elements not present, subscripts not maintained. -'
    # # The undefined elements do not exist; they are not being dropped.

    dest=( ${array1[@]} ${array2[@]} )
    # dest=${array1[@]}${array2[@]}     # Strange results, possibly a bug.

    # Now, list the result.
    echo
    echo '- - Testing Array Append - -'
    cnt=${#dest[@]}

    echo "Number of elements: $cnt"
    for (( i = 0 ; i < cnt ; i++ ))
    do
        echo "Element [$i]: ${dest[$i]}"
    done

    # Assign an array to a single array element (twice).
    dest[0]=${array1[@]}
    dest[1]=${array2[@]}

    # List the result.
    echo
    echo '- - Testing modified array - -'
    cnt=${#dest[@]}

    echo "Number of elements: $cnt"
    for (( i = 0 ; i < cnt ; i++ ))
    do
        echo "Element [$i]: ${dest[$i]}"
    done

    # Examine the modified second element.
    echo
    echo '- - Reassign and list second element - -'

    declare -a subArray=${dest[1]}
    cnt=${#subArray[@]}

    echo "Number of elements: $cnt"
    for (( i = 0 ; i < cnt ; i++ ))
    do
        echo "Element [$i]: ${subArray[$i]}"
    done

    #  The assignment of an entire array to a single element
    #+ of another array using the '=${ ... }' array assignment
    #+ has converted the array being assigned into a string,
    #+ with the elements separated by a space (the first character of IFS).

    # If the original elements didn't contain whitespace . . .
    # If the original array isn't subscript sparse . . .
    # Then we could get the original array structure back again.

    # Restore from the modified second element.
    echo
    echo '- - Listing restored element - -'

    declare -a subArray=( ${dest[1]} )
    cnt=${#subArray[@]}

    echo "Number of elements: $cnt"
    for (( i = 0 ; i < cnt ; i++ ))
    do
        echo "Element [$i]: ${subArray[$i]}"
    done
    echo '- - Do not depend on this behavior. - -'
    echo '- - This behavior is subject to change - -'
    echo '- - in versions of Bash newer than version 2.05b - -'

    # MSZ: Sorry about any earlier confusion folks.

    exit 0

--

Arrays permit deploying old familiar algorithms as shell scripts. Whether this is necessarily a good idea is left for the reader to decide.

Example 27-11. The Bubble Sort

    #!/bin/bash
    # bubble.sh: Bubble sort, of sorts.

    # Recall the algorithm for a bubble sort. In this particular version...

    #  With each successive pass through the array to be sorted,
    #+ compare two adjacent elements, and swap them if out of order.
    #  At the end of the first pass, the "heaviest" element has sunk to bottom.
    #  At the end of the second pass, the next "heaviest" one has sunk next to bottom.
    #  And so forth.
    #  This means that each successive pass needs to traverse less of the array.
    #  You will therefore notice a speeding up in the printing of the later passes.

    exchange()
    {
      # Swaps two members of the array.
      local temp=${Countries[$1]} #  Temporary storage
                                  #+ for element getting swapped out.
      Countries[$1]=${Countries[$2]}
      Countries[$2]=$temp

      return
    }

    declare -a Countries  #  Declare array,
                          #+ optional here since it's initialized below.

    #  Is it permissable to split an array variable over multiple lines
    #+ using an escape (\)?
    #  Yes.

    Countries=(Netherlands Ukraine Zaire Turkey Russia Yemen Syria \
    Brazil Argentina Nicaragua Japan Mexico Venezuela Greece England \
    Israel Peru Canada Oman Denmark Wales France Kenya \
    Xanadu Qatar Liechtenstein Hungary)

    # "Xanadu" is the mythical place where, according to Coleridge,
    #+ Kubla Khan did a pleasure dome decree.

    clear                      # Clear the screen to start with.

    echo "0: ${Countries[*]}"  # List entire array at pass 0.

    number_of_elements=${#Countries[@]}
    let "comparisons = $number_of_elements - 1"

    count=1 # Pass number.

    while [ "$comparisons" -gt 0 ]          # Beginning of outer loop
    do

      index=0  # Reset index to start of array after each pass.

      while [ "$index" -lt "$comparisons" ] # Beginning of inner loop
      do
        if [ ${Countries[$index]} \> ${Countries[`expr $index + 1`]} ]
        #  If out of order...
        #  Recalling that \> is ASCII comparison operator
        #+ within single brackets.

        #  if [[ ${Countries[$index]} > ${Countries[`expr $index + 1`]} ]]
        #+ also works.
        then
          exchange $index `expr $index + 1`  # Swap.
        fi
        let "index += 1"  # Or,   index+=1   on Bash, ver. 3.1 or newer.
      done # End of inner loop

    # ----------------------------------------------------------------------
    # Paulo Marcel Coelho Aragao suggests for-loops as a simpler altenative.
    #
    # for (( last = $number_of_elements - 1 ; last > 0 ; last-- ))
    ##                     Fix by C.Y. Hunt          ^   (Thanks!)
    # do
    #     for (( i = 0 ; i < last ; i++ ))
    #     do
    #         [[ "${Countries[$i]}" > "${Countries[$((i+1))]}" ]] \
    #             && exchange $i $((i+1))
    #     done
    # done
    # ----------------------------------------------------------------------

    let "comparisons -= 1" #  Since "heaviest" element bubbles to bottom,
                           #+ we need do one less comparison each pass.

    echo
    echo "$count: ${Countries[@]}"  # Print resultant array at end of each pass.
    echo
    let "count += 1"                # Increment pass count.

    done                            # End of outer loop
                                    # All done.

    exit 0

--

Is it possible to nest arrays within arrays?

    #!/bin/bash
    # "Nested" array.

    #  Michael Zick provided this example,
    #+ with corrections and clarifications by William Park.

    AnArray=( $(ls --inode --ignore-backups --almost-all \
    	--directory --full-time --color=none --time=status \
    	--sort=time -l ${PWD} ) )  # Commands and options.

    # Spaces are significant . . . and don't quote anything in the above.

    SubArray=( ${AnArray[@]:11:1}  ${AnArray[@]:6:5} )
    #  This array has six elements:
    #+     SubArray=( [0]=${AnArray[11]} [1]=${AnArray[6]} [2]=${AnArray[7]}
    #      [3]=${AnArray[8]} [4]=${AnArray[9]} [5]=${AnArray[10]} )
    #
    #  Arrays in Bash are (circularly) linked lists
    #+ of type string (char *).
    #  So, this isn't actually a nested array,
    #+ but it's functionally similar.

    echo "Current directory and date of last status change:"
    echo "${SubArray[@]}"

    exit 0

--

Embedded arrays in combination with indirect references create some fascinating possibilities

Example 27-12. Embedded arrays and indirect references

    #!/bin/bash
    # embedded-arrays.sh
    # Embedded arrays and indirect references.

    # This script by Dennis Leeuw.
    # Used with permission.
    # Modified by document author.

    ARRAY1=(
            VAR1_1=value11
            VAR1_2=value12
            VAR1_3=value13
    )

    ARRAY2=(
            VARIABLE="test"
            STRING="VAR1=value1 VAR2=value2 VAR3=value3"
            ARRAY21=${ARRAY1[*]}
    )       # Embed ARRAY1 within this second array.

    function print () {
            OLD_IFS="$IFS"
            IFS=$'\n'       #  To print each array element
                            #+ on a separate line.
            TEST1="ARRAY2[*]"
            local ${!TEST1} # See what happens if you delete this line.
            #  Indirect reference.
    	#  This makes the components of $TEST1
    	#+ accessible to this function.

            #  Let's see what we've got so far.
            echo
            echo "\$TEST1 = $TEST1"       #  Just the name of the variable.
            echo; echo
            echo "{\$TEST1} = ${!TEST1}"  #  Contents of the variable.
                                          #  That's what an indirect
                                          #+ reference does.
            echo
            echo "-------------------------------------------"; echo
            echo

            # Print variable
            echo "Variable VARIABLE: $VARIABLE"

            # Print a string element
            IFS="$OLD_IFS"
            TEST2="STRING[*]"
            local ${!TEST2}      # Indirect reference (as above).
            echo "String element VAR2: $VAR2 from STRING"

            # Print an array element
            TEST2="ARRAY21[*]"
            local ${!TEST2}      # Indirect reference (as above).
            echo "Array element VAR1_1: $VAR1_1 from ARRAY21"
    }

    print
    echo

    exit 0

    #   As the author of the script notes,
    #+ "you can easily expand it to create named-hashes in bash."
    #   (Difficult) exercise for the reader: implement this.

--

Arrays enable implementing a shell script version of the Sieve of Eratosthenes. Of course, a resource-intensive application of this nature should really be written in a compiled language, such as C. It runs excruciatingly slowly as a script.

Example 27-13. The Sieve of Eratosthenes

    #!/bin/bash
    # sieve.sh (ex68.sh)

    # Sieve of Eratosthenes
    # Ancient algorithm for finding prime numbers.

    #  This runs a couple of orders of magnitude slower
    #+ than the equivalent program written in C.

    LOWER_LIMIT=1       # Starting with 1.
    UPPER_LIMIT=1000    # Up to 1000.
    # (You may set this higher . . . if you have time on your hands.)

    PRIME=1
    NON_PRIME=0

    let SPLIT=UPPER_LIMIT/2
    # Optimization:
    # Need to test numbers only halfway to upper limit. Why?

    declare -a Primes
    # Primes[] is an array.

    initialize ()
    {
    # Initialize the array.

    i=$LOWER_LIMIT
    until [ "$i" -gt "$UPPER_LIMIT" ]
    do
      Primes[i]=$PRIME
      let "i += 1"
    done
    #  Assume all array members guilty (prime)
    #+ until proven innocent.
    }

    print_primes ()
    {
    # Print out the members of the Primes[] array tagged as prime.

    i=$LOWER_LIMIT

    until [ "$i" -gt "$UPPER_LIMIT" ]
    do

      if [ "${Primes[i]}" -eq "$PRIME" ]
      then
        printf "%8d" $i
        # 8 spaces per number gives nice, even columns.
      fi

      let "i += 1"

    done

    }

    sift () # Sift out the non-primes.
    {

    let i=$LOWER_LIMIT+1
    # Let's start with 2.

    until [ "$i" -gt "$UPPER_LIMIT" ]
    do

    if [ "${Primes[i]}" -eq "$PRIME" ]
    # Don't bother sieving numbers already sieved (tagged as non-prime).
    then

      t=$i

      while [ "$t" -le "$UPPER_LIMIT" ]
      do
        let "t += $i "
        Primes[t]=$NON_PRIME
        # Tag as non-prime all multiples.
      done

    fi

      let "i += 1"
    done

    }

    # ==============================================
    # main ()
    # Invoke the functions sequentially.
    initialize
    sift
    print_primes
    # This is what they call structured programming.
    # ==============================================

    echo

    exit 0

    # -------------------------------------------------------- #
    # Code below line will not execute, because of 'exit.'

    #  This improved version of the Sieve, by Stephane Chazelas,
    #+ executes somewhat faster.

    # Must invoke with command-line argument (limit of primes).

    UPPER_LIMIT=$1                  # From command-line.
    let SPLIT=UPPER_LIMIT/2         # Halfway to max number.

    Primes=( '' $(seq $UPPER_LIMIT) )

    i=1
    until (( ( i += 1 ) > SPLIT ))  # Need check only halfway.
    do
      if [[ -n ${Primes[i]} ]]
      then
        t=$i
        until (( ( t += i ) > UPPER_LIMIT ))
        do
          Primes[t]=
        done
      fi
    done
    echo ${Primes[*]}

    exit $?

Example 27-14. The Sieve of Eratosthenes, Optimized

    #!/bin/bash
    # Optimized Sieve of Eratosthenes
    # Script by Jared Martin, with very minor changes by ABS Guide author.
    # Used in ABS Guide with permission (thanks!).

    # Based on script in Advanced Bash Scripting Guide.
    # http://tldp.org/LDP/abs/html/arrays.html#PRIMES0 (ex68.sh).

    # http://www.cs.hmc.edu/~oneill/papers/Sieve-JFP.pdf (reference)
    # Check results against http://primes.utm.edu/lists/small/1000.txt

    # Necessary but not sufficient would be, e.g.,
    #     (($(sieve 7919 | wc -w) == 1000)) && echo "7919 is the 1000th prime"

    UPPER_LIMIT=${1:?"Need an upper limit of primes to search."}

    Primes=( '' $(seq ${UPPER_LIMIT}) )

    typeset -i i t
    Primes[i=1]='' # 1 is not a prime.
    until (( ( i += 1 ) > (${UPPER_LIMIT}/i) ))  # Need check only ith-way.
      do                                         # Why?
        if ((${Primes[t=i*(i-1), i]}))
        # Obscure, but instructive, use of arithmetic expansion in subscript.
        then
          until (( ( t += i ) > ${UPPER_LIMIT} ))
            do Primes[t]=; done
        fi
      done

    # echo ${Primes[*]}
    echo   # Change to original script for pretty-printing (80-col. display).
    printf "%8d" ${Primes[*]}
    echo; echo

    exit $?

Compare these array-based prime number generators with alternatives that do not use arrays, Example A-15, and Example 16-46.

--

Arrays lend themselves, to some extent, to emulating data structures for which Bash has no native support.

Example 27-15. Emulating a push-down stack

    #!/bin/bash
    # stack.sh: push-down stack simulation

    #  Similar to the CPU stack, a push-down stack stores data items
    #+ sequentially, but releases them in reverse order, last-in first-out.

    BP=100            #  Base Pointer of stack array.
                      #  Begin at element 100.

    SP=$BP            #  Stack Pointer.
                      #  Initialize it to "base" (bottom) of stack.

    Data=             #  Contents of stack location.
                      #  Must use global variable,
                      #+ because of limitation on function return range.

                      # 100     Base pointer       <-- Base Pointer
                      #  99     First data item
                      #  98     Second data item
                      # ...     More data
                      #         Last data item     <-- Stack pointer

    declare -a stack

    push()            # Push item on stack.
    {
    if [ -z "$1" ]    # Nothing to push?
    then
      return
    fi

    let "SP -= 1"     # Bump stack pointer.
    stack[$SP]=$1

    return
    }

    pop()                    # Pop item off stack.
    {
    Data=                    # Empty out data item.

    if [ "$SP" -eq "$BP" ]   # Stack empty?
    then
      return
    fi                       #  This also keeps SP from getting past 100,
                             #+ i.e., prevents a runaway stack.

    Data=${stack[$SP]}
    let "SP += 1"            # Bump stack pointer.
    return
    }

    status_report()          # Find out what's happening.
    {
    echo "-------------------------------------"
    echo "REPORT"
    echo "Stack Pointer = $SP"
    echo "Just popped \""$Data"\" off the stack."
    echo "-------------------------------------"
    echo
    }

    # =======================================================
    # Now, for some fun.

    echo

    # See if you can pop anything off empty stack.
    pop
    status_report

    echo

    push garbage
    pop
    status_report     # Garbage in, garbage out.

    value1=23;        push $value1
    value2=skidoo;    push $value2
    value3=LAST;      push $value3

    pop               # LAST
    status_report
    pop               # skidoo
    status_report
    pop               # 23
    status_report     # Last-in, first-out!

    #  Notice how the stack pointer decrements with each push,
    #+ and increments with each pop.

    echo

    exit 0

    # =======================================================

    # Exercises:
    # ---------

    # 1)  Modify the "push()" function to permit pushing
    #   + multiple element on the stack with a single function call.

    # 2)  Modify the "pop()" function to permit popping
    #   + multiple element from the stack with a single function call.

    # 3)  Add error checking to the critical functions.
    #     That is, return an error code, depending on
    #   + successful or unsuccessful completion of the operation,
    #   + and take appropriate action.

    # 4)  Using this script as a starting point,
    #   + write a stack-based 4-function calculator.

--

Fancy manipulation of array "subscripts" may require intermediate variables. For projects involving this, again consider using a more powerful programming language, such as Perl or C.

Example 27-16. Complex array application: Exploring a weird mathematical series

    #!/bin/bash

    # Douglas Hofstadter's notorious "Q-series":

    # Q(1) = Q(2) = 1
    # Q(n) = Q(n - Q(n-1)) + Q(n - Q(n-2)), for n>2

    #  This is a "chaotic" integer series with strange
    #+ and unpredictable behavior.
    #  The first 20 terms of the series are:
    #  1 1 2 3 3 4 5 5 6 6 6 8 8 8 10 9 10 11 11 12

    #  See Hofstadter's book, _Goedel, Escher, Bach: An Eternal Golden Braid_,
    #+ p. 137, ff.

    LIMIT=100     # Number of terms to calculate.
    LINEWIDTH=20  # Number of terms printed per line.

    Q[1]=1        # First two terms of series are 1.
    Q[2]=1

    echo
    echo "Q-series [$LIMIT terms]:"
    echo -n "${Q[1]} "             # Output first two terms.
    echo -n "${Q[2]} "

    for ((n=3; n <= $LIMIT; n++))  # C-like loop expression.
    do   # Q[n] = Q[n - Q[n-1]] + Q[n - Q[n-2]]  for n>2
    #    Need to break the expression into intermediate terms,
    #+   since Bash doesn't handle complex array arithmetic very well.

      let "n1 = $n - 1"        # n-1
      let "n2 = $n - 2"        # n-2

      t0=`expr $n - ${Q[n1]}`  # n - Q[n-1]
      t1=`expr $n - ${Q[n2]}`  # n - Q[n-2]

      T0=${Q[t0]}              # Q[n - Q[n-1]]
      T1=${Q[t1]}              # Q[n - Q[n-2]]

    Q[n]=`expr $T0 + $T1`      # Q[n - Q[n-1]] + Q[n - Q[n-2]]
    echo -n "${Q[n]} "

    if [ `expr $n % $LINEWIDTH` -eq 0 ]    # Format output.
    then   #      ^ modulo
      echo # Break lines into neat chunks.
    fi

    done

    echo

    exit 0

    #  This is an iterative implementation of the Q-series.
    #  The more intuitive recursive implementation is left as an exercise.
    #  Warning: calculating this series recursively takes a VERY long time
    #+ via a script. C/C++ would be orders of magnitude faster.

--

Bash supports only one-dimensional arrays, though a little trickery permits simulating multi-dimensional ones.

Example 27-17. Simulating a two-dimensional array, then tilting it

    #!/bin/bash
    # twodim.sh: Simulating a two-dimensional array.

    # A one-dimensional array consists of a single row.
    # A two-dimensional array stores rows sequentially.

    Rows=5
    Columns=5
    # 5 X 5 Array.

    declare -a alpha     # char alpha [Rows] [Columns];
                         # Unnecessary declaration. Why?

    load_alpha ()
    {
    local rc=0
    local index

    for i in A B C D E F G H I J K L M N O P Q R S T U V W X Y
    do     # Use different symbols if you like.
      local row=`expr $rc / $Columns`
      local column=`expr $rc % $Rows`
      let "index = $row * $Rows + $column"
      alpha[$index]=$i
    # alpha[$row][$column]
      let "rc += 1"
    done

    #  Simpler would be
    #+   declare -a alpha=( A B C D E F G H I J K L M N O P Q R S T U V W X Y )
    #+ but this somehow lacks the "flavor" of a two-dimensional array.
    }

    print_alpha ()
    {
    local row=0
    local index

    echo

    while [ "$row" -lt "$Rows" ]   #  Print out in "row major" order:
    do                             #+ columns vary,
                                   #+ while row (outer loop) remains the same.
      local column=0

      echo -n "       "            #  Lines up "square" array with rotated one.

      while [ "$column" -lt "$Columns" ]
      do
        let "index = $row * $Rows + $column"
        echo -n "${alpha[index]} "  # alpha[$row][$column]
        let "column += 1"
      done

      let "row += 1"
      echo

    done

    # The simpler equivalent is
    #     echo ${alpha[*]} | xargs -n $Columns

    echo
    }

    filter ()     # Filter out negative array indices.
    {

    echo -n "  "  # Provides the tilt.
                  # Explain how.

    if [[ "$1" -ge 0 &&  "$1" -lt "$Rows" && "$2" -ge 0 && "$2" -lt "$Columns" ]]
    then
        let "index = $1 * $Rows + $2"
        # Now, print it rotated.
        echo -n " ${alpha[index]}"
        #           alpha[$row][$column]
    fi

    }

    rotate ()  #  Rotate the array 45 degrees --
    {          #+ "balance" it on its lower lefthand corner.
    local row
    local column

    for (( row = Rows; row > -Rows; row-- ))
      do       # Step through the array backwards. Why?

      for (( column = 0; column < Columns; column++ ))
      do

        if [ "$row" -ge 0 ]
        then
          let "t1 = $column - $row"
          let "t2 = $column"
        else
          let "t1 = $column"
          let "t2 = $column + $row"
        fi

        filter $t1 $t2   # Filter out negative array indices.
                         # What happens if you don't do this?
      done

      echo; echo

    done

    #  Array rotation inspired by examples (pp. 143-146) in
    #+ "Advanced C Programming on the IBM PC," by Herbert Mayer
    #+ (see bibliography).
    #  This just goes to show that much of what can be done in C
    #+ can also be done in shell scripting.

    }

    #--------------- Now, let the show begin. ------------#
    load_alpha     # Load the array.
    print_alpha    # Print it out.
    rotate         # Rotate it 45 degrees counterclockwise.
    #-----------------------------------------------------#

    exit 0

    # This is a rather contrived, not to mention inelegant simulation.

    # Exercises:
    # ---------
    # 1)  Rewrite the array loading and printing functions
    #     in a more intuitive and less kludgy fashion.
    #
    # 2)  Figure out how the array rotation functions work.
    #     Hint: think about the implications of backwards-indexing an array.
    #
    # 3)  Rewrite this script to handle a non-square array,
    #     such as a 6 X 4 one.
    #     Try to minimize "distortion" when the array is rotated.

A two-dimensional array is essentially equivalent to a one-dimensional one, but with additional addressing modes for referencing and manipulating the individual elements by row and column position.

For an even more elaborate example of simulating a two-dimensional array, see Example A-10.

--

For more interesting scripts using arrays, see:

- Example 12-3

- Example 16-46

- Example A-22

- Example A-44

- Example A-41

- Example A-42

Prev | Home | Next
List Constructs | Up | Indirect References