之前其实很少用到git merge and git rebase，一般都是通过Github UI Pull Request merges feature branch’s updates into master.
也可在CLI 中merge feature branch into master，但这样master中的git history不是线性的，并且会制造一个merge commit:
1 2 3 4 5 6 7 8 9 10 11 12
# merge feature to master, first go to master git checkout master # squash will summary all commits in feature branch into merge commit git merge --squash feature # or 不用fast-forward merge, 用普通的recursive merge # by default is --ff (try fast-forward first, not create merge commit) git merge --no-ff feature
# need to commit this merge git commit -m "feature and master merged" # then push to master git push origin master
Highlight:
fast-forward merge: less commits, no merge commit will be generated.
recusive merge: has merge commit, can do revert, clear what was done on a branch.
# 如果dev branch不干净 # later can run `git stash pop` git stash
# 可以先在dev中squash一下多个commits # HEAD~5: rebase HEAD 向下的5个commits # HEAD 可以用git log看一下位置 git checkout dev git rebase -i HEAD~5
# pull latest to master branch # because it may be updated by others git checkout master git pull origin master
# current in dev branch, reword commits on top of master history git checkout dev # rebase against master，这样之后create pull/merge request就不会产生冲突了 # -i: interactively git rebase -i master
# may have conflicts, fix conflicts as prompted git add <fixed files> git rebase --continue
# abort rebase git rebase --abort
# after rebase (squash), check commit log changes git log -n 5
# update merge request on github # 之前的merge request就会被重写 # -f: force override merge request on remote git push -f origin dev
Note, below 2 args formats are tested and worked for shell(sh/bash), other init process may not fit, for example tini, you have to use array format and one token a line.
The format of command and args syntax in kubernetes yaml really makes me crazy, when the entrypoint has long or multiple arguments, can be wrote in multiple lines instead of one line for better view:
apiVersion:v1 kind:Pod metadata: ... spec: containers: -name:myapp-container image:busybox command: ["/bin/sh", "-c", "--"] args: ["echo \"check something...\"; if [[ ! -f /root/.link ]]; then echo \"file does not exist!\"; echo \"no!\"; else echo \"file is there!\"; fi; echo \"done!\""]
The format is error-prone, every command should end with ;, for example the while and if above. I separate them into several lines for good looking, but they actually are one line command.
For one command with multiple options or parameters, can be described as below, no args field:
I want to wrap several commands into one line format to put in command/args field in yaml file for Kubernetes. But always get syntax error with bad ;, for example: -bash: syntax error near unexpected token ;', finally understand the error is from &.
timeout command used to run command with a time limit. One case is when you know exactly how long you want a process to run for. A common use-case is to have timeout control a logging or data-capture program so that the log files don’t relentlessly devour your hard drive space.
Another case is when you don’t know how long you want a process to run for, but you do know you don’t want it to run indefinitely. You might have a habit of setting processes running, minimizing the terminal window, and forgetting about them. For example:
1
timeout 12 ping 127.0.0.1
By default 12 is second unit, to use a time value measured in minutes, hours or days append an m, h or a d.
Send Right Signal
When timeout wants to stop a program it sends the SIGTERM signal. This politely asks the program to terminate. Some programs may choose to ignore the SIGTERM signal. When that happens, we need to tell timeout to be a little more forceful.
we use the -k (kill after) option. The -k option requires a time value as a parameter. If tcpdump is still running after 20 seconds, it means the SIGTERM was ignored and timeout should send in SIGKILL to finish the job.
Retrieving the Program’s Exit Code
timeout provides its own exit code, The exit code is 124. This is the value timeout uses to indicate the program was terminated using SIGTERM.
But we may not care about that. We are probably more interested in the exit code from the process that timeout is controlling.
If the execution of the program ends before timeout terminates it, timeout can pass the exit code from the program back to the shell. (no need --preserve-status)
we must use the --preserve-status option if the program timeout but we still want to get its status exit code! For example:
1
timeout --preserve-status 10 <program>
1 2 3 4 5 6 7 8 9 10
# Start Engine Conductor LogMsg "Starting Engine Conductor Pod..." (cd$BASE/Engine/engine-conductor; pwd; ./createConductorDepAndSvc.sh $NAME_SPACE$ENGINE_HOST) check_return_code $? conductor_pod=$(kubectl get pods -n $NAME_SPACE | grep -i -E $POD_STATES | grep -v NAME|awk '{print $1}'|grep $ENGINE_HOST) # Check for Engine background processes to complete # <() is process substitution and redirect the command output to cat # 其实这里没必要用这种形式 cat <() timeout --preserve-status 3.0m cat <(check_k8s_start_loop $conductor_pod 7 $NAME_SPACE) check_timeout_return $? $conductor_pod
Sometimes you will find the demo run in your kubernetes cluster doesn’t work exactly or doesn’t fit your expectation from official documentation. That maybe a feature gate you didn’t enable it or you don’t have it.
This is an interesting issue which involves 4 topices: Volume, Security Context, NFS and initContainer.
The issue comes from the permission denied error. The process fail to create the file under the mount path, I check the owner and group of that path, they are both root.
In the yaml file, I specify the fsGroup field as id 9092, from the official document here (The example use id 2000):
1
Since fsGroup field is specified, all processes of the container are also part of the supplementary group ID 2000. The owner for volume /data/demo and any files created in that volume will be Group ID 2000.
so the owner of the volume should be 9092, but they don’t.
It seems fsGroup securityContext does not apply to nfs mount especially we run the containers as non-root user we cannot access the mount. This issue may be solved in later version, need to take care.
!!! Why this happens? Because we use hostPath, it by default will create root owned path if path does not exist. Here the NFS is not the NFS way kubernetes use, we use hostPath then manually nfs the nodes externally, not by the setting of K8s(need to do experiment).
The workaround is using initContainers with busybox run as root and chown to the nfs mount with expected id, for example:
When we speak of the command line, we are really referring to the shell. The shell is a program that takes keyboard commands and passes them to the operating system to carry out.
If the last character of the prompt is a hash mark(#) rather than a $, the terminal session has superuser privileges.
Unix-like systems such as Linux always have a single file system tree, regardless of how many drives or storage devices are attached to the computer. Starts from /.
Linux has no concept of a file extension like some other operating systems. Although Unix-like operating systems don’t use file extensions to determine the contents/purpose of files, many application programs do.
Some options and arguments unknown to me or not common:
# cd to home of vagrant user cd ~vagrant # list multiple directories ls ~ /usr /opt
# A: list all but . and .. # F: append / for directories ls -AF
# 3: number of existing hard link to this file data part drwxr-xr-x 3 root root 24 Jun 6 2020 mnt
# in less page hit 'h': display help screen less file
# re-initialize the terminal, deeper than clear # 很少用到 reset
# -u: When copying files from one directory to another # only copy files that either don’t exist or are newer # than the existing corresponding files in the destination directory. cp -u|--update
# Normally, copies take on the default attributes of the user performing the copy. # a: preserve the original permission and user/group # for example, root user cp oridinary user file cp -a
Remember that a file will always have at least one hard link because the file’s name is created by a link. When we create hard links, we are actually creating additional name parts that all refer to the same data part.
Directory 是不能有hard link的，一个文件夹的hardlink 部分数字一般是2+n, 2代表文件夹内部., .., n 代表subdirectories数量。
The type command is a shell builtin that displays the kind of command the shell will execute, command can be categorized as:
executable binary
script
function
alias
which is for locating executables only.
apropos is the same as man -k:
1 2
apropos partition man -k partition
Modern version of redirection:
1 2 3 4
# &> is equal to 2>&1 ls -l /bin/usr &> ls-output.txt # append echo"123" &>> ls-output.txt
Interesting accident below, The lesson here is that the redirection operator silently creates or overwrites files, so you need to treat it with a lot of respect.
1 2 3 4
cd /usr/bin # overwrite less program under /usr/bin # instead use ls | less ls > less
# arithmetic # 注意区别(( ))是针对integer的compound command, used in condition such as if, while, for, etc. echo $((3 + 4)) # can use single parentheses inside without $ prefix echo"$(((5**2) * 3))"
Remember, parameter expansion “$USER”, arithmetic expansion, and command substitution still take place within double quotes. If we need to suppress all expansions, we use single quotes.
Command history quick rerun:
1 2 3 4 5 6 7 8 9 10
# execute last command !! # execute command at 123 line in hsitory !123
# be cautious # last command starts with "string" !string # last command contains "string" !?string
Mentioned a new command script, used to record the terminal typing as a hardcopy into a file, for example for students learning. The history is controlled by env vars:
Load average in top command: refers to the number of processes that are waiting to run; that is, the number of processes that are in a runnable state and are sharing the CPU. Three values are shown, each for a different period of time. The first is the average for the last 60 seconds, the next the previous 5 minutes, and finally the previous 15 minutes. Values less than 1.0 indicate that the machine is not busy (还和有多少cores有关，多核则可以大于1).
Other commands for top useful are ?/h for help, f for column and sort selection, m check memory utilization, sorted by memory can type shift + m.
About signal (这部分解释得比较好):
HUP: also used by many daemon programs to cause a reinitialization. This means that when a daemon is sent this signal, it will reload its configuration file. The Apache web server is an example of a daemon that uses the HUP signal in this way.
INT: Interrupt. This performs the same function as ctrl-c sent from the terminal. It will usually terminate a program.
TERM: Terminate. This is the default signal sent by the kill command. If a program is still “alive” enough to receive signals, it will terminate.
STOP: Stop. This signal causes a process to pause without terminating. Like the KILL signal, it is not sent to the target process, and thus it cannot be ignored.
TSTP: Terminal stop. This is the signal sent by the terminal when ctrl-z is pressed. Unlike the STOP signal, the TSTP signal is received by the program, but the program may choose to ignore it.
CONT: Continue. This will restore a process after a STOP or TSTP signal. This signal is sent by the bg and fg commands. For example, use ctrl-z with a fg and then run bg to make it run in background.
Part 2
The set command will show both the shell and environment variables as well as any defined shell functions, while printenv will display only the environment variables.
Environment variables:
PS1: Stands for prompt string 1. This defines the contents of the shell prompt. Also prompt can be customized by shell function and scripts. 还有PS2, PS3, etc. see this article.
Low-level tools that handle tasks such as installing and removing package files.
High-level tools that perform metadata searching and dependency resolution.
主要了解安装，更新，删除，查找这几个操作即可，low/high level都有相关的选项.
Linux storage device names convention:
/dev/fd* Floppy disk drives
/dev/hd* IDE (PATA) disks on older systems
/dev/lp* printer
/dev/sd* SCSI disks. On modern Linux systems, the kernel treats all disk-like device as SCSI disks
/dev/sr* Optical drives (CD/DVD readers and burners)
/var/log/messages vs /var/log/syslog, /var/log/messages is the syslog on non-Debian/non-Ubuntu systems such as RHEL or CentOS systems usually. /var/log/messages instead aims at storing valuable, non-debug and non-critical messages. This log should be considered the “general system activity” log.
/var/log/syslog in turn logs everything, except auth related messages.
1 2
# a great way to watch what the system is doing in near real-time. tail -f /var/log/messages
The last digit in /etc/fstab file is used to specify integrity checking privilege when system boots by fsck (file system check), 0 means not routinely checked:
1
/dev/sdb /data xfs defaults 0 0
fsck can also repair corrupt file systems with varying degrees of success, depending on the amount of damage. On Unix-like file systems, recovered portions of files are placed in the lost+found directory, located in the root of each file system.
1 2
# unmount /dev/sdb1 first sudo fsck /dev/sdb1
dd (data definition) can be used to copy block of data:
1 2 3 4
# backup sdb to sdc ddif=/dev/sdb of=/dev/sdc # backup to ordinary file ddif=/dev/sdb of=/tmp/sdb.bk
When it comes to networking, there is probably nothing that cannot be done with Linux. Linux is used to build all sorts of networking systems and appliances, including firewalls, routers, name servers, network-attached storage (NAS) boxes, and on and on.
The locate database is created by another program named updatedb(you can run it manually or by system cronjob).
find command can have logic operators:
# -and/-a # -or/-o # -not/! # escape () as it has special meaning to shell # object can only be a file or directory, so use -or find ~ \( -type f -not -perm 0600 \) -or \( -type d -not -perm 0700 \)
# -print, always test first find ~ -type f -name '*.bak'[-print] # the same as find ~ -type f -and -name '*.bak' -and -print # order matters!!! just like if condition and group, from left to right, short execution find ~ -print -and -type f -and -name '*.bak' # -delete action find ~ -type f -name '*.bak' -delete
# Here, command is the name of a command, {} is a symbolic # representation of the current pathname, and the semicolon # is a required delimiter indicating the end of the command # {} and ; need to quote as special meaning to shell find ~ -type f -name 'foo*' -execls -l '{}'';' # -ok: interactive with user find ~ -type f -name 'foo*' -ok ls -l '{}'';'
# more powerful and flexible find /tmp -mindepth 1 -maxdepth 1 -type d -mmin +50 -execls -d {} \;
xargs is more efficient then regular -exec {} ; because it executes in one time (when setting properly), 关于这2个效率的比较: FIND -EXEC VS. FIND | XARGS. 不同options是不同的效率！！
# -r: when no input for xargs, don't run it # 如果不用-r, 则xargs总会执行，输出不符合的结果 find ~ -type f -name 'foo*' -print | xargs -r ls -l # same as -exec with + find ~ -type f -name 'foo*' -execls -l '{}''+'
# practices find ~ \( -type f -not -perm 0600 -execchmod 0600 '{}'';' \) -or \( -type d -not -perm 0700 -execchmod 0700 '{}'';' \) # with tar # r: append mode to tar find ~ -name 'file-A' -exec tar rf old.tar '{}''+'
# -: stdin or stdout as needed # --files-from=- 表示从upper pipeline得到文件列表 # cf - 表示tar的东西又输出到下一个pipeline了 find playground -name 'file-A' | tar cf - --files-from=- | gzip > playground.tgz # --files-from=- abbr is equal to -T - find playground -name 'file-A' | tar cf target.tar -T -
# tar and transfer remote dir to local and untar ssh remote-sys 'tar cf - Documents' | tar xf -
If the filename - is specified, it is taken to mean standard input or output, as needed. (By the way, this convention of using - to represent standard input/output is used by a number of other programs, too).
下面介绍一些text processing commands，其实就是git 使用的部分功能.
commonly use and known: cat, sort, uniq, cut, paste, join, comm, diff, patch, tr, sed, aspell.
sort: Many uses of sort involve the processing of tabular data
1 2 3 4 5 6 7 8 9 10 11
# --key/-k 1,1: starts from field 1 and end at field 1 # -key=2n: field 2 sort as numeric sort --key=1,1 --key=2n distros.txt
# sort on specific part of a field # Fedora 10 11/25/2008 # Ubuntu 8.10 10/30/2008 sort -k 3.7nbr -k 3.1nbr -k 3.4nbr distros.txt
# sort on shell field sort -t ':' -k 7 /etc/passwd | head
cut: cut is best used to extract text from files that are produced by other programs, rather than text directly typed by humans.
1 2 3 4 5 6 7 8 9
# Fedora 10 11/25/2008 # Ubuntu 8.10 10/30/2008 # extrace year number # -c: character range # -f: field index, start from 1 cut -f 3 file | cut -c 7-10 # can convert tab to spaces # tab 是会对齐field的 expand file | cut -c 23-
paste: it adds one or more columns of text to a file in argument order
1 2
# assume file1 file2 file3 are columned format paste file1 file2 file3
The join program it joins data from multiple files based on a shared key field. The same as DB join operation:
1 2
# assume file1 and file2 has shared field join file1 file2
comm: Compare Two Sorted Files Line by Line
diff: Compare Files Line by Line
1 2 3 4 5
# column 1 is in file1 # column 2 is in file2 # column 3 is in both file1 and file2 # -n (-12): remove 1 and 2 column in output comm -12 file1.txt file2.txt
patch: Apply a diff to an Original, It accepts output from diff and is generally used to convert older version files into newer versions. git其实push的也是diff的部分，然后patch 到target branch中.
1 2 3 4
# run in the same dir level diff -Naur old_file new_file > diff_file # diff_file has enough information for patch patch < diff_file
where old_file and new_file are either single file or directory containing files. The r option supports recursion of a directory tree.
tr: think of this as a sort of character-based search-and-replace operation.
nl: number Lines, add header, body, footer
fold: wrap each line to a specified length
1 2 3
# -w: width 50 character # -s: bread at speace not in the middle of word fold -w 50 -s test.file
fmt: simple text formetter, it fills and joins lines in text while preserving blank lines and indentation.
1 2 3
# -w: width 50 characters # -c: operate in crown margin mode, better format than fold fmt -cw 50 test.file
printf: not used for pipelines, used mostly in scripts where it is employed to format tabular data, similar to C printf:
1 2 3 4
printf"I formatted '%s' as a string.\n" foo # the same placeholder as C program # %[flags][width][.precision]placeholder printf"%d, %f, %o, %s, %x, %X\n" 380 380 380 380 380 380
make command to compile C/C++ program.
Some compilers translate high level instructions into assembly language and then use an assembler to perform the final stage of translation into machine language. A process often used in conjunction with compiling is called linking. A program called a linker is used to form the connections between the output of the compiler and the libraries that the compiled program requires. The final result of this process is the executable program file, ready for use.
Shell scripts that do not require compiling. They are executed directly. These are written in what are known as scripting or interpreted languages. These languages have grown in popularity in recent years and include Perl, Python, PHP, Ruby, and many others.
we need Makefile (usually generated by configure script) to instruct make command to compile source code:
1 2 3 4 5 6 7 8 9
# usually this file is to collect system configuration and check # required libraries ./configure # compile and build app # will choose Makefile in the same directory automatically make # install app # install is also a build target from running make command sudo make install
install will install the final product in a system directory for use. Usually, this directory is /usr/local/bin, the traditional location for locally built software. However, this directory is not normally writable by ordinary users, so we must become the superuser to perform the installation.
Part 4
Part 4 primarily talks about scripting, I place the notes in shell and scripting blogs.
Positional arguments, $0 will always contain the first item appearing on the command line(exactly what it is). When parameters size is large, use shift to access:
1 2 3 4 5 6 7
count=1 # $2 keep moving to $1 while [[ $# -gt 0 ]]; do echo"Argument $count = $1" count=$((count + 1)) shift done
Just as positional parameters are used to pass arguments to shell scripts, they can also be used to pass arguments to shell functions.
Here document(<<[-]) and here string(<<<), <<- is related to stripe leading tab, depends on your use case (usually << is fine).
If the optional in words portion of the for command is omitted, for defaults to processing the positional parameters:
1 2 3 4 5 6
# set positional parameters set a b c # without in keywork, for loops the positional parameters for i; do echo"$i" done
Modern for loop mimics C program:
1 2 3 4
# i is treated as integer, no need $i prefix in (( )) for (( i=0; i<5; i=i+1 )); do echo$i done
# the same as (( )) for integer compution echo $(( a + 2 )) echo $(( 5 % 2 )) # number bases, output is converted to 10 based value # $1 is a 16 based value echo $((0x$1)) echo $((0xff)) # 2#: 2 based echo $((2#11111111))
# can use C style shortcut, also used in for loop (( )) declare -i a=0 $((a+=1)) $((a-=1)) $((a/=1)) $((a*=1)) $((a++)) $((a--)) $((++a)) $((--a))
# bit operations are the same as C for ((i=0;i<8;++i)); doecho $((1<<i)); done # logic operation: < > <= >= == != && || ! echo $((a<1?++a:--a))
类似C中的用法，同时做赋值和判断:
1 2 3 4
foo= # foo is assigned 5 and valued as true # notice that here = is not ==, although in [ ], = is the same as == if (( foo = 5 )); thenecho"It is true."; fi
The bc program reads a file written in its own C-like language and executes it. A bc script may be a separate file, or it may be read from standard input.
1 2 3 4
# start and quiet bc, use stdin to input bc -q # use here string bc <<< "2+2"
Group command does not create subshell:
1 2 3 4 5 6
# note the space after { and before } # each cmd must has ; { cmd1; cmd2; [cmd3; ...] }
# { } returns the total output { ls -l; echo"Listing of foo.txt"; cat foo.txt; } > output.txt
Therefore, in most cases, unless a script requires a subshell, group commands are preferable to sub shells. Group commands are both faster and require less memory.
Process substitution, it feeds the output of a process (or processes) into the stdin of another process.
这里的例子主要是和read 结合使用:
1 2 3 4 5
# rediect output from process substitution to read read < <(echo"foo") # or using here string read <<< "foo" echo$REPLY
Process substitution allows us to treat the output of a subshell as an ordinary file for purposes of redirection. 可以看做一个文件.
1 2
# output is /dev/fd/63 echo <(echo"foo")
Process substitution is often used with loops containing read.
Other example of process substitution:
1 2 3
grep word /usr/share/dict/linux.words | wc # can be modified as wc <(grep word /usr/share/dict/linux.words)
In most Unixlike systems, it is possible to create a special type of file called a named pipe. Named pipes are used to create a connection between two processes and can be used just like other types of files. Named pipes behave like files but actually form firstin firstout (FIFO) buffers.
1 2 3 4 5 6 7 8 9
# type is p # prw-r--r--. 1 root root 0 Mar 13 23:11 pipe1 mkfifo pipe1
When we design a large, complicated script, it is important to consider what happens if the user logs off or shuts down the computer while the script is running. When such an event occurs, a signal will be sent to all affected processes. In turn, the programs representing those processes can perform actions to ensure a proper and orderly termination of the program.
Signals are software interrupts sent to a program to indicate that an important event has occurred. The events can vary from user requests to illegal memory access errors.
## output signal name or number kill -l 9 kill -l usr1
Kill process in command line:
1 2 3
kill -9 <pid> kill -s 9 <pid> kill -SIGKILL <pid>
Some commonly use signals:
SIGHUP: Many daemons will reload their configuration instead of exiting when receiving this signal, see here for explanation
SIGINT: Issued if the user sends an interrupt signal (Ctrl + C), usually will terminate a program
SIGQUIT: Issued if the user sends a quit signal (Ctrl + D), it will dump core
SIGKILL: If a process gets this signal it must quit immediately and will not perform any clean-up operations, this one cannot be ignored or trapped! (这里提一下，force kill会导致程序，比如Python中finally clauses 和 exit handler 无法工作)
SIGTERM: Software termination signal (sent by kill by default)
SIGSTOP: Stop process (Ctrl + Z), for example, stop fg mode, see SIGCONT below
SIGCONT: Continue if stopped, for example, after stop fg mode by (Ctrl + Z), switch to bg will trigger this signal
SIGSTP: Stop typed at terminal
SIGCHLD: When a child process stops or terminates, SIGCHLD is sent to the parent process. (这个可以用来trap回收subprocess的资源)
For signal default behavior, see man 7 signal!!
Note that process cannot define handler for SIGKILL and SIGSTOP, the default behavior is in use.
Note that non-init process cannot ignore SIGKILL and SIGSTOP, they are used for root user and kernel for process management. But, remember, process in D state is uninterruptable even with SIGKILL and SIGSTOP.
Note that you cannot kill init process(PID 1) by kill -9 1, this is the exception as described in man 2 kill: “The only signals that can be sent to process ID 1, the init process, are those for which init has explicitly installed signal handlers. This is done to assure the system is not brought down accidentally.” See here for hint.
About trap the signals in script, for example:
1 2 3 4 5 6 7 8 9
## Set a Trap for Signals for graceful shutdown declare -a SIGNALS_TRAPPED=(INT TERM) shudown_hook() { ... (/opt/servers/stop.sh) echo"Shutdown and Cleanup Successful..." exit 0 } trap'shudown_hook'" ${SIGNALS_TRAPPED[@]}"
The general format is trap command signals, for example:
1
trap"rm -rf /tmp/peek.txt; exit 0" 1 2
To ignore signals, for example:
1
trap"" 2 3 15
If main process ignore a signal, all subshells also ignore that signal. However, if you specify an action to be taken on the receipt of a signal, all subshells will still take the default action on receipt of that signal.
The first thing to do is to distinguish between bash indexed array and bash associative array. The former are arrays in which the keys are ordered integers, while the latter are arrays in which the keys are represented by strings.
Although indexed arrays can be initialized in many ways, associative ones can only be created by using the declare command.
# create array with out declare # 注意这里index 只能是number，否则会出问题，因为associated array 必须用declare -A先声明 array=([0]=Sun [1]=Mon [100]=Tue [3]=Wed [4]=Thu [5]=Fri [6]=Sat) # or array=( x y z ) # create array via declare declare -a array=(x y z) # create separately declare -a array # the same as array=xx array[0]=xx array[1]=yy array[8]=zz
# print declare -p array # delete item unset array[2] # empty and delete the array unset array # this will not have any impact on array content array=
Add new element into array:
1 2 3
array+=(${var}) # or multi-item array+=('foo''cat')
Note that in the context where an assignment statement is assigning a value to a shell variable or array index (see Arrays), the += operator can be used to append to or add to the variable’s previous value.
Using "${array[@]}"(have double quotes) in for loop to fetch array item. "${array[*]}" is not recommended, the same reason as shell positional parameters with @ and *.
1 2 3 4
for item in"${array[@]}" do echo${item} done
Or fetch item by index(key):
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
# this will only return the value existed index # the same as associated array iteration, but here it returns index number instead for i in"${!array[@]}" do # ${array[i]} also OK echo${array[$i]} done
# ${#array[@]}: length of array # but it bash array may have gaps! some items are empty for((i=1;i<${#array[@]};i++)) do echo${array[i]} echo${array[i-1]} done
Sort array, 这里对array的输出使用了pipeline，一个很好的启发:
1 2 3 4 5 6
a=(f e d c b a)
echo"Original array: ${a[@]}" # out most () is for forming a array a_sorted=($(for i in"${a[@]}"; doecho$i; done | sort)) echo"Sorted array: ${a_sorted[@]}"
Create Associated Array
Statement declare is the only way to go, see reference for more details. Actually this is Map in bash. 这里的key 默认是string，虽然可以写成数字.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
# create all at once declare -A array=([a]=xx [b]=yy [c]=zz) # create separately declare -A array array[a]=xx array[b]=yy array[c]=zz # or write in one line # 注意这里先declare -A 了 array=([a]=xx [b]=yy [c]=zz)
# print declare -p array # delete item unset array[2] # empty and delete array unset array # this will not have any impact on array content array=
Iterate over the associated array:
1 2 3 4 5 6 7 8 9 10 11 12 13
declare -a array=([a]=a [b]=b [c]=c) # The keys are accessed using an exclamation point # can do this to index array as well! for key in"${!array[@]}" do echo"key : $key" echo"value: ${array[$key]}" done
# assign all keys to other array array2=("${!array[@]}") # copy whole array array3=("${array[@]}")
Array Counterpart
Loop through comma separated string, or other delimiter.
1 2 3 4 5 6 7 8
var="1,2,3,4,5" # use sed for i in $(echo$var | sed "s/,/ /g") # or use pattern match for i in${var//,/" "} do echo$i done
还可以使用IFS设置不同的separator (IFS also is used with read), 也可以用tr去替换分隔符。
Caveat
If you use declare -a or declare -A to create array in shell function, it by default is local scope in that function. You can move the declare out to make the array globally access, or use declare -ga and `declare -gA (only bash 4.2 and later support this).
Notice that variable defined without local in function is global access.
Assign array to other variables:
1 2 3
a=('a''b''c') # must have double qoutes to protect key that has whitespace b=("${a[@]}")
Notice that you cannotexport array directly. Array variables may not (yet) be exported. in some bash verions, but there are some workarounds.