Tag Archives: shell

Write commands with multiple lines in Dockerfile

There are several ways to write commands with multiple lines in Dockerfile, for example you wanna echo a bash file entrypoint.sh with content:

#!/bin/bash
echo 3
echo 2
echo 1
echo run

You could:

Using printf

RUN printf '#!/bin/bash\necho 3\necho 2\necho 1\necho run' > /entrypoint.sh

Using cat

RUN sh -c "$(/bin/echo -e "cat > /entrypoint.sh <<EOF\
\n#!/bin/bash\
\necho 3\
\necho 2\
\necho 1\
\necho run\
\nEOF\n")"

Using echo -e

RUN echo -e " #!/bin/bash\n\
echo 3\n\
echo 2\n\
echo 1\n\
echo run" > /entrypoint.sh

Using $'...'

The $’…’ feature is known as "ANSI-C quoting" but it’s not a POSIX shell > feature. According to unix.stackexchange.com/a/371873/109111 it was > originally a ksh93 feature but it is now available in bash, zsh, mksh, > FreeBSD sh and in busybox’s ash

RUN echo $'#!/bin/bash\n\
echo 3\n\
echo 2\n\
echo 1\n\
echo run' > /entrypoint.sh

echo -e & $'...' are both similar in that they support the following escape sequences:

\a     alert (bell)
\b     backspace
\e
\E     an escape character
\f     form feed
\n     new line
\r     carriage return
\t     horizontal tab
\v     vertical tab
\\     backslash
\0nnn  the eight-bit character whose value is the octal value nnn (zero to three octal digits)
\xHH   the eight-bit character whose value is the hexadecimal value HH (one or two hex digits)
\uHHHH the Unicode (ISO/IEC 10646) character whose value is the hexadecimal value HHHH (one to four hex digits)
\UHHHHHHHH
     the Unicode (ISO/IEC 10646) character whose value is the hexadecimal value HHHHHHHH (one to eight hex digits)

They do have differences. In addition to the above, echo -e supports:

\c     suppress further output
\0nnn  the eight-bit character whose value is the octal value nnn (zero to three octal digits)

By contrast, $'....' supports:

 \'     single quote
 \"     double quote
 \nnn   the eight-bit character whose value is the octal value nnn (one to three digits)
 \cx    a control-x character

How to bash multithread?

I do think using xargs to run code multithread in bash is a better way, you still could use fifo. Just like the sample below:

#!/bin/bash
function a_sub
{
    sleep 3
}
tmp_fifofile="/tmp/$$.fifo"
mkfifo $tmp_fifofile
exec 6<>$tmp_fifofile

rm $tmp_fifofile

thread=15

for((i=0;i<$thread;i++));do
    echo
done >&6

for((i=0;i<50;i++));do
    read -u 6
    {
        a_sub && {
            echo "a_sub is finished"
        } || {
            echo "sub error"
        }
        echo >&6
    }&
done

exec 6>&-

wait

Useful bash functions

Get bash file absolute path

realpath(){
  path="$1"
  while [ -h "$path" ] ; do path="$(readlink "$path")"; done
  echo "$(cd "$(dirname "$path")"; echo -n "$(pwd)/$(basename "$path")")";
}

Log

log() {
  if [[ -n "$VERBOSE" ]]; then echo -e "[email protected]"; else test 1; fi
}

error() {
  echo "[email protected]" >&2
  exit 1
}
warning() {
  echo "[email protected]" >&2
}

function check_status {
  if [ $? -ne 0 ];then
    error ${@:-"Encountered an error, aborting!"}
  fi
}

Loop find

find . -type f -iname "*.txt" -print0 | while IFS= read -r -d $'\0' line; do
  echo "$line"
  ls -l "$line"
done

Invisible characters mess js and css up

Some invisible characters may cause program fault, and it’s hard to locate.

For example UTF-8 BOM in the middle of css file may interrupt the css parser, you can find it by
find . -name '*.js' -type f -print0 | xargs -0 grep -r $'\xEF\xBB\xBF'

UTF-8 Line Separator may cause the debugger misunderstand the right line, you can find it by
find . -name '*.js' -type f -print0 | xargs -0 grep -r $'\xe2\x80\xa8'

And you also can cat to display all non-printing characters
BSD cat
cat -evt file
GNU cat
cat -A file

Or sed (better for multi-byte characters)
sed -n "l" file

Shell script for running in multiple processes

If you wanna using multiple processes in shell script, you’d better use file descriptor to control it.
For example you want to process the files in current directory

#!/bin/bash
exec 3< <(ls ~)

for i in 1 2
do
(while read; do echo "Process $i received $REPLY."; sleep 1;done) <&3 &
done

wait

Of course, you can write it more commonly, just like


#!/bin/bash
function a_sub
{
sleep 3
}
tmp_fifofile="/tmp/$$.fifo"
mkfifo $tmp_fifofile
exec 6<>$tmp_fifofile
rm $tmp_fifofile

thread=15
for((i=0;i<$thread;i++));do
echo
done >&6
for((i=0;i<50;i++));do
read -u 6
{
a_sub && {
echo "a_sub is finished"
} || {
echo "sub error"
}
echo >&6
}&
done

exec 6>&-
wait