Category Archives: Coding

Write commands with multiple lines in Dockerfile

There are several ways to write commands with multiple lines in Dockerfile, for example you wanna echo a bash file entrypoint.sh with content:

#!/bin/bash
echo 3
echo 2
echo 1
echo run

You could:

Using printf

RUN printf '#!/bin/bash\necho 3\necho 2\necho 1\necho run' > /entrypoint.sh

Using cat

RUN sh -c "$(/bin/echo -e "cat > /entrypoint.sh <<EOF\
\n#!/bin/bash\
\necho 3\
\necho 2\
\necho 1\
\necho run\
\nEOF\n")"

Using echo -e

RUN echo -e " #!/bin/bash\n\
echo 3\n\
echo 2\n\
echo 1\n\
echo run" > /entrypoint.sh

Using $'...'

The $’…’ feature is known as "ANSI-C quoting" but it’s not a POSIX shell > feature. According to unix.stackexchange.com/a/371873/109111 it was > originally a ksh93 feature but it is now available in bash, zsh, mksh, > FreeBSD sh and in busybox’s ash

RUN echo $'#!/bin/bash\n\
echo 3\n\
echo 2\n\
echo 1\n\
echo run' > /entrypoint.sh

echo -e & $'...' are both similar in that they support the following escape sequences:

\a     alert (bell)
\b     backspace
\e
\E     an escape character
\f     form feed
\n     new line
\r     carriage return
\t     horizontal tab
\v     vertical tab
\\     backslash
\0nnn  the eight-bit character whose value is the octal value nnn (zero to three octal digits)
\xHH   the eight-bit character whose value is the hexadecimal value HH (one or two hex digits)
\uHHHH the Unicode (ISO/IEC 10646) character whose value is the hexadecimal value HHHH (one to four hex digits)
\UHHHHHHHH
     the Unicode (ISO/IEC 10646) character whose value is the hexadecimal value HHHHHHHH (one to eight hex digits)

They do have differences. In addition to the above, echo -e supports:

\c     suppress further output
\0nnn  the eight-bit character whose value is the octal value nnn (zero to three octal digits)

By contrast, $'....' supports:

 \'     single quote
 \"     double quote
 \nnn   the eight-bit character whose value is the octal value nnn (one to three digits)
 \cx    a control-x character

How to bash multithread?

I do think using xargs to run code multithread in bash is a better way, you still could use fifo. Just like the sample below:

#!/bin/bash
function a_sub
{
    sleep 3
}
tmp_fifofile="/tmp/$$.fifo"
mkfifo $tmp_fifofile
exec 6<>$tmp_fifofile

rm $tmp_fifofile

thread=15

for((i=0;i<$thread;i++));do
    echo
done >&6

for((i=0;i<50;i++));do
    read -u 6
    {
        a_sub && {
            echo "a_sub is finished"
        } || {
            echo "sub error"
        }
        echo >&6
    }&
done

exec 6>&-

wait

Useful bash functions

Get bash file absolute path

realpath(){
  path="$1"
  while [ -h "$path" ] ; do path="$(readlink "$path")"; done
  echo "$(cd "$(dirname "$path")"; echo -n "$(pwd)/$(basename "$path")")";
}

Log

log() {
  if [[ -n "$VERBOSE" ]]; then echo -e "[email protected]"; else test 1; fi
}

error() {
  echo "[email protected]" >&2
  exit 1
}
warning() {
  echo "[email protected]" >&2
}

function check_status {
  if [ $? -ne 0 ];then
    error ${@:-"Encountered an error, aborting!"}
  fi
}

Loop find

find . -type f -iname "*.txt" -print0 | while IFS= read -r -d $'\0' line; do
  echo "$line"
  ls -l "$line"
done

Process Substitution <(LIST) >(LIST)

Process substitution is supported on systems that support named pipes (FIFOs) or the /dev/fd method of naming open files. It takes the form of

<(list)
or
>(list)

The process list is run with its input or output connected to a FIFO or some file in /dev/fd. The name of this file is passed as an argument to the current command as the result of the expansion. If the >(list) form is used, writing to the file will provide input for list. If the <(list) form is used, the file passed as an argument should be read to obtain the output of list. Note that no space may appear between the < or > and the left parenthesis, otherwise the construct would be interpreted as a redirection.

For example:
$ cat < <(ls)
$ (echo "YES")> >(read str; echo "1:${str}:first";)> >(read sstr; echo "2:$sstr:two")> >(read ssstr; echo "3:$ssstr:three")

See also:

Bash Reference Manual
Bash Hackers Wiki

Shell script for running in multiple processes

If you wanna using multiple processes in shell script, you’d better use file descriptor to control it.
For example you want to process the files in current directory

#!/bin/bash
exec 3< <(ls ~)

for i in 1 2
do
(while read; do echo "Process $i received $REPLY."; sleep 1;done) <&3 &
done

wait

Of course, you can write it more commonly, just like


#!/bin/bash
function a_sub
{
sleep 3
}
tmp_fifofile="/tmp/$$.fifo"
mkfifo $tmp_fifofile
exec 6<>$tmp_fifofile
rm $tmp_fifofile

thread=15
for((i=0;i<$thread;i++));do
echo
done >&6
for((i=0;i<50;i++));do
read -u 6
{
a_sub && {
echo "a_sub is finished"
} || {
echo "sub error"
}
echo >&6
}&
done

exec 6>&-
wait

Weighted Round Robin In Nodejs

Round-robin (RR) is one of the algorithms employed by schedulers in computing. Jobs are assigned to each worker in circular order, It’s simple but useful.

I have a web server writing by node just for front-end display, APIs are most holding by back-end servers. So I have 2 functions supporting my structure. The first is health check helping me check whether the back-end server is ready; and the other one is round robin scheduling, to ensure an even distribution.
Continue reading

XBase64

Basics

xbase64 is a fast javascript base64 encoder and decoder;

Is it the fastest?

As far as I know it’s fast enough. Although it depends on far too many variables to enumerate, I wrote a test, the result below is for reference.

Tell me if you know a faster one. Continue reading

Webp-test

Webp-test is a javascript tool to check whether the browser support webp.

Why Webp-test?

WebP is an image format that employs both lossy and lossless compression. The degree of compression is adjustable so a user can choose the trade-off between file size and image quality. WebP typically achieves an average of 39% more compression than JPEG and JPEG 2000, without loss of image quality.
But a few browsers support WebP right now (as of April 2010, Google Chrome 9+ and Opera 11.10 beta), how to use it?
Now you can use webptest to test browser whether it supports.
Continue reading