Kubernetes version 1.13.2

Recently I use Job to run some one time tasks (later switch back to regular pod because we want to control it’s start point), it’s just like other pod controller in K8s.

Job

Job setup pod, note that: hostname job-name, check env pass

volume mount

Job type

In upgrade, usage

CronJob

crontab

这里补充一下，常用的关于disk排查的命令就是df, du, dd, lsof, lsblk, blkid, mount, fdisk, mkfs, sync, lvm. More about Linux Storage, see my blog <<Linux Storage System>>.

I get error messages when run docker load command, this is caused by disk space run out. How do I know the disk utilization?

Shows the amount of disk space used and available on Linux file systems.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

# disk space report tool, check mount point and fs
# -h: readable
# -T: file system type
$ df [-hT]

Filesystem             Size  Used Avail Use% Mounted on
/dev/mapper/rhel-root  241G  219G   23G  91% /
devtmpfs               3.9G     0  3.9G   0% /dev
tmpfs                  3.9G     0  3.9G   0% /dev/shm
tmpfs                  3.9G  8.6M  3.9G   1% /run
tmpfs                  3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/vda1             1014M  208M  807M  21% /boot
tmpfs                  783M     0  783M   0% /run/user/0

# check mounted filesystem of directory
df -h <directory>
# inode usage of directory
df -i <directory>

Shows total size of the directory and it’s subdirectories

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

# du: estimate file space usage

# 统计当前目录下文件大小，并按照大小排序
# -s: display only a total
# -m: --block-size=1M
# *: shell globs
du -sm * | sort -nr
# -BG: blocksize as G
du -s -BG * | sort -nr

# include hidden files
du -sch .[!.]* * | sort -h

# current directory total size
du -sm .

lsof 在这方面主要就是检查哪些文件正在被什么user, process使用:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

# see what is interacting with this file
# -l: show user id instead of user name
lsof -l /path/to/file/or/directory

# see files have been opened from a directory
# +D: directory option
lsof +D /var/log

# files opened by process name, wild match by beginning characters
lsof -c ssh
lsof -c init

# files opened by process id
# ^: exclude
lsof -p 5644,^3122

# files opened by user
lsof -u root,1011
lsof -u 1000

# ^: excluding root
lsof +D /home -u ^root

# -a: and operation
# default is or operation
lsof -u mary -c ssh -a

# find deleted file hold by which process
lsof -nP +L1 | grep '(deleted)'

当然在network上lsof也很强大: An lsof primer How to use lsof command

看这种书挺枯燥，还是learn by doing 比较好，使用一段时间就很明确自己需要什么功能了。

By default oc/kubectl edit will use vi in Linux if environment variable OC _EDITOR is empty, if you have .vimrc that is not compatible with vi, will get error message, to use vim set export OC_EDITOR=vim or export KUBE_EDITOR=vim.

Other resources

How I booted my Vim Informative!! I grab lots of configurations from it, go and poke around the author’s vimrc file.

Vim Tutorial Videos There are a lot, see O`Reilly.

Actually you can learn Vim by running command vimtutor in your terminal, it is a simple and quick tutorial. For a quick review, just see summary after each lesson.

Daily Foundation

掌握好这些操作基本上就满足日常需求了。

Configuration

.vimrc is just like .bashrc. you create a .vimrc (even it’s empty) to tell vim use vim not vi-compatiable mode!

关于vimrc的配置，参考my github repo: https://github.com/chengdol/vim-configuration

1. Basic configuration
2. Custom key mapping
3. Vimscript plugin

About mapping:

• nmap: normal mode
• vmap: visual mode
• imap: insert mode
• map: normal, visual and operating-pending mode
• map!: command and insert mode

To list mapping: :map or more specific :vmap

Which mode is currently use should be clear: Normal mode, Insert mode and Visual mode, Command-Line mode, Replace mode, etc. Visual mode lets us select text in the buffer and then operate on the selection.

If you suspect that your customizations are causing interference, here’s a quick test. Try quitting Vim and then launching it with these options:

1

vim -u NONE -N

The -u NONE flag tells Vim not to source your vimrc on startup. That way, your customizations won’t be applied and plugins will be disabled. In older versions of Vim, not loading a vimrc file activates vi compatible mode, which causes many useful features to be disabled. The -N flag prevents this by setting the nocompatible option. Since version 8.0 of Vim, the nocompatible option is set by default, making the -N flag unnecessary.

vim -N file: -N not using vi compatible mode, use newer vim mode. or can configuring in .vimrc file.

:set list, :set nolist, :set list?, :set list&: set, unset, check value, set as default. 以此类推.

1
2

set autochdir "auto set current dir as working dir
set wildmode=list:longest "activate TAB auto-complete file path

Reload .vimrc Reload .vimrc without restarting vim: :source ~/.vimrc. 已经加入vimrc了。

Plugin

Vim 8.0 has its own built-in package manager. For Vim version less than 8.0, I use vim-plug as the plugin manager: https://github.com/junegunn/vim-plug You can even specify git address of the plugin, for example:

1

Plug 'https://github.com/tfnico/vim-gradle.git'

If the plugin does not get auto installed, run :PlugInstall, check the plug status by :PlugStatus. 这些在vim-plug README中都有说明。

Please see my vim-configuration git repo, download the vimrc file.

• nerdtree: file system explorer for vim editor
• fuzzyfinder: fzf
• buffer explorer: 比默认的buffer要好看
• taglist: source code browser

NERDTree，在目录界面中通过m启动常规文件/文件夹操作。C回车 进入子文件夹。

Display Management

Vim中切换编辑多个文件： :cd /tmp change vim working directory :pwd show current working directory :set hidden (put it in .vimrc): https://medium.com/usevim/vim-101-set-hidden-f78800142855 :e . browse current directory, select file to edit

:ls list buffer (open files)

• %: buffer in current window
• +: unsave changes
• =: read-only

:bn go to next buffer :bp go to previous buffer :bd delete buffer, such as :bd5 :bf go to first buffer :bl go to last buffer :b3 go to 3rd buffer gf jump to the file under the cursor, use ctrl+o jump back

可以在vim :e <path> 中直接创建，删除文件或者文件夹，显示long format, sort等

在使用vim-plug加载插件后，这部分功能失效了, 不过可以使用command mode去做查看。

• i: long, thin, tree format
• s: sort by name, time, file size
• r: reverse sort order
• gh: hide or unhide dotfiles
• d: make a directory
• D: delete file or directory
• R: rename file or dir
• -: go up a directory

windows & splits

All start with ctrl+w:

• s: split horizontally

• h: move focus left

• l: move focus right

• v: split vertically

• j: move focus down

• k: move focus up

• w: cycle focus

• p: focus previous win

• c: close win current focus

• o: close all win except current focus

:h window-resize, check for window resize.

• +: enlarge windows, 5+
• -: reduce windows, 3-

Tab

各个文件单独的tab，不用划分window了, 这个部分是放在vimrc中的:

1
2
3
4
5
6
7
8
9
10
11
12
13

" Tab mappings
" The default leader key is \
let mapleader=","                        " remap leader key to ,
" invoke for example: ,tt
map <leader>tt :tabnew<cr>
map <leader>te :tabedit
map <leader>tc :tabclose<cr>
map <leader>to :tabonly<cr>
map <leader>tn :tabnext<cr>
map <leader>tp :tabprevious<cr>
map <leader>tf :tabfirst<cr>
map <leader>tl :tablast<cr>
map <leader>tm :tabmove

What is Leader key in Vim? https://stackoverflow.com/questions/1764263/what-is-the-leader-in-a-vimrc-file The Leader key is a way of extending the power of VIM’s shortcuts by using sequences of keys to perform a command.

Operation

Using hjkl (在后续许多命令中都有涉及) or arrow keys to move around, can have number ahead to indicate how many line to move.

:q! quit without saving. :wq quit with saving, always retouch the timestamp. sometimes using :q! and :wq!: https://unix.stackexchange.com/questions/88247/use-of-in-vim

:x the same as :wq, but write only when changes have been made. :w filename used to save file to filename. :w !sudo tee % write to sudo with current file name, % represent current file name. (用在保存更改read-only文件的内容)

Quick Movement A append at end of the line, a appends after cursor. I insert ata head of the line, i insert at cursor. W jump contiguous words, w jump one word. o will open a line below cursor, O open above. 3w move 3 words forward, 0 move to the start of the line. ^ go to first non-empty char in line 3fn find 3rd n char at this line. repeat by ;

Screen line movement

screen line指被terminal因宽度限制产生的行，并不是指原始的很长的那一行。 g0, gm, g$: start, middle, end of a line. gk, gj: move up/down in screan line (can use arrow instead k/j ).

Scrolling

Press G to move you to the bottom of the file. Type gg to move you to the start of the file. line number + G or line number + gg will go to the line specified. To go back to where you came from press Ctrl-o (Keep Ctrl down while pressing the letter o). To go forward Ctrl-i H, M and L move the cursor to top, medium and bottom of current page zt, zz, zb: move cursor line to top, middle, bottom of screen

Make Mark

For example, jump back and forth spots: https://www.linux.com/news/vim-tips-moving-around-using-marks-and-jumps/ :marks

Shift

Shift configuration is in .vimrc:

1
2
3
4
5
6
7

" shift with tab
set tabstop=2                    " Global tab width.
set shiftwidth=2                 " And again, related.
set expandtab                    " Use spaces instead of tabs
" extremely useful to edit yaml file
set autoindent                   " always set autoindenting on
set copyindent                   " copy the previous indentation on autoindentin

:retab: replace all tabs with spaces in current buffer.

>>, << shift current line, 4>>, 4<< block shift with 4 lines together. In insert mode, can use tab itself to shift (now override by auto-complete in my vimrc file), or using ctrl+t, ctrl+d: shift right and left.

对于code block indention，我在.vimrc 中设置了vmap, 用visual mode选中后shift就很方便了。

Search and Replace

/ search forward, ? search backword. when hit the search, n go forward, N go backword. If the cursor is at search word, use * to search forward, # search back, use g*, g# do partial match search.

/case\c search case-insensitive, /CaSe\C search case-sensitive. \c and \C can be anywhere in pattern.

:s/old/new/g to substitute old by new in a line for all occrurences. you must place the cursor in that line. To change every occurrence of a character string between two lines, :#,#s/old/new/g where #,# are the line numbers of the range of lines where the substitution is to be done. :%s/old/new/g to change every occurrence in the whole file. :%s/old/new/gc to find every occurrence in the whole file, with a prompt whether to substitute or not, c is confirmation.

For case-sensitive: :%s/old/new/Igc, but actually can be :%s/old\C/new/gc.

Copy and Paste

这里和vim 的 register有关. Default is unnamed register, delete, change, substitute, search and yank all use registers.

Pasting text into a terminal running Vim with automatic indentation enabled can destroy the indentation of the pasted text: https://vim.fandom.com/wiki/Toggle_auto-indenting_for_code_paste, use vim-bracketed-paste can fix this.

Show register contents: :registers

y is copy operator, for example, yw copy one word, y$ copy to end of the line and yy used to copy a line, then you can paste through p. you can use v to select and copy. By default it uses unnamed register. Register 0 always stores the yank content, so you can use "0p to paste.

"ayy, "ap use register a to yank and paste. Usually we use a~z and A~Z register, the uppercase register can append text to current content.

使用上不同文件之间copy/paste没啥问题，但如果需要copy到system clipboard，需要设置: https://stackoverflow.com/questions/3961859/how-to-copy-to-clipboard-in-vim vim has to be compiled with clipboard support for any of the suggestions mentioned here to work. Mine wasn’t configured that way on Mac OS X by default and I had to rebuild vim. Use this the command to find out whether you have it or not:

1

vim --version | grep 'clipboard'

+clipboard means you’re good and the suggestions here will work for you, while -clipboard means you have to recompile and rebuild vim.

d is the delete operator, use motion to specify the action, for example dw, de, d$. Without d, vim just move the cursor: w, e, $ 2dw delete 2 words ahead 4dd delete 4 lines in a row Type p to put previously deleted text after the cursor. for example, you delete a line and replace it in another place

v and then you move the cursor to select the text you want, if you want to delete them next, type d, if you want to save the selected text to another file, press :, then type w filename. V is moved linewise.

Folding

这个非常有用，比如编辑yaml 文件。 Fold by syntax or indent (yaml). :sed foldmethod=indent/syntax zM, zR: fold and unfold all zi: toggle fold all zc, zo: close, open fold block za: toggle fold block zk, zj: move fold focus up and down

还可以根据file type去设定, for example:

1
2
3

" Automatic fold settings for specific files. Uncomment to use.
" autocmd FileType ruby setlocal foldmethod=syntax
" autocmd FileType css  setlocal foldmethod=indent shiftwidth=2 tabstop=2

Recording

暂时没用到，对重复的复杂操作有用。

Block comment and uncomment

quick way to comment and uncomment block of codes comment:

1. press Esc
2. hit ctrl+v, enter visual block mode
3. use arrow keys select the lines, it won’t highlight everthing, it’s ok
4. shift+i
5. insert the text you want, e.g: #, //, %
6. press Esc twice

uncomment:

1. press Esc
2. hit ctrl+v, enter visual block mode
3. use arrow keys select text you want to delete
4. press x to delete the text

Others

Press u to undo the last commands, U to undo all the changes in a line. crtl+r to undo undo.

Type rx to replace the one character at the cursor with x. R is used to replace more then one chars, type R then get into insert mode.

ce deletes the word and places you in Insert mode, so you can type new word, it will replace old ones. c is change operator, just like d. 比如要更改3个连续的words 在某行中: c3w, then type.

Ctrl+g will show cursor location in the file and the file status, also the file name.

% is used tp match another part of (, [ or {. This is very useful in debugging a program with unmatched parentheses. 也可以用来match其他block语句，比如if-end。

:!<commands> is used to execute external commands, for example :!ls -ltr /tmp. (the same as IPython)

:r filename will retrive the file placed under the cursor. you can also read the output of an external command, :r !ls -ltr.

Python build-ins modules: https://docs.python.org/3/library/functions.html#open

Template

With Docstring, you can use help() command to get module information, for example:

1
2
3
4
5

## "words" is the script name: words.py
import words
help(words)
## fetch_words is a function inside words.py script
help(words.fetch_words)

Acutally help() works on every object.

This is a python script named words.py with demonstration for Docstring:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59

"""Retrieve and print words from a URL.

Usage:

   python3 words.py <URL>
"""

import sys
from urllib.request import urlopen


def fetch_words(url):
    """Fetch a list of words from a URL.

    Args:
        url: The URL of a UTF-8 text document.

    Returns:
        A list of strings containing the words from
        the document.
    """
    ## can use with-block here
    story = urlopen(url)
    story_words = []
    for line in story:
        line_words = line.decode('utf8').split()
        for word in line_words:
            story_words.append(word)
    ## file like object, still need to close
    story.close()
    return story_words


def print_items(items):
    """Print items one per line.

    Args:
        An iterable series of printable items.
    """

    for item in items:
        print(item)


def main(url):
    """Print each word from a text document from at a URL.

    Args:
        url: The URL of a UTF-8 text document.
    """

    words = fetch_words(url)
    print_items(words)


if __name__ == '__main__':
    ## pass parameter from command line
    ## argv[0] is the module name
    main(sys.argv[1])

On Linux, if you add shebang #!/usr/bin/env python3 at the top of the script, then you can run it by:

1
2

chmod +x words.py
./words.py http://sixty-north.com/c/t.txt

后面会专门学习一下python script方面的知识, see my blog <<Python3 Scripting>>.

Exception

try statements do not create a new scope! the variables in try block can be seen from outside try block:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

import sys

def convert(s):
    try:
        number = ''
        for token in s:
            number += DIGIT_MAP[token]
        return int(number)
    except (KeyError, TypeError) as e:
        ## !r is a shortcut to call the repr of the value supplied.
        print(f"Conversion error: {e!r}", file=sys.stderr)
        ## re-raising exception
        raise
    else:
        print("I will be called only if exception didn't occur!")
    finally:
        ## will execute no matter normal or not
        pass


def sqrt(x):
    """Compute square roots using the method 
    of Heron of Alexandria.

    Args:
        x: The number for which the square root 
            is to be computed.

    Returns:
        The square root of x.

      Raises:
          ValueError: If x is negative.
    """
    if x < 0:
        ## upper level can catch this exception
        raise ValueError(
            "Cannot compute square root of "
            f"negative number {x}")

    guess = x
    i = 0
    while guess * guess != x and i < 20:
        guess = (guess + x / guess) / 2.0
        i += 1
    return guess

Common exception types:

• indexError: index out of boundary
• keyError: mapping
• TypeError: usually avoiding check this, increase function usability.
• valueError: int(“hello”)
• OSError: os module open file

Modularity

Import module and attribute, 掌握import的语法, module normally is a single python source file, e.g. hello.py. When import, it is represented by module objects.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

## import custom module
import hello
## Hello is a class in hello.py
from hello import Hello

## system modules
import math
math.factorial(10)

from math import factorial
factorial(10)

from math import factorial as fac
fac(10)

# other import forms
from math import (factorial, exp)
from math import *

In the interactive python console, use help to explore modules:

1
2
3
4
5
6
7
8
9
10
11
12
13

# you can search for modules, keywords, symbols, topics
help()
>>> math
>>> urllib.request

# or check math module by first import it
import math
help(math)
## request is a nested module after urllib
import urllib.request
## or
from urllib import request as req
help(urllib.request)

You can check the attributes of an object:

1
2
3
4
5
6

## show all methods
dir(math)
dir(time.ctime)
## show type
type(math)
type(time.ctime)

Commonly used modules:

• requests: simple http library
• urllib: urllib is a package that collects several modules for working with URLs
• sys: access argv, stdin, stdout, etc.
• time: principally for working with unix time stamps
• datetime: UTC datetime, Unix timestamp, timedetla
• pprint: pretty print
• os: interface to system services
• itertools: iteration processing
• contextlib: use with with statement
• typing: type hints, built-in after python 3.5
• functools: functools.wraps() 用来 copy original function metadata

最近在做project的时候遇到几个新的modules:

• threading: threading operation
• subprocess: spawn new processes

time vs datetime modules: https://stackoverflow.com/questions/7479777/difference-between-python-datetime-vs-time-modules the time module is principally for working with unix time stamps; expressed as a floating point number taken to be seconds since the unix epoch. the datetime module can support many of the same operations, but provides a more object oriented set of types, and also has some limited support for time zones.

Function

Function name in python uses lowercase and - as delimiter. def keywork bind a function to a name, function in Python is treated as object.

Extended arguments, for example: *args(act as tuple), **kwargs(act as dict). This is called parameters packing, these applies to all types of callables, for example lambda.

The parameter type order must follow: regular positional -> *args -> keyword -> **kwargs, for example:

1
2
3
4
5
6
7
8
9

def print_args(arg1, arg2, *args, kwarg1, kwarg2, **kwargs):
    print(arg1)
    print(arg2)
    print(args)
    print(kwarg1)
    print(kwarg2)
    print(kwargs)

print_args(1, 2, 3, 4, 5, kwarg1 = 6, kwarg2 = 7, kwarg3 = 8, kwarg4 = 9)

The parameters after * must be passed by key word:

1
2
3
4

def function_name(para1, *, para2 = "hello", para3):
    pass

function_name(1, para2 = "world", para3 = 567)

Correspondingly, we have extended call syntax, unpacking when pass the parameters to function call, * is for tuple or list, ** is for dict.

Unpacking parameters:

1
2
3
4
5
6
7
8
9

def fun(a, b, c, d): 
    print(a, b, c, d) 

my_list = [1, 2, 3, 4] 
  
# Unpacking list into four arguments 
# 在变量前加单星号表示将元组（列表、集合）拆分为单个元素
# 双星号同上，区别是目标为字典，字典前加单星号的话可以得到“键”
fun(*my_list)

Positional-only arguments:

1
2
3

## no kwarg can be used here
def function_name(x, /):
    print(x)

If no return, then will implicitly return None.

1
2
3
4
5

def function_name(para1 = "hello", para2 = 34):
    ## rebind the global variable to local
    global count
    ## return None
    return

Notice that always use immutable value for default value!! Default value的赋值会在最初执行函数的时候运行一次，之后调用不会再重新赋值，看样子是一直存在内存里了。

1
2
3
4
5
6
7

def append_word(org=[]):
    org.append("!")
    return org
## if you call multiple times with default value, the "!" get accumulated
print(append_word()) # ["!"]
print(append_word()) # ["!", "!"]
print(append_word()) # ["!", "!", "!"]

Another example:

1
2
3
4
5
6
7

import time
def show_time(t = time.ctime()):
    print(t)

## the print will not get updated
show_time()
show_time()

Function is also an object, can be used as parameters:

1
2
3
4
5
6
7
8
9
10
11
12

def print_card(words):
    banner = "+" + "-" * (len(words) + 2) + "+"
    output = "| " + words + " |"
    lines = [banner, output, banner]
    print("\n".join(lines))
    print()

def print_words(printer, words):
    printer(words)

## print_card passed as parameter
print_words(print_card, "hello, world!")

*args and **kwargs 可以用来argument forwarding:

1
2
3

def trace(f, *args, **kwargs):
    res = f(*args, **kwargs)
    return res

Special Functions

Detect whether a module is run as a script or imported as a module.

1
2
3

# only execute function when it is run as a script
if __name__ == "__main__":
    main()

Functional Programming

The special function __call__, 使用后class object可以当做function来调用，__call__就相当于定义了一个调用接口，并且加上其他数据结构，可以实现caching的效果 stateful.

You can use timeit module timeit method to measure exection time.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

## resolver.py file
import socket

class Resolver:
    def __init__(self):
        self._cache = {}

    def __call__(self, host):
        if host not in self._cache:
            self._cache[host] = socket.gethostbyname(host)
        return self._cache[host]

    def clear(self):
        self._cache.clear()

    def has_host(self, host):
        return host in self._cache

Run in REPL:

1
2
3
4
5
6

from resolver import Resolver
res = Resolver()
## just like call function
res("www.google.com")
## call second time, the execution time reduces a lot
res("www.google.com")

How to know object is callable, use callable() function to test.

Lambda

Create anonymous callable objects, syntax: lambda [args]: expr, the args are separated by commas or empty, the body is a single expression.

1
2
3

## the key is assigned a callable function
## similar to Java
sorted(name_list, key=lambda name: name.split()[-1])

Functional-style Tools

map(): maps function to a sequence, lazy implementation, return iterator. filter(): remove elements from sequence which don’t meet some criteria, lazily. functools.reduce(): 2-arguments function with a sequence, reduce the sequence to one result.

Local Function

functions defined inside function.

1
2
3
4
5
6

## 实现了和前面lambda类似的功能
def sort_by_last_letter(strings):
    def last_letter(s):
        return s[-1]
    return sorted(strings, key=last_letter)
sort_by_last_letter(["hesd", "sddn", "pplea"])

Name resulation in the scope is checked by LEGB rule: Local -> Enclosing (the containing function) -> Global -> Build-in:

1
2
3
4
5
6
7
8
9
10
11

g = "global"
def outer(p = "param"):
    l = "local"
    def inner():
        print(g, p, l)
    inner()

## global param local
outer()
## this call is wrong!
outer.inner()

Local function usage cases:

1. define one-off functions close to their use.
2. code organization and readability.
3. similar to lambda but more general, can have mutliple expressions.

Local function can be returned, working with Closure(在返回local function时，对其需要的资源进行保留，防止被垃圾回收, keep enclosing-scope objects alive):

1
2
3
4
5
6
7
8
9
10
11
12

def enclosing():
    x = "closed over"
    def local_func():
        print(x)
    return local_func
lf = enclosing()
## call it
lf()

## check closure env
## (<cell at 0x106739e88: str object at 0x1067b5ab0>,)
lf.__closure__

Function factories, return other functions, returned function use both their own arguments as well as arguments to the factory.

1
2
3
4
5
6
7

def raise_to(exp):
    def raise_to_exp(x):
        return pow(x, exp)
    return raise_to_exp

square = raise_to(2)
square(9)

nonlocal is like global keyword, to name binding in enclosing scope. 有点类似于local function使用的全局变量，但只针对同一个local function.

Function Decorators

Allow you to modify existing functions or methods without changing their definition (在原函数中加入上下文). Decorators can be:

1. Local function 以及 closure 结合使用.
2. class, the class must implement __call__(), all class define variables are gave to decorated function.
3. instance of a class, can control decorator behavior via instance variable.

You can think decorator as a function accepting a function and returning a function (callable).

这里举一个local function作为decorator的例子，其他类型decorator暂时没用到:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

## f: the target decorated function
def escape_unicode(f):
    ## local function wrap
    def wrap(*args, **kwargs):
        ## in below, f is `city_name`
        res = f(*args, **kwargs)
        return ascii(res)
    ## wrap uses a closure to access f after escape_unicode returns
    return wrap

## original function
def city_name():
    return "Tomの"

## the city_name pass to decorator
@escape_unicode
def city_name():
    return "Tomの"

## now unicode is translated to ascii
print(city_name())

You can have multiple decorators, act in order 3->2->1:

1
2
3
4
5

@decorator1
@decorator2
@decorator3
def function():
    pass

Keep original function metadata, for example __name__ and __doc__, using functooks.wraps():

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

import functools

def noop(f):
    ## 
    @functools.wraps(f)
    def noop_wrapper():
        return f()
    return noop_wrapper

@noop
def hello():
    """Print a dummy message
    """
    print("hello. world!")

## check metadata is there
help(hello)
hello.__name__
hello.__doc__

Parameterized decorator的一个用途是检查传入原函数的参数，比如这里检查第二个参数不能为负数:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

def check_non_negative(index):
     ## real decorating part                                      
     def validator(f):                                                       
         def wrap(*args):                                                    
             if args[index] < 0:                                             
                 raise ValueError(                                           
                     'Argument {} must be non-negative.'.format(index))      
             return f(*args)                                                 
         return wrap                                                         
     return validator                                                        

## 这里实际上是调用了check_non_negative，返回的结果作为decorator 
## 和上面的用法不一样了                                                     
@check_non_negative(1)                                                      
def create_list(value, size):                                               
    return [value] * size

## good
create_list('a', 3) 
## bad
create_list('a', -4)

Basic

Unlike other programming languages, Python has no command for declaring a variable, python is dynamic type.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

# explicit conversion
int("234")
# toward 0 cutting
int("-342.134")
# 2 is the base
int("100", 2)
float("34.56")

# special
float("nan")
float("inf")
float("-inf")

None 
True, False
# True
bool(3.4)
bool("False")
# False
bool(0)
bool([])
bool(())
bool("")

Operators

Python will not perform implicit type conversion, for example "123" + 56 is wrong. Exception is in if and while condition.

Notice that == vs is when compare strings, == compare the value but is compares the identity equality, you can check the unique number by id(). And comparsion by value can be controlled programatically.

The logic operators are and, or and not. 这里注意它们会返回最后eval的值，可以利用这个特点:

1
2
3
4

## 999
"hello" and 999
### "world"
0 or 'world'

Check if an object is None using is operator. Function parameters and return are transferred using pass-by-object-reference.

Notice that sequence of the same type also support comparison, just like string comparison, item by item from left to right

1
2

(3, 99, 5, 2) < (5, 7, 3)
[5, 3, 1] > [1, 9, 2]

Control Flows

Python does not have switch statement, there are several ways to mimic it: Python switch replacements

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

## condition
if x > 100 and x <= 999:
    pass
elif x >= 34 or x < -23:
    pass
elif not x:
    pass
else:
    pass

## ternary
res = "big" if x >= 100 else "small"

## loop
while True:
    pass

for item in iterable:
    pass

break/continue

String

Unicode characters.

Python does not have a character data type, a single character is simply a string with a length of 1. Square brackets can be used to access elements of the string or slice string.

Escape will work on both " and '.

The same as Java, string in Python is immutable.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

## raw string, no escape
x = r"\n\r\x 234\n\r"
type(x[2])

## list str methods
help(str) ## have definition
dir(str)

## string length
len(str)
## more efficient than "+"
items = "+".join(["hello", "world"])
items.split("+")
"   abc   ".strip()

## _ is the dummy var that will not be used
up, _, down = "good:bad".partition(":")

## operator
"123" + "456"
"123" * 5

Python f-Strings is better and concise then format():

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

a = 23
b = "apple"
print(f"I have {a} {b}s")
## others
print("{1} and {0}".format(a, b))
print("%d and %s" % (a, b))

# Example of accessing the attributes of an object
import os
file = "/etc"
info = os.stat(file)
print("file {0} uid {1.st_uid}, size {1.st_size}".format(file, info))

# Example of specifying field width and precision
# Print table of powers of two and their square roots
import math
x = 1
for i in range(10):
    x = x * 2
    y = math.sqrt(x)
    print("{0:4}{1:10}{2:10.3f}".format(i, x, y))
    # print("%4d%10d%10.3f" % (i, x, y))

Bytes

In python3, Strings are represented by sequences of unicodes, but textual data in Linux is a sequence of bytes, we need to use encode() and decode() to convert python string to/from bytes.

You get byte object from HTTP request, need to convert to str to use.

1
2
3
4
5
6
7
8
9
10

x = "hello world"
type(x)
##convert to byte stream
data = x.encode("utf-8")
## convert back
x = data.decode("utf-8")

## byte string
b = b"hello world"
type(b)

List

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

## can have comma at end
a = [1, 2, "abc", 34, 76.887, "jark", -84.124]
a = []
## slice
a[1: -1: 3]
a[: 5: 3]
a[2:]
## reverse the sequence
a[::-1]
## remove element from list
def a[3]

a.append(998)
## join list, not append, they are different!
a += [1, 2, 3]
a.append([1, 2, 3])

a.pop()
a.index("abc")
a.count(34)
a.remove("abc")
a.insert(3, "apple")
a.reverse()
## reverse sort
a.sort(reverse = True)
## sort by length of each object
a.sort(key=len)

## copy the list, but they are all shallow copy!!
a[:]
a.copy()
list(a)

除了list自带的sort and reverse, out-of-place functions: sorted(), reversed() can also be used, they create a new list, reversed() will return a reversed iterator.

Dict

1
2
3
4
5
6
7
8
9
10
11
12

d = dict()
d = {}
## add or update
d["one"] = 34
del d["one"]

## key can be string, number and bool
d = {"k1": "v1", "k2", 123}
d = dict(a="343", b="uuys", c=123)
## can even be generated by tuple
a = [("sure", 643), (98, "341"), ("name", "monkey")]
d = dict(a)

The copy of dict is shallow.

1
2

d.copy()
dict(d)

merge dict:

1
2

## if keys are overlapped, the value will be updated by the merged one
d.update({"away": 998})

iterate dict via foreach loop:

1
2
3

for k in d.keys()
for v in d.values()
for k, v in d.items()

Use in and not in to check the existence.

Set

Immutable collection with unique immutable objects.

1
2
3
4
5
6
7
8

## s = () is tuple!
s = set()
s = {3, 676, 1, 34, 89}
type(s)

s.add(99)
s.update({5, 232, 89, -45})
s.remove(3)

Use in and not in to check the existence.

The copy of dict is shallow.

1
2

s.copy()
set(d)

Set 有很多代数运算法则可以使用:

1
2
3
4
5
6
7

s.union(t)
s.intersection(t)
s.difference(t)
s.symmetric_difference(t)
s.issubset(t)
s.issuperset(t)
s.isdisjoint(t)

Tuple

Tuples are unchangeable, or immutable as it also is called.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

## useless, because immutable
t1 = ()

t1 = ("apple", "banana", "cherry")
## create one item tuple, must append comma!
t2= ("apple",)
type(t2)

## this is also tuple, used in function return
t2 = 1, 2, 4, "ok"
## for example
def min_max(items):
    return min(items), max(items)

## tuple unpacking
lower, upper = min_max([2, 3, 7, 4, 34])
## can be nested unpacking
(a, (b, c)) = (1, (2, 3))
## swap value
a, b = b, a

Other operations:

1
2
3
4

t3 = ("hello", 10.23, 99)
len(t3)
t3 += ("world!")
t3 * 2

Range

1
2
3

range(stop)
range(start, stop)
range(start, stop, step)

Used usually for loop counter:

1
2

for i in range(10):
    pass

Other usages, for example, generate a list:

1

list(range(0, 10, 2))

Enumerate

If you want to have index pair with the item, use enumerate():

1
2
3

t = [3, 35, 546, 76, 123]
for i, v in enumerate(t):
    print(f"i = {i}, v = {v}")

Iteration and Iterables

Comprehensions with filtering

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

## list comprehension
[expr(item) for item in iterable if filtering(item)]
## set comprehension, no meaningful order
{expr(item) for item in iterable if filtering(item)}
## dict
{key_expr(item): val_expr(item) for item in iterable if filtering(item)}

## can also be mixed and nested
## this will create a list with 3 identical dict that value is None
## _ here is just for count
[{letter: None for letter in "ABCDEFG"} for _ in "123" if int(_) % 2 != 0]

## y is nested
## x is outer loop
[(x, y) for x in range(3) for y in range(5)]

Iterable can be passed to iter() to produce a iterator. Iterator can be passed to next() to get the next value in the sequence.

1
2
3
4

iterable = [1, 2, 3, 4, 5, 6, 7]
iterator = iter(iterable)
next(iterator)
next(iterator)

Generator function, stateful, laziness with yield:

1
2
3
4
5
6
7
8
9
10
11
12

def gen():
    yield 0
    a = 1
    b = 2
    while True:
        yield a
        a, b = b, a + b

for i in gen():
    print(i)
    if i > 1000:
        break

Generator expression, can save big memory than list comprehension:

1
2
3
4

(expr(item) for item in iterable if filtering(item))
## if used as function parameter, no parenthesis is needed
## 多余的()可以省掉
sum(x * x for x in range(10000000) if x % 2 == 0)

Generator is only single use object, 用完就没了，需要重新造一个。

There are several aggregation functions: bool: any(), all(); sync unlimited number of iterables zip():

1
2

for i, j in zip([1,2,3],[4,5,6]):
    print(f"output: {i}, {j}")

Class

Python does not have public, private, protected key word, everything is public.

Polymorphism is implemented by late binding in Python, 并不是和Java 通过继承实现多态，Python中你可以对一个Object 调用任意method，只要它有这个method就行，用的时候才会检查。 并且继承在Python中主要用来分享共用的方法，当然也可以来多态, 但是继承在python中用得不多。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

class Parent:
    """ A Parent class """

    ## no number() in this class, but child class has
    def get_treble_number(self):
        """ description """
        ## editor会报错，但构造parent()不会，除非你调用这个方法才会出错
        return self.get_number() * 3

## inheritance
class Child(Parent):
    """ A Child class """

    ## class variable, shared among all instances
    class_variable = 0
    ## initializer not quite a constructor
    ## self is similar to this in Java
    def __init__(self, number):
        """ description """

        ## _ prefix variable is class property, treated as private, although you can access it
        self._number = number
        ## still needs self to call method
        self._lst = self._gen_list()
        Child.class_variable += 1

    def _gen_list(self):
         """ description """
        return [None] + [item for item in range(1, 10)]

    def get_number(self):
        """ description """

        return self._number

    def get_double_number(self):
         """ description """

        ## method local variable
        double = self._number * 2
        return double

Parent.__doc__ can be used to access doc string.

注意，class method的第一个parameter self 可以是任意其他的名字，就是个标记而已，比如:

1
2
3

class Test:
    def __init__(other, number):
        other._number = number

File I/O and Resource Management

Check default encoding, if not specified in open(), will use this default encoding format.

1
2

import sys
sys.getdefaultencoding()

Open file with options, for example:

1
2
3

## mode can be any combination of `crwa`|`tb`
## r+ read/write
f = open('words.txt', mode = 'rwt', encoding = 'utf-8')

Some useful methods after open the file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

## read all return string
f.read()
## read one line
f.readline()
## read all lines return a list
f.readlines()

## use print, f is the file descriptor
print("wirte this line", file = f)
## write lines into file
f.writelines(["first\n", "second\n"])
## rewind to beginning
f.seek(0)
## close file after use, to flush updates to disk
f.close(0)

For reading file, you can also use loop:

1
2
3

for line in f:
    ## will not print \n, or you can use print(line.strip())
    sys.stdout.wirte(line)

Use with-block to auto close the resource (or you can use finally block)，不仅仅是用在file上，比如网络上的读写也可以，它们背后的实现都遵循了同样的规则，所以可以使用with-block:

1
2

with open(...) as f:
    pass

注意这个as 是可以省略的，在threading module的lock使用中，就没有as.

Data Structure

这里主要是和Java 对比，一些常用的数据结构，比如Stack, Queue, Deque, priorityQueue, Map, etc.

Queue

https://www.geeksforgeeks.org/queue-in-python/ There are 3 ways to implement queue in python:

1. list, quite slow by append() and pop(0), need shift all elements.
2. collections module import deque, can be used as queue.
3. queue module import Queue, A maxsize of zero 0 means a infinite queue. (this can be a synchronzied queue for thread programming)

Deque

see Queue section:

1. collections module import deque, can be used as queue.

Stack

https://www.geeksforgeeks.org/stack-in-python/

There are 3 ways to implement stack in python:

1. list, but when growing it has speed issues.
2. collections module import deque, can be used as stack (use this one).
3. queue module import LifoQueue, A maxsize of zero 0 means a infinite stack.

Priority Queue

https://www.geeksforgeeks.org/priority-queue-in-python/

By using heap data structure to implement Priority Queues, Time complexity: Insert Operation: O(log(n)) Delete Operation: O(log(n))

Min heap can be implemented by heapq module: https://www.geeksforgeeks.org/heap-queue-or-heapq-in-python/

Max heap is not implemented in heapq, the alternative way is invert the priority: https://stackoverflow.com/questions/2501457/what-do-i-use-for-a-max-heap-implementation-in-python

from queue import PriorityQueue, min heap too: https://www.geeksforgeeks.org/priority-queue-using-queue-and-heapdict-module-in-python

Map

use primitive type dict.

I have a post talks about IPC <<Linux IPC>>, it’s about IPC resource requirements for DB2 and how to adjust the ipc parameters permanently or temporarily. This time we face different ipc issue, interesting.

Description

We bring up DataStage container in particular order to make them function correctly, but when Engine is running, Xmeta DB2 went down silently whihout any warning or error messages. unquiesce will make DB2 up again, but it’s not stable.

This did not happen in K8s cluster, things get weired.

Reason

We update docker run command for Xmeta and Engine with --ipc=host, so they will share the ipc resouces with host machine (They reside on the same machine) and each other.

The trick is there are some ipcrm -a in the Engine start and quiesce scripts. So when Engine is up, it cleans host ipc resource and DB2 goes down.

we remove all ipcrm -a commands inside container and things get work.

Another thing is before start up the containers, make sure the ipc is clean, we encountered the problem that the old non-used ipc overstep in new containers, so run ipcrm -a in host machine to clean them.

Why K8s does not have this problem? Because we schedule Xmeta and Engine in different nodes, the ipcrm -a will not affect each other, but docker we hold all containers in one node.

Some usefully commands

1
2
3
4

ipcs -l      ## list ipc configuration

ipcrm -a     ## remove all ipc resources
ipcs -a      ## list ipc in use

Kubernetes version 1.13.2

I have talked about setting up pod network in Kubernetes cluster using Calico network add-on in this post <<Set Up K8s Cluster by Kubeadm>>. Recently I was involved in one issue from performance team, they complained that the network has bottleneck in calico, let’s see what happened and learn new things!

Description

The performance team set up a 6 nodes cluster with 1 master and 5 workers. Each machine has 48 cpu cores, 128GB memory, 2T+ disk and 10000Mb/s network speed.

These are the test cases:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

10 jobs for each user(8 small jobs, 1 middle job and 1 large job)

8 small jobs(RowGen -> XX -> Peek), 1 job unit
aggregate job                         -- 10 millions records (neally 2.2G)
filter job                            -- 10 millions records (neally 2.2G)
funnel_continuous                     -- 10 millions records (neally 2.2G)
funnel sort                           -- 10 millions records (neally 2.2G)
join job                              -- 10 millions records (neally 2.2G)
lookup                                -- 10 millions records (neally 2.2G)
sort job                              -- 10 millions records (neally 2.2G)
Transformation                        -- 10 millions records (neally 2.2G)

1 middle job, 2 job units
Middle_Job                            -- 50 million records(nearly 11G)

1 Large job, 3 job units
Large job                             -- 10 million records(nearly 2.1GB)

They ran concurrent users with N compute pods and found that the bottleneck is in calico network:

BTY, there are still enough resources(CPU, Memory and Disk I/O) to support DataStage scale up for DS concurrent users to run jobs on nodes and pods. But the network bandwidth between pods is not enough to support it.

iperf3 Command

They use iperf3, a TCP, UDP, and SCTP network throughput measurement tool, measure memory-to-memory performance access a network, server-client mode.

To install, for example in Centos:

1

sudo yum install iperf3 -y

The Usage is simple. More simple demos

Node to Node

For command options and flags see user guide.

This will transfer /large_file in client to /large_file in server, time interval is 40 seconds.

1
2
3
4
5
6
7
8
9
10

## server
# -s: server
# -F: read data from network and write to the file
iperf3 -s -F /large_file
## client
# -c: client mode connect to server ip
# -i 1: 1 sec report
# -t 40: transfer time 40 sec
# -F: read from file and write to network
iperf3 -c <server ip> -i 1 -t 40 -F /large_file

UDP traffic benchmark:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

## server
# -s: server
# -f M: format Mbytes/sec
# -V: verbose
# -p 5999: port 
iperf3 -s -f M -V -p 5999

## client
# -p 5999: port 
# -c: client mode connect to server ip
# -u: udp
# -b 24M: bandwidth 24Mbits/sec -> 3Mbytes/sec
# -t 600: time 600 sec
# -l 300: packet len 300 bytes
# so IP packet rate is around 3Mbytes/300bytes = 10K/sec
iperf3 -p 5999 -c 10.19.1.15 -u -b 24M -t 600 -l 300

Pod to Pod

The same as Node to Node, but wget and build iperf3 inside pod container and use the container’s IP (in container run hostname -I), for example, I flood data from is-en-conductor-0 pod to is-engine-compute-12 pod, they reside in different host machine.

Thinking

After I reproducing the tests, I was thinking Calico is a widely used add-on that shouldn’t have such obvious bottleneck, otherwise many people will complain and improve it.

Is there any improper configuration?

• Configuring IP-in-IP By default, the manifests enable IP-in-IP encapsulation across subnets (additional overhead compare to non IP-in-IP), if don’t need it (when? I am not very clear), disable it in calico manifest yaml file:

  1 2 3

  # Enable IPIP - name: CALICO_IPV4POOL_IPIP value: "off"

  See this IP-in-IP issue I am using Calico version 3.3, document about IP-in-IP

• Which Network Interface is Used Another question I have is which network interface is used for node-to-node and pod-to-pod test?

  There are several network interfaces in host machine, one is for public IP with MTU 9000 and in K8s we use private IP interface with MTU 1500. This will have impact on iperf3 testing.

  It shows that pod-to-pod test uses MTU 1500 but node-to-node uses MTU 9000.

  Need to test after enlarging MTU size to see if that change improves network throughput, also remember to update Calico manifest yaml, refer this document

  1 2	# Configure the MTU to use veth_mtu: "9000"

ethtool Command

The ethtool can display speed property of the network interface, for example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

# ethtool eno1

Settings for eno1:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Half 1000baseT/Full
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Half 1000baseT/Full
        Advertised pause frame use: Symmetric
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Link partner advertised link modes:  10baseT/Full
                                             100baseT/Full
                                             1000baseT/Full
        Link partner advertised pause frame use: Transmit-only
        Link partner advertised auto-negotiation: Yes
        Link partner advertised FEC modes: Not reported
        Speed: 1000Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: on
        MDI-X: on
        Supports Wake-on: g
        Wake-on: d
        Current message level: 0x000000ff (255)
                               drv probe link timer ifdown ifup rx_err tx_err
        Link detected: yes

The output depends on the what the network driver can provide, you may get nothing if the virtual machine does not have much data available (for example, in IBM softlayer cluster), refer to this question

In a virtual machine the link speed or duplex mode is usually meaningless, as the network interface is most often just a virtual link to the host system, with no actual physical Ethernet layer. The speed is as high as the CPU and memory can handle (or as low, as the connection rate limit is configured), cabling type does no exist, as there is no cable, etc. This virtual interface is bridged or routed to the actual physical network by the host system and only on the host system the physical port parameters can be obtained.

you can check if the network interface is virtual or not:

1
2
3
4
5
6

ls -l /sys/class/net

# output example
# both virtual: virtio1
lrwxrwxrwx. 1 root 0 Apr 27 16:11 eth0 -> ../../devices/pci0000:00/0000:00:04.0/virtio1/net/eth0
lrwxrwxrwx. 1 root 0 Apr 27 16:11 lo -> ../../devices/virtual/net/lo

Summary

1. Performance test big picture
2. iperf3, node-to-node, pod-to-pod tests
3. ethtool
4. Calico configutation: IP-in-IP, MTU
5. calicoctl, haven’t got time to learn

Other Blogs

k8s calico flannel cilium 网络性能测试 Benchmark results of Kubernetes network plugins (CNI) over 10Gbit/s network

I have some doubts about dnsdomainname command in docker container when I developed non-root DataStage, in Engine Conductor tier.

References to Configure container DNS

How can Docker supply each container a hostname and DNS configuration without having to build a custom image with the hostname written inside? The trick is to overlay three crucial /etc files inside the container with virtual files where it can write fresh information.

In container, run:

1
2
3
4
5

# mount | grep etc

/dev/mapper/rhel-root on /etc/resolv.conf type xfs (rw,relatime,attr2,inode64,noquota)
/dev/mapper/rhel-root on /etc/hostname type xfs (rw,relatime,attr2,inode64,noquota)
/dev/mapper/rhel-root on /etc/hosts type xfs (rw,relatime,attr2,inode64,noquota)

Four different options affect container domain name services, please see official documentation for explanation:

• -h HOSTNAME or --hostname=HOSTNAME
• --link=CONTAINER_NAME or ID:ALIAS
• --dns=IP_ADDRESS
• --dns-search=DOMAIN
• --dns-opt=OPTION

So what should dnsdomainname return exactly? for non-root engine docker container, the hostname is is-en-conductor-0.en-cond, but why sometimes dnsdomainname return empty but sometimes return en.cond.

I find the return depends on /etc/hosts file first non-self-loop IP hostname pair, for example:

1

9.30.223.186    is-en-conductor-0.en-cond is-servicesdocker is-xmetadocker

the dnsdomainname will return en-cond, but if change to

1

9.30.223.186    is-xmetadocker is-en-conductor-0.en-cond is-servicesdocker

the command returns empty, and if you run hostname -f (long host name), it returns is-xmetadocker.

From the CloudPak certification requirments, the uid for non-root user in container should be set in a higher range. We change it from 1000 to 100321000 but the image build got stuck and failed.

The reason is there is a range for uid number, it’s described in /etc/login.defs file (you can edit this file).

1
2
3
4

cat /etc/login.defs | grep -i UID

UID_MIN                  1000
UID_MAX                 60000

So if you create a user by useradd without specify user id explicitly, its uid will start from 1000. Also when change the uid by usermod -u <new uid> <user name>, you need to follow the limitation.

The gid has the same restriction:

1
2
3
4

cat /etc/login.defs | grep -i GID

GID_MIN                  1000
GID_MAX                 60000

09/29/2021 docker commit is not a good way to create docker image as it makes image size bigger, using multi-stage build instead.

From major security hardening items: Try to explicitly set a USER/uid – to avoid even accidental startup as root. This is a good point I agree. But in our case it’s impossible to use USER directive with non-root user in dockerfile at beginning, as we need root user to install and configure services in containers, then alter the settings that cater to non-root user.

The solution is in the last docker commit, use --change 'USER 1000' to set default user as non-root. For example:

1
2
3
4

docker commit --change 'USER 1000' \
--change 'ENTRYPOINT ["/opt/xx/initScripts/startcontainer.sh"]' \
-c 'ENV SETUPINPROGRESS ""' \
${SERVICES_HOST} ${DOCKER_TEMPIMAGE_TAG_SERVICES}:3

Note that the non-root user must exist in image, if it belongs to multiple groups (one primary and several supplementaries), only specify id is enough.

If later we need to run as root, just specify runAsUser: 0 in K8s yaml or --user 0 in docker run command, it will overwrite the default setting.

You can use docker inspect to check default USER:

1

docker inspect <image>:<tag> | grep -i user

Jenkins is a great tool for continuous integration and continuous delivery.The CI/CD and their responsibilities:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

            +-----------------------+           +----------------------+           +------------------------+
            |                       |           |                      |           |                        |
            |   continuous          |           |   continuous         |           |   continuous           |
            |         integration   +----------->         delivery     +-----------+       deployment       |
            |                       |           |                      |           |                        |
            +---------+-------------+           +--------------+-------+           +---------------------+--+
                      ^                                        ^                                         ^
                      |                                        |                                         |
                      |                                        |                                         |
+---------------------+----------+              +--------------+----------------------+      +-----------+--------------------+
|  kick off depends on commits   |              |  platform specific testings:        |      | ready to be used by customers  |
|  or schedule                   |              |  security, performance, API...      |      |                                |
+-----------+--------------------+------+       |                                     |      +--------------------------------+
            |   code version control    |       +-------------------------------------+
            |   branching strategy      |
+-----------+-------------+-------------+
|  regression testings    |
|                         |
+-------------------------+

Regression testing: re-running functional and non-functional tests to ensure that previously developed and tested software still performs after a change. If not, that would be called a regression. Changes that may require regression testing include bug fixes, software enhancements, configuration changes, etc.

Git branching (tag): Branching strategy: feature branch and primary development branch

Devops practice: Blue-green deploy, blue live and green idle. Smoke/Sanity test TDD test-driven development A/B testing, user experience research methodology.

Resource

What is CI/CD? Continuous integration and continuous delivery explained Continuous integration is a coding philosophy and set of practices that drive development teams to implement small changes and check in code to version control repositories frequently.

Continuous delivery automates the delivery of applications to selected infrastructure environments.

A mature CI/CD practice has the option of implementing continuous deployment where application changes run through the CI/CD pipeline and passing builds are deployed directly to production environments.

A best practice is to enable and require developers to run all or a subset of regressions tests in their local environments (or use Travis). This step ensures that developers only commit code to version control after regression tests pass on the code changes.

Test that require a full delivery environment such as performance and security testing are often integrated into CD and performed after builds are delivered to target environments.

To recap, CI packages and tests software builds and alerts developers if their changes failed any unit tests. CD is the automation that delivers changes to infrastructure and executes additional tests.

CI/CD is a devops best practice because it addresses the misalignment between developers who want to push changes frequently, with operations that want stable applications.