TWpower Reading time ~1 minute

Pring docker images name and tag list with using awk and tail command.


Environment and Prerequisite

  • Linux base system
  • Bash shell(/bin/bash)
  • Docker
  • awk, tail command


Get docker images name and tag list

$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
nginx               latest              f68d6e55e065        2 weeks ago         109MB
nginx               stable              ac44715da54a        5 weeks ago         109MB
ubuntu              latest              7698f282e524        2 months ago        69.9MB
centos              centos7.5.1804      cf49811e3cdb        4 months ago        200MB
centos              latest              9f38484d220f        4 months ago        202MB
hello-world         latest              fce289e99eb9        6 months ago        1.84kB


Get docker images ID

  • Use -qa option to get IDs
  • -qa: Show all docker images and only show images ID
$ sudo docker images -qa
f68d6e55e065
ac44715da54a
7698f282e524
cf49811e3cdb
9f38484d220f
fce289e99eb9


Get docker images name

  • Use tail -n +2 command to exclude first line(first line is header of each column) and to print from second line.
  • Use awk to get first column.
  • $1 means first column
$ sudo docker images | tail -n +2 | awk '{print $1}'
nginx
nginx
ubuntu
centos
centos
hello-world


Get docker images tag

  • Use same awk command like above except modify variable from $1 to $2
  • $2 means second column
$ sudo docker images | tail -n +2 | awk '{print $2}'
latest
stable
latest
centos7.5.1804
latest
latest


Get docker images name with tag

  • Use same awk command like above except modify variable and expression
$ sudo docker images | tail -n +2 | awk '{print $1":"$2}'
nginx:latest
nginx:stable
ubuntu:latest
centos:centos7.5.1804
centos:latest
hello-world:latest


Reference

TWpower Reading time ~1 minute

업데이트(2020.03.28): 기본 형태 예제 추가

도커(Docker) 이미지를 파일로 저장하고 불러오자


환경

  • Linux 기반 시스템
  • Bash shell(/bin/bash)
  • Docker


도커(Docker) 이미지를 파일로 저장하기

docker save

기본 형태

docker save [OPTIONS] IMAGE(with tag) [IMAGE...]

기본 형태 예제

sudo docker save nginx:stable > nginx-stable-image.tar

sudo docker save nginx:stable -o nginx-stable-image.tar

옵션

  • --output , -o: 파일명을 명시합니다.

예시

  • docker images list
$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
nginx               stable              ac44715da54a        3 weeks ago         109MB
centos              centos7.5.1804      cf49811e3cdb        3 months ago        200MB
  • centos image example by using redirection
$ sudo docker save centos:centos7.5.1804 > centos-centos7.5.1804-image.tar

$ ls -al centos-centos7.5.1804-image.tar
-rw-rw-r-- 1 twpower twpower 207841280 Jul  4 22:55 centos-centos7.5.1804-image.tar
  • nginx image example by using -o option
$ sudo docker save nginx:stable -o nginx-stable-image.tar

$ ls -al nginx-stable-image.tar
-rw------- 1 root root 113060352 Jul  4 22:56 nginx-stable-image.tar


파일을 도커(Docker) 이미지로 불러오기

docker load

기본 형태

docker load [OPTIONS]

기본 형태 예제

$ sudo docker load < nginx-stable-image.tar

$ sudo docker load -i nginx-stable-image.tar

옵션

  • --input , -i: 불러올 파일명을 명시합니다.

예시

  • centos image example by using redirection
$ ls -al centos-centos7.5.1804-image.tar
-rw-rw-r-- 1 twpower twpower 207841280 Jul  4 22:55 centos-centos7.5.1804-image.tar

$ sudo docker load < centos-centos7.5.1804-image.tar
4826cdadf1ef: Loading layer  207.8MB/207.8MB
Loaded image: centos:centos7.5.1804

$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
centos              centos7.5.1804      cf49811e3cdb        3 months ago        200MB
  • nginx image example by using -i option
$ ls -al nginx-stable-image.tar
-rw------- 1 root root 113060352 Jul  4 22:56 nginx-stable-image.tar

$ sudo docker load -i nginx-stable-image.tar
da9fed87e1d3: Loading layer  54.59MB/54.59MB
0d1174230cc6: Loading layer  3.584kB/3.584kB
Loaded image: nginx:stable

$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
nginx               stable              ac44715da54a        3 weeks ago         109MB
centos              centos7.5.1804      cf49811e3cdb        3 months ago        200MB


참고자료

TWpower Reading time ~1 minute

Update(2020.03.28): Add basic form example

Save and Load docker image files.


Environment and Prerequisite

  • Linux base system
  • Bash shell(/bin/bash)
  • Docker


Save docker image to file

docker save

Basic Form

docker save [OPTIONS] IMAGE(by tag) [IMAGE...]

Basic Form Example

sudo docker save nginx:stable > nginx-stable-image.tar

sudo docker save nginx:stable -o nginx-stable-image.tar

Options

  • --output , -o: Specify output file name.

Examples

  • docker images list
$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
nginx               stable              ac44715da54a        3 weeks ago         109MB
centos              centos7.5.1804      cf49811e3cdb        3 months ago        200MB
  • centos image example by using redirection
$ sudo docker save centos:centos7.5.1804 > centos-centos7.5.1804-image.tar

$ ls -al centos-centos7.5.1804-image.tar
-rw-rw-r-- 1 twpower twpower 207841280 Jul  4 22:55 centos-centos7.5.1804-image.tar
  • nginx image example by using -o option
$ sudo docker save nginx:stable -o nginx-stable-image.tar

$ ls -al nginx-stable-image.tar
-rw------- 1 root root 113060352 Jul  4 22:56 nginx-stable-image.tar


Load docker image from file

docker load

Basic Form

docker load [OPTIONS]

Basic Form Example

$ sudo docker load < nginx-stable-image.tar

$ sudo docker load -i nginx-stable-image.tar

Options

  • --input , -i: Specify image file name that will be loaded.

Examples

  • centos image example by using redirection
$ ls -al centos-centos7.5.1804-image.tar
-rw-rw-r-- 1 twpower twpower 207841280 Jul  4 22:55 centos-centos7.5.1804-image.tar

$ sudo docker load < centos-centos7.5.1804-image.tar
4826cdadf1ef: Loading layer  207.8MB/207.8MB
Loaded image: centos:centos7.5.1804

$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
centos              centos7.5.1804      cf49811e3cdb        3 months ago        200MB
  • nginx image example by using -i option
$ ls -al nginx-stable-image.tar
-rw------- 1 root root 113060352 Jul  4 22:56 nginx-stable-image.tar

$ sudo docker load -i nginx-stable-image.tar
da9fed87e1d3: Loading layer  54.59MB/54.59MB
0d1174230cc6: Loading layer  3.584kB/3.584kB
Loaded image: nginx:stable

$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
nginx               stable              ac44715da54a        3 weeks ago         109MB
centos              centos7.5.1804      cf49811e3cdb        3 months ago        200MB


Reference

TWpower Reading time ~5 minutes

업데이트(2020.11.15): Victoria 버전을 추가했으며 Rocky 버전 또한 테스트 완료

업데이트(2020.11.15): Ubuntu 내용 추가 및 Python3와 pip3 사용으로 변경

업데이트(2019.10.13): 이슈 해결 방법들 추가

Openstack kolla image를 source를 이용해 빌드해보자


환경

  • Linux 기반 시스템(Mac, Ubuntu 18.04.5 LTS 그리고 CentOS 7.6에서 테스트 완료)
  • Bash shell(/bin/bash)
  • Docker
  • pip -> 2020년 1월부터 Python 2의 지원을 하지 않기 때문에 Python 2.X.X의 사용은 피해야 합니다.
  • pip3
  • Git


Kolla Image?

Kolla Image

  • Kolla는 Openstack 컴포넌트들을 컨테이너화하여 실제 운영할 수 있는 상태로 배포하는 것을 목표로 두고 있는 프로젝트입니다.
  • Kolla Image는 오픈스택의 컴포넌트(nova, neutron…)들을 컨테이너화하여 만든 이미지들을 의미합니다.
  • 관련 프로젝트로는 Openstack Helm 그리고 Kolla-ansible이 있습니다.
  • 관련 링크: https://docs.openstack.org/kolla/latest/


이미지 빌드 방법

빌드하기 전에!

  • 이미지를 빌드하는 방법에는 크게 binary와 source 두 가지 방법이 있는데 기본값은 binary이며 다른 작업 없이 kolla 저장소를 받아서 이미지를 빌드하면 됩니다.
  • binary는 저장소를 통해 외부에서 빌드된 컴포넌트 관련 파일들을 가져와 생성하며 source의 경우에는 소스코드들로부터 직접 빌드를 합니다.
  • 아래 시나리오는 source를 통해서 빌드하는 방법이며 nova 컴포넌트만 빌드해보겠습니다.
  • source로 빌드하기 위해서는 해당 컴포넌트의 소스파일이 필요합니다.(git repository에서 가져올 예정입니다.)
  • 해당 시나리오는 둘 다 Mac과 CentOS에서 테스트를 완료했습니다.
  • Ubuntu와 CentOS는 root 사용자에서 테스트 했습니다.
  • MacOS의 경우 sudo를 이용한 설치가 포함되어있습니다.
  • 개발을 목적으로 kolla안에 있는 Dockerfile들을 이용해 빌드하실거면 kolla-build 명령어 대신 kolla 안에 있는 tools/build.py를 사용하셔야 합니다.
  • 2020년 1월부터 Python 2의 지원을 하지 않기 때문에 Python 2.X.X의 사용은 피해야 합니다. 여기서 해당 내용 확인이 가능합니다.


0. 사전 준비

CentOS

sudo yum install epel-release -y
sudo yum install python-devel git python3 python-pip gcc -y

Ubuntu

sudo apt-get install git python3 python3-pip python-pip gcc -y

MacOS

brew install git python3 -y


1. Clone kolla repository

  • Clone kolla repository and switch to stable/victoria branch
  • This post will build stable/victoria version images.
  • Also done test on stable/rocky version.
$ git clone https://github.com/openstack/kolla.git
$ cd kolla/
$ git checkout stable/victoria


2. Install needed packages

  • You can use kolla-build command after installing kolla
# In kolla directory path
$ cd ..
$ pip3 install kolla/

Issue Solving - 1

  • Upgrade pip version if there is error like below in CentOS
  • After not using pip(which used in Python 2), this issue not occurs
...
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-GB1G0n/GitPython/
...
  • Upgrade pip version
pip install --upgrade pip

Issue Solving - 2

  • Remove yum package if there is error like below in CentOS
  • After not using pip(which used in Python 2), this issue not occurs.
...
ERROR: Cannot uninstall 'requests'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
...
  • Remove yum package
$ rpm -qa | grep requests
$ yum remove python-requests-2.6.0-1.el7_1.noarch -y


3. Generate kolla-build.conf

$ pip3 install tox
$ cd kolla/
$ tox -e genconfig
...
genconfig: commands succeeded
congratulations :)

Issue Solving 1 - Python.h

  • Install packages if there is error like below in CentOS
  • After not using pip(which used in Python 2), this issue not occurs.
...
configure: error: no acceptable C compiler found in $PATH
...
_posixsubprocess.c:16:20: fatal error: Python.h: No such file or directory
...
  • Install python-devel and gcc
$ sudo yum install python-devel gcc -y

Issue Solving 2 - more_itertools

  • Downgrade pip packagee if there is error like below in CentOS
  • After not using pip(which used in Python 2), this issue not occurs.
...
File "/usr/lib/python2.7/site-packages/more_itertools/more.py", line 340
  def _collate(*iterables, key=lambda a: a, reverse=False):
                             ^
SyntaxError: invalid syntax
...
  • Downgrade version of more-itertools
pip install more-itertools==5.0.0


4. Clone nova repository

  • Clone nova repository and switch to stable/victoria branch
  • Remember the path of nova directory
# In kolla directory path
$ cd ..
$ git clone https://github.com/openstack/nova.git
$ cd nova/
$ git checkout stable/victoria
$ pwd
/root/nova


5. Modify kolla-build.conf

  • Modify [nova-base] part’s type and location
  • (Option) Modify tag if you want to change image tag name
  • location value must be your nova directory path on your local computer
  • The location of the generated configuration file is etc/kolla/kolla-build.conf, it can also be copied to /etc/kolla. The default location is one of /etc/kolla/kolla-build.conf or etc/kolla/kolla-build.conf.
# In nova directory path
$ cd ..
$ cd kolla
$ vi etc/kolla/kolla-build.conf # File is in kolla repo path

...
[nova-base]

type = local

# Below value should be your local computer nova directory path
location = /root/nova
...

...
# The Docker tag (string value)
tag = victoria
...
  • Copy modified file etc/kolla/kolla-build.conf to /etc/kolla/kolla-build.conf
$ mkdir -p /etc/kolla/ && cp etc/kolla/kolla-build.conf /etc/kolla/kolla-build.conf


6. Build

  • Build nova from source code in kolla directory
# In kolla directory path
$ python tools/build.py -t source nova

# OR

$ kolla-build -t source nova
  • Build all components using binary
# In kolla directory path
$ python tools/build.py

# OR

$ kolla-build
  • Build specific component
# In kolla directory path
$ python tools/build.py keystone

# OR

$ kolla-build keystone
  • Build using specific os base image
  • Use -b option with ubuntu or centos
# In kolla directory path
$ python tools/build.py -b centos -t source nova

# OR

$ kolla-build -b centos -t source nova


이미지 빌드 결과

$ docker images
REPOSITORY                                TAG                 IMAGE ID            CREATED             SIZE
kolla/centos-source-nova-compute          victoria            5145ea1a6ba7        About an hour ago   2.48GB
kolla/centos-source-nova-novncproxy       victoria            d8338f6be7fa        About an hour ago   1.66GB
kolla/centos-source-nova-ssh              victoria            bbd9d11448b4        About an hour ago   1.61GB
kolla/centos-source-nova-compute-ironic   victoria            18d58cc4b892        About an hour ago   1.61GB
kolla/centos-source-nova-conductor        victoria            395fa15a5da4        2 hours ago         1.58GB
kolla/centos-source-nova-api              victoria            20cc9cbe4234        2 hours ago         1.58GB
kolla/centos-source-nova-scheduler        victoria            b8b967a7dc3d        2 hours ago         1.58GB
kolla/centos-source-nova-mksproxy         victoria            90e969a20b73        2 hours ago         1.58GB
kolla/centos-source-nova-serialproxy      victoria            b8a4f46b847d        2 hours ago         1.58GB
kolla/centos-source-nova-base             victoria            4fdef07ed1c6        2 hours ago         1.58GB
kolla/centos-source-novajoin-server       victoria            9359d838c42b        2 hours ago         956MB
kolla/centos-source-novajoin-notifier     victoria            7704d7b68a0a        2 hours ago         956MB
kolla/centos-source-novajoin-base         victoria            75aca84b6a12        2 hours ago         956MB
kolla/centos-source-openstack-base        victoria            215a22192097        2 hours ago         809MB
kolla/centos-source-nova-libvirt          victoria            1672530e7c27        2 hours ago         1.28GB
kolla/centos-source-base                  victoria            36674f426fd1        2 hours ago         314MB
centos                                    8                   0d120b6ccaa8        3 months ago        215MB

참고자료

TWpower Reading time ~5 minutes

Update(2020.11.15): Add Victoria version. Rocky version also tested

Update(2020.11.15): Added Ubuntu content and changed to use Python3 and pip3

Update(2019.10.13): Add issue solving parts

Build openstack kolla image by using source


Environment and Prerequisite

  • Linux base system(Done tests on Mac, Ubuntu 18.04.5 LTS and CentOS 7.6)
  • Bash shell(/bin/bash)
  • Docker
  • pip -> From January 1, 2020 Python 2 is no longer supported any more so avoid using Python 2.X.X.
  • pip3
  • Git


Kolla Image?

Kolla Image

  • Kolla’s mission is to provide production-ready containers and deployment tools for operating OpenStack clouds.
  • Kolla Image means containerized images of openstack components
  • Related projects are Openstack Helm and Kolla-ansible
  • Link: https://docs.openstack.org/kolla/latest/


How to build image

Before build!

  • There are two ways to build kolla images. One is from binary and the other is from source. Default value setting is binary.
  • binary build images by using remote binary component file. source build images from source codes.
  • Below scenario introduce build from source and build only nova component.
  • For using source, we need each components source code files.(We will clone it from git repository)
  • Below scenario is tested on both Mac and CentOS-7.6
  • In Ubuntu and CentOS, tested on root user.
  • In MacOS, use sudo command.
  • When developing Kolla it can be useful to build images using files located in a local copy of Kolla. Use the tools/build.py script instead of kolla-build command in all below instructions.
  • From January 1, 2020 Python 2 is no longer supported any more so avoid using Python 2.X.X. You can check it on here


0. Prerequisites

CentOS

sudo yum install epel-release -y
sudo yum install python-devel git python3 python-pip gcc -y

Ubuntu

sudo apt-get install git python3 python3-pip python-pip gcc -y

MacOS

brew install git python3 -y


1. Clone kolla repository

  • Clone kolla repository and switch to stable/victoria branch
  • This post will build stable/victoria version images.
  • Also done test on stable/rocky version.
$ git clone https://github.com/openstack/kolla.git
$ cd kolla/
$ git checkout stable/victoria


2. Install needed packages

  • You can use kolla-build command after installing kolla
# In kolla directory path
$ cd ..
$ pip3 install kolla/

Issue Solving - 1

  • Upgrade pip version if there is error like below in CentOS
  • After not using pip(which used in Python 2), this issue not occurs.
...
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-GB1G0n/GitPython/
...
  • Upgrade pip version
pip install --upgrade pip

Issue Solving - 2

  • Remove yum package if there is error like below in CentOS
  • After not using pip(which used in Python 2), this issue not occurs.
...
ERROR: Cannot uninstall 'requests'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
...
  • Remove yum package
$ rpm -qa | grep requests
$ yum remove python-requests-2.6.0-1.el7_1.noarch -y


3. Generate kolla-build.conf

$ pip3 install tox
$ cd kolla/
$ tox -e genconfig
...
genconfig: commands succeeded
congratulations :)

Issue Solving 1 - Python.h

  • Install packages if there is error like below in CentOS
  • After not using pip(which used in Python 2), this issue not occurs.
...
configure: error: no acceptable C compiler found in $PATH
...
_posixsubprocess.c:16:20: fatal error: Python.h: No such file or directory
...
  • Install python-devel and gcc
$ sudo yum install python-devel gcc -y

Issue Solving 2 - more_itertools

  • Downgrade pip packagee if there is error like below in CentOS
  • After not using pip(which used in Python 2), this issue not occurs.
...
File "/usr/lib/python2.7/site-packages/more_itertools/more.py", line 340
  def _collate(*iterables, key=lambda a: a, reverse=False):
                             ^
SyntaxError: invalid syntax
...
  • Downgrade version of more-itertools
pip install more-itertools==5.0.0


4. Clone nova repository

  • Clone nova repository and switch to stable/victoria branch
  • Remember the path of nova directory
# In kolla directory path
$ cd ..
$ git clone https://github.com/openstack/nova.git
$ cd nova/
$ git checkout stable/victoria
$ pwd
/root/nova


5. Modify kolla-build.conf

  • Modify [nova-base] part’s type and location
  • (Option) Modify tag if you want to change image tag name
  • location value must be your nova directory path on your local computer
  • The location of the generated configuration file is etc/kolla/kolla-build.conf, it can also be copied to /etc/kolla. The default location is one of /etc/kolla/kolla-build.conf or etc/kolla/kolla-build.conf.
# In nova directory path
$ cd ..
$ cd kolla
$ vi etc/kolla/kolla-build.conf # File is in kolla repo path

...
[nova-base]

type = local

# Below value should be your local computer nova directory path
location = /root/nova
...

...
# The Docker tag (string value)
tag = victoria
...
  • Copy modified file etc/kolla/kolla-build.conf to /etc/kolla/kolla-build.conf
$ mkdir -p /etc/kolla/ && cp etc/kolla/kolla-build.conf /etc/kolla/kolla-build.conf


6. Build

  • Build nova from source code in kolla directory
# In kolla directory path
$ python tools/build.py -t source nova

# OR

$ kolla-build -t source nova
  • Build all components using binary
# In kolla directory path
$ python tools/build.py

# OR

$ kolla-build
  • Build specific component
# In kolla directory path
$ python tools/build.py keystone

# OR

$ kolla-build keystone
  • Build using specific os base image
  • Use -b option with ubuntu or centos
# In kolla directory path
$ python tools/build.py -b centos -t source nova

# OR

$ kolla-build -b centos -t source nova


Image build result

$ docker images
REPOSITORY                                TAG                 IMAGE ID            CREATED             SIZE
kolla/centos-source-nova-compute          victoria            5145ea1a6ba7        About an hour ago   2.48GB
kolla/centos-source-nova-novncproxy       victoria            d8338f6be7fa        About an hour ago   1.66GB
kolla/centos-source-nova-ssh              victoria            bbd9d11448b4        About an hour ago   1.61GB
kolla/centos-source-nova-compute-ironic   victoria            18d58cc4b892        About an hour ago   1.61GB
kolla/centos-source-nova-conductor        victoria            395fa15a5da4        2 hours ago         1.58GB
kolla/centos-source-nova-api              victoria            20cc9cbe4234        2 hours ago         1.58GB
kolla/centos-source-nova-scheduler        victoria            b8b967a7dc3d        2 hours ago         1.58GB
kolla/centos-source-nova-mksproxy         victoria            90e969a20b73        2 hours ago         1.58GB
kolla/centos-source-nova-serialproxy      victoria            b8a4f46b847d        2 hours ago         1.58GB
kolla/centos-source-nova-base             victoria            4fdef07ed1c6        2 hours ago         1.58GB
kolla/centos-source-novajoin-server       victoria            9359d838c42b        2 hours ago         956MB
kolla/centos-source-novajoin-notifier     victoria            7704d7b68a0a        2 hours ago         956MB
kolla/centos-source-novajoin-base         victoria            75aca84b6a12        2 hours ago         956MB
kolla/centos-source-openstack-base        victoria            215a22192097        2 hours ago         809MB
kolla/centos-source-nova-libvirt          victoria            1672530e7c27        2 hours ago         1.28GB
kolla/centos-source-base                  victoria            36674f426fd1        2 hours ago         314MB
centos                                    8                   0d120b6ccaa8        3 months ago        215MB

Reference