TWpower Reading time ~1 minute

파이썬(Python) List Comprehensions에 대해 알아보자.


환경

  • Linux
  • Python 3.X.X


파이썬(Python) List Comprehensions

List Comprehensions

results=[]
for i in range(10):
    results.append(i*2)

results=[x*2 for x in range(10)]
  • List Comprehensions: 파이썬에서 리스트 자료구조를 조금 더 간결하게 만들 수 있도록 도와주는 기능입니다.
  • 리스트를 새로 생성할 때 각각의 원소가 어떠한 연산의 결과이거나 조건을 만족하는 부분집합을 만들 때 사용할 수 있습니다.


형태

list_name = [element_format for clause]
  • [ ]로 감싸고 제일 앞에 추가할 원소의 형태를 넣으며 그 다음에 for문을 넣습니다.
  • 아래에 나오지만 for문 중첩도 가능합니다.


예제

제곱

results=[]
for i in range(10):
    results.append(i**2)

results=[x**2 for x in range(10)]

튜플 만들기

results=[]
for x in [1,3,5]:
    for y in [2,4,6]:
        results.append((x,y))

results=[(x,y) for x in [1,3,5] for y in [2,4,6]]

함수와 함께 사용하기

from math import pi

results=[]
for i in range(1, 6):
    results.append(str(round(pi, i)))

from math import pi
results=[str(round(pi, i)) for i in range(1, 6)]

중첩

results=[]
for x in [1,2]:
    for y in [3,4]:
        for z in [5,6]:
            results.append((x,y,z))

results=[(x,y,z) for x in [1,2] for y in [3,4] for z in [5,6]]

조건문과 함께 사용하기

results=[]
for x in range(20):
    if x%2 == 0:
        results.append(x)

results=[x for x in range(20) if x%2 == 0]


참고자료

TWpower Reading time ~1 minute

Practice Python List Comprehensions


Environment and Prerequisite

  • Linux
  • Python 3.X.X


Python List Comprehensions

List Comprehensions

results=[]
for i in range(10):
    results.append(i*2)

results=[x*2 for x in range(10)]
  • List Comprehensions: List comprehensions provide a concise way to create lists. It can make list much simpler.
  • It can be used when you make new list which is result of each element’s operation or subset elements of some conditions.


Format

list_name = [element_format for clause]
  • Cover with brackets([ ]), add element format in the front and add for clause in the end.
  • As like below example, we can also use nested for loop.


Examples

Square

results=[]
for i in range(10):
    results.append(i**2)

results=[x**2 for x in range(10)]

Make tuple

results=[]
for x in [1,3,5]:
    for y in [2,4,6]:
        results.append((x,y))

results=[(x,y) for x in [1,3,5] for y in [2,4,6]]

Use with function

from math import pi

results=[]
for i in range(1, 6):
    results.append(str(round(pi, i)))

from math import pi
results=[str(round(pi, i)) for i in range(1, 6)]

Nested loop

results=[]
for x in [1,2]:
    for y in [3,4]:
        for z in [5,6]:
            results.append((x,y,z))

results=[(x,y,z) for x in [1,2] for y in [3,4] for z in [5,6]]

Use with if statement

results=[]
for x in range(20):
    if x%2 == 0:
        results.append(x)

results=[x for x in range(20) if x%2 == 0]


Reference

TWpower Reading time ~1 minute

systemd service의 Type 옵션 중에서 simple과 oneshot의 차이를 정리해보자.


환경

  • Linux
  • systemd


목표

  • 평소에 모호했던 systemd service의 Type 옵션중 하나인 simple과 oneshot의 차이에 대해 직접 정리해본다.
  • 이 포스트에서 systemd나 service에 대해서는 정리하지 않습니다.
  • Reference를 참고해 작성하였습니다.


systemd service type

simple

[Service]
Type=simple
  • 메인 프로세스가 시작되면 바로 systemd unit이 정상적으로 실행되었다고 판단하고 active 상태로 바꾼다. 실행된 메인 프로세스가 끝날때까지 기다려주지 않고 연결된 다음 systemd unit을 실행한다. 만약 네트워크를 다 올리는 작업같이 기다렸다가 다음 systemd unit을 실행해야하는 상황이면 다른 옵션과 함께 사용하던가 아래 oneshot을 이용해야 한다.
  • ExecStart=를 하나만 가질 수 있다.
  • 프로세스가 실행되면 active 상태가 되며 다 끝났을때 inactive가 됩니다.


oneshot

[Service]
Type=oneshot
  • 메인 프로세스가 시작되면 상태를 activating으로 바꾸고 끝나때까지 기다리며 해당 메인 프로세스가 끝나야지만 다음 systemd unit으로 넘어간다.
  • ExecStart=를 여러개 가질 수 있다.
  • 프로세스가 실행되면 activating 상태가 되며 프로세스가 다 끝나고 inactive 상태로 바뀐다. 만약, RemainAfterExit=true 옵션을 사용하면 프로세스가 다 끝났을때 inactive가 아닌 active 상태가 된다.(실행되고 있는 프로세스는 없을수도 있다!)


둘의 공통점

  • forking된 프로세스에 대해서는 신경쓰지 않으며 자식 프로세스를 생성하고 싶다면 forking 옵션을 사용해야한다.


참고자료

TWpower Reading time ~1 minute

Summarize difference between simple and oneshot in systemd service’s Type option


Environment and Prerequisite

  • Linux
  • systemd


Goal

  • Summarize by myself about difference between simple and oneshot in systemd service’s Type option.
  • This post does not explain about systemd or service
  • I referred to many things on Reference.


systemd service type

simple

[Service]
Type=simple
  • It changes its state to active because its option considers process started successfully just after starting the main process. It does not wait until the main process exits and moves to next systemd unit. If you are in condition like wait for the network up and then run the next systemd unit, you rather use oneshot option or use other options.
  • It can only has one ExecStart=.
  • State will be changed to active after running process and it will be changed to inactive after finish of it.



oneshot

[Service]
Type=oneshot
  • It waits until the main process exits in activating state after running the main process. It moves to next systemd unit after the main process exits.
  • It can have multiple ExecStart=.
  • State will be changed to activating after running process and it will be changed to inactive after finish of it. If use RemainAfterExit=true option, then state will be changed to active not inactive(even though there is no running process!)


common things between two

  • Both don’t care about forking child processes. Use Type=forking instead.


Reference

TWpower Reading time ~4 minutes

cf dev를 이용해 Cloud Foundry의 CFAR(Cloud Foundry Application Runtime)에 파이썬 앱을 올려보자.


환경

  • Ubuntu 18.04
  • Baremetal Server


목표

  • Cloud Foundry에서 CFAR(Cloud Foundry Application Runtime)의 기본을 실습해보기 위해 cf dev를 이용해 파이썬 어플리케이션을 올려보자.
  • 해당 포스트는 실습 위주이며 참고자료에 있는 내용을 토대로 작성하였습니다.


Cloud Foundry

Cloud Foundry

  • Cloud Foundry: Cloud Foundry Foundation에 의해서 운영되는 멀티클라우드 오픈소스 platform as a service (PaaS).
  • 많은 프로젝트들이 들어가 있으며 배포, 관리 및 앱 환경 구축에 이르기까지 다양한 서비스를 제공한다.
  • 예시) BOSH 그리고 CFAR

CFAR(Cloud Foundry Application Runtime)

  • CFAR: Cloud Foundry Application Runtime (CFAR)은 개발자가 코드에만 신경쓸 수 있게 해주는 코드 중심 플랫폼입니다. 어떤 언어나 프레임워크인지 상관없이 코드를 받아서 어떤 클라우드에서라도 실행할 수 있게 해줍니다.
  • 코드를 작성하고 설정 파일을 추가한 다음에 CFAR에 업로드하면 앱 실행환경을 만들 수 있습니다.

CF Dev

  • CF Dev: CF Dev는 개발자 컴퓨터에 하이퍼바이저와 BOSH Director를 이용해 Cloud Foundry를 사용할 수 있도록 고안된 배포판입니다.
  • CF Dev를 이용하면 개인의 컴퓨터에서 간단하게 CFAR의 기능을 이용해 앱을 구축할 수 있습니다.
  • 본 글에서는 간단한 파이썬 플라스크(Flask) 앱을 만들어볼 예정입니다.


CF Dev

CF Dev 설치

  • QEMU와 libvirt 설치
sudo apt install qemu-kvm libvirt-bin libvirt-doc -y
  • CF CLI 설치
# ...first add the Cloud Foundry Foundation public key and package repository to your system
wget -q -O - https://packages.cloudfoundry.org/debian/cli.cloudfoundry.org.key | sudo apt-key add -
echo "deb https://packages.cloudfoundry.org/debian stable main" | sudo tee /etc/apt/sources.list.d/cloudfoundry-cli.list
# ...then, update your local package index, then finally install the cf CLI
sudo apt-get update -y
sudo apt-get install cf-cli -y
  • CF Dev plugin 설치
cf install-plugin -r CF-Community cfdev


CF Dev 환경 구축

  • cf dev start
cf dev start
  • 결과 화면
Downloading Resources...
Progress: |====================>| 100.0%
Setting State...

CF Dev collects anonymous usage data to help us improve your user experience. We intend to share these anonymous usage analytics with user community by publishing quarterly reports at :

https://github.com/pivotal-cf/cfdev/wiki/Telemetry

Are you ok with CF Dev periodically capturing anonymized telemetry [y/N]?> y
WARNING: CF Dev requires 8192 MB of RAM to run. This machine may not have enough free RAM.
Creating the VM...
Starting the VM...
Fetching VM Address...
Waiting for the VM...
Deploying the BOSH Director...
Deploying CF...
  Done (11m24s)

 	  ██████╗███████╗██████╗ ███████╗██╗   ██╗
 	 ██╔════╝██╔════╝██╔══██╗██╔════╝██║   ██║
 	 ██║     █████╗  ██║  ██║█████╗  ██║   ██║
 	 ██║     ██╔══╝  ██║  ██║██╔══╝  ╚██╗ ██╔╝
 	 ╚██████╗██║     ██████╔╝███████╗ ╚████╔╝
 	  ╚═════╝╚═╝     ╚═════╝ ╚══════╝  ╚═══╝
 	             is now running!

 	To begin using CF Dev, please run:
 	    cf login -a https://api.dev.cfdev.sh --skip-ssl-validation

 	Admin user => Email: admin / Password: admin
 	Regular user => Email: user / Password: pass

 	To deploy a particular service, please run:
 	    cf dev deploy-service <service-name> [Available services: mysql]


로그인

  • Email: user / Password: pass
cf login -a https://api.dev.cfdev.sh --skip-ssl-validation
  • 결과 화면
API endpoint: https://api.dev.cfdev.sh

Email: user

Password:
Authenticating...
OK

Targeted org cfdev-org

Targeted space cfdev-space



API endpoint:   https://api.dev.cfdev.sh (API version: 3.76.0)
User:           user
Org:            cfdev-org
Space:          cfdev-space


어플리케이션 파일 작성

  • 4개의 파일이 필요합니다.
  • 폴더 생성
mkdir flask_test_cf
cd flask_test_cf

app.py

from flask import Flask
import os

app = Flask(__name__)

@app.route('/')
def hello_world():
    return "Hello World!"

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=int(os.getenv("PORT", 5001)))

Procfile

  • CFAR에서 실행할 명령어
web: python app.py

requirements.txt

  • pip freeze 명령어를 이용해서 생성 가능
pip freeze > requirements.txt

Click==7.0
Flask==1.1.1
itsdangerous==1.1.0
Jinja2==2.11.1
MarkupSafe==1.1.1
Werkzeug==1.0.0

runtime.txt

  • 사용할 파이썬 버전 명시
python-3.6.9


작성한 파일 확인

$ ls
app.py  Procfile  requirements.txt  runtime.txt


cf push를 이용해 application 올리기

  • 프로젝트 디렉토리에서
# -m: memory size to use
# -b: buildpack option(we use python in this case)
# python-hello-world: app name, this will be subdomain name
cf push -m 128M -b python_buildpack python-hello-world
  • 결과
Pushing app python-hello-world to org cfdev-org / space cfdev-space as user...
Getting app info...
Creating app with these attributes...
+ name:         python-hello-world
  path:         /home/twpower/flask_test_cf
  buildpacks:
+   python_buildpack
+ memory:       128M
  routes:
+   python-hello-world.dev.cfdev.sh

Creating app python-hello-world...
Mapping routes...
Comparing local files to remote cache...
Packaging files to upload...
Uploading files...
 963 B / 963 B [=========================================================================] 100.00% 1s

Waiting for API to complete processing files...

Staging app and tracing logs...
   Downloading python_buildpack...
   Downloaded python_buildpack (5M)
   Cell 19330b69-fa32-43e6-9884-9e89fcbbbc40 creating container for instance 6616abe5-490e-4854-9ee6-8485ad258dd6
   Cell 19330b69-fa32-43e6-9884-9e89fcbbbc40 successfully created container for instance 6616abe5-490e-4854-9ee6-8485ad258dd6
   Downloading app package...
   Downloaded app package (963B)
   -----> Python Buildpack version 1.6.36
   -----> Supplying Python
   -----> Installing python 3.6.9
          Download [https://buildpacks.cloudfoundry.org/dependencies/python/python-3.6.9-linux-x64-cflinuxfs3-f30bc832.tgz]
   -----> Installing pip-pop 0.1.3
          Download [https://buildpacks.cloudfoundry.org/dependencies/manual-binaries/pip-pop/pip-pop-0.1.3-fc106ef6.tar.gz]
   -----> Running Pip Install
          Collecting Click==7.0 (from -r /tmp/app/requirements.txt (line 1))
            Downloading https://files.pythonhosted.org/packages/fa/37/45185cb5abbc30d7257104c434fe0b07e5a195a6847506c074527aa599ec/Click-7.0-py2.py3-none-any.whl (81kB)
          Collecting Flask==1.1.1 (from -r /tmp/app/requirements.txt (line 2))
            Downloading https://files.pythonhosted.org/packages/9b/93/628509b8d5dc749656a9641f4caf13540e2cdec85276964ff8f43bbb1d3b/Flask-1.1.1-py2.py3-none-any.whl (94kB)
          Collecting itsdangerous==1.1.0 (from -r /tmp/app/requirements.txt (line 3))
            Downloading https://files.pythonhosted.org/packages/76/ae/44b03b253d6fade317f32c24d100b3b35c2239807046a4c953c7b89fa49e/itsdangerous-1.1.0-py2.py3-none-any.whl
          Collecting Jinja2==2.11.1 (from -r /tmp/app/requirements.txt (line 4))
            Downloading https://files.pythonhosted.org/packages/27/24/4f35961e5c669e96f6559760042a55b9bcfcdb82b9bdb3c8753dbe042e35/Jinja2-2.11.1-py2.py3-none-any.whl (126kB)
          Collecting MarkupSafe==1.1.1 (from -r /tmp/app/requirements.txt (line 5))
            Downloading https://files.pythonhosted.org/packages/b2/5f/23e0023be6bb885d00ffbefad2942bc51a620328ee910f64abe5a8d18dd1/MarkupSafe-1.1.1-cp36-cp36m-manylinux1_x86_64.whl
          Collecting Werkzeug==1.0.0 (from -r /tmp/app/requirements.txt (line 6))
            Downloading https://files.pythonhosted.org/packages/ba/a5/d6f8a6e71f15364d35678a4ec8a0186f980b3bd2545f40ad51dd26a87fb1/Werkzeug-1.0.0-py2.py3-none-any.whl (298kB)
          Installing collected packages: Click, Werkzeug, itsdangerous, MarkupSafe, Jinja2, Flask
            The script flask is installed in '/tmp/contents020609285/deps/0/python/bin' which is not on PATH.
            Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
          Successfully installed Click-7.0 Flask-1.1.1 Jinja2-2.11.1 MarkupSafe-1.1.1 Werkzeug-1.0.0 itsdangerous-1.1.0
          You are using pip version 18.1, however version 20.0.2 is available.
          You should consider upgrading via the 'pip install --upgrade pip' command.
   Exit status 0
   Uploading droplet, build artifacts cache...
   Uploading droplet...
   Uploading build artifacts cache...
   Uploaded build artifacts cache (48.9M)
   Uploaded droplet (49.7M)
   Uploading complete
   Cell 19330b69-fa32-43e6-9884-9e89fcbbbc40 stopping instance 6616abe5-490e-4854-9ee6-8485ad258dd6
   Cell 19330b69-fa32-43e6-9884-9e89fcbbbc40 destroying container for instance 6616abe5-490e-4854-9ee6-8485ad258dd6

Waiting for app to start...
   Cell 19330b69-fa32-43e6-9884-9e89fcbbbc40 successfully destroyed container for instance 6616abe5-490e-4854-9ee6-8485ad258dd6

name:              python-hello-world
requested state:   started
routes:            python-hello-world.dev.cfdev.sh
last uploaded:     Mon 10 Feb 14:37:53 KST 2020
stack:             cflinuxfs3
buildpacks:        python

type:            web
instances:       1/1
memory usage:    128M
start command:   python app.py
     state     since                  cpu    memory           disk           details
#0   running   2020-02-10T05:35:17Z   0.0%   118.4K of 128M   177.7M of 1G


응답 확인

$ curl http://python-hello-world.dev.cfdev.sh
Hello World!


CF Dev 삭제

  • cf dev 중지
cf dev stop
  • cfdev plugin 삭제
cf uninstall-plugin cfdev
  • 디렉토리 삭제
rm -rf ~/.cfdev


이슈

  • 가상머신 환경에서 실행하면 virbr0가 활성화되지 않는 이슈가 있다.


참고자료