명령어 치환(command substitution)의 결과가 있는지 없는지를 조건문에서 확인할 수 있는데 이에 대해 알아보자


환경

  • Linux
  • Bash shell(/bin/bash)


상황 및 방법

상황

  • 아래처럼 쉘 스크립트에서 명령어 치환(command substitution)의 결과가 있는지 없는지를 조건문에 사용해야할 상황
  • 명령어 치환(command substitution)이란 서브쉘(subshell)을 사용해서 명령어의 결과를 가져와 대체하는걸 말한다.
#!/bin/sh

result=$(ls -al | grep sample)
...


방법

  • 아래와 같은 조건문 형식을 사용하면 된다.
  • -z 옵션을 사용해서 처리를 다르게 할 수 있다.
  • (방법 1) 조건문 안에 변수만 넣고 확인해 볼 수 있으며 크기가 0이거나 없다면 거짓이다.
  • (방법 2) -z 옵션을 사용할 수 있으며 다음에 오는 변수의 크기가 0이거나 없다면 참이다.
#!/bin/sh

result=$(ls -al | grep sample)

# Return true if its length is not zero.
if [ "${result}" ]
then
    echo "result is not empty"
else
    echo "result is empty"
fi

# Checks if the given string operand size is zero; if it is zero length, then it returns true.
if [ -z "${result}" ]
then
    echo "result is empty"
else
    echo "result is not empty"
fi


예제

  • 파일에 특정 문자열이 있는지 확인하고 없으면 추가
  • 여기서 파일은 있다고 가정
$ cat test.txt
String
$ vi test.sh
#!/bin/sh

set -e

set +e
result=$(cat test.txt | grep "test string")
set -e

if [ -z "${result}" ]
then
    echo "test string" >> test.txt
fi
$ chmod +x test.sh
$ ./test.sh
$ cat test.txt
String
test string


기타

  • grep 명령어의 경우 줄이 하나라도 생기면 0, 한 줄도 안생기면 1 그리고 오류에 대해서는 2를 종료 상태로 반환한다. 그렇기 때문에 set -e 옵션을 쓸 경우 쓰기전에 제거하고 다시 설정해줘야한다.
  • 참고: https://www.gnu.org/software/grep/manual/grep.html#Exit-Status
...
set +e
result=$(ls -al | grep sample)
set -e
...


참고자료

Check command substitution’s result exists or not in conditional statement


Environment and Prerequisite

  • Linux
  • Bash shell(/bin/bash)


Situatiaton and Method

Situatiaton

  • Below is situatiaton that conditional statement is needed to check command substitution’s result exists or not
  • command substitution is allowing the output of a command to replace the command itself by using subshell.
#!/bin/sh

result=$(ls -al | grep sample)
...


Method

  • Use below conditional statements.
  • Handle different way by using -z option.
  • (Method 1) You can check it by only adding variable in conditional statement. If its length is 0 or it is not set, then it is considered as false.
  • (Method 2) Use -z option. If variable next to -z is not set or length is 0, then it is considered as true.
#!/bin/sh

result=$(ls -al | grep sample)

# Return true if its length is not zero.
if [ "${result}" ]
then
    echo "result is not empty"
else
    echo "result is empty"
fi

# Checks if the given string operand size is zero; if it is zero length, then it returns true.
if [ -z "${result}" ]
then
    echo "result is empty"
else
    echo "result is not empty"
fi


Example

  • Add specific string if it not exists in file
  • Consider file exists
$ cat test.txt
String
$ vi test.sh
#!/bin/sh

set -e

set +e
result=$(cat test.txt | grep "test string")
set -e

if [ -z "${result}" ]
then
    echo "test string" >> test.txt
fi
$ chmod +x test.sh
$ ./test.sh
$ cat test.txt
String
test string


etc

...
set +e
result=$(ls -al | grep sample)
set -e
...


Reference

forEach에서 async await를 사용할 때 결과값이 순차적으로 오지 않는 상황이 발생했는데 이에 대한 해결법과 원인에 대해서 알아보자


환경

  • Javascript
  • async and await
  • Python
  • Flask


들어가기에 앞서

  • 간단하게 async await에 대해 정리하고 테스트할 환경을 정의한다.

async await

async

  • 비동기 함수를 정의하는 키워드
  • 하단의 await를 사용하기 위해 붙여줘야합니다.

await

  • Promise 객체를 기다리며 async 함수안에서 사용 가능합니다.
  • Promiseresolve되거나 reject될 때까지 기다립니다.
  • 쉽게 설명해 비동기 요청에 대한 응답이 올때까지 기다리는 키워드라고 보시면 됩니다.

예제

function timer(x) {
  return new Promise(resolve => {
    setTimeout(() => {
      resolve(x);
    }, 2000);
  });
}

async function test() {
  var x = await timer(10); // wait 2000 milliseconds
  console.log(x); // 10
}

test();


테스트 환경

  • 클라이언트에서 본인의 ID를 보내서 그 ID값을 그대로 반환하게 환경을 구축
  • 클라이언트는 자바스크립트로 서버는 파이썬으로 했으나 테스트 용도이기에 서버측은 몰라도 이해에 문제는 없습니다.

테스트 구조

서버 코드

from flask import Flask, request
import json
import time

app = Flask(__name__)

host_addr = "0.0.0.0"
port_num = "8080"

@app.route("/test", methods = ['POST'])
def test():
    print("Time:" + str(int(round(time.time() * 1000))) + ", reqeust from client : " + str(request.json['data']))
    return str(request.json['data'])

if __name__ == "__main__":
    app.run(host = host_addr, port = port_num)


문제

  • forEach를 사용 시 순차적으로 요청 응답이 오지 않는다.

요청 코드

arr = [1, 2, 3, 4];

arr.forEach(async (i) => {
  const response = await axios.post("http://0.0.0.0:8080/test", {
      data: i,
  });
  console.log("forEachRequestUsingAsyncAwait server response: " + response.data);
});

응답

  • 응답이 순서대로 오지 않았다.
$ node test.js
forEachRequestUsingAsyncAwait server response: 1
forEachRequestUsingAsyncAwait server response: 4
forEachRequestUsingAsyncAwait server response: 3
forEachRequestUsingAsyncAwait server response: 2

서버측 로그

  • 순서대로 요청이 오지 않았다.
Time:1602409922301, reqeust from client : 1
127.0.0.1 - - [11/Oct/2020 18:52:02] "POST /test HTTP/1.1" 200 -
Time:1602409922302, reqeust from client : 4
127.0.0.1 - - [11/Oct/2020 18:52:02] "POST /test HTTP/1.1" 200 -
Time:1602409922303, reqeust from client : 3
127.0.0.1 - - [11/Oct/2020 18:52:02] "POST /test HTTP/1.1" 200 -
Time:1602409922303, reqeust from client : 2
127.0.0.1 - - [11/Oct/2020 18:52:02] "POST /test HTTP/1.1" 200 -


이유

이유

  • forEach는 배열에 대해 Callback을 순차적으로 실행하기 때문이다. Callback을 순차적으로 실행하지만 Callback이 하나가 끝나기까지 기다렸다가 다음이 실행되는게 아니기 때문이다.
  • 반복문처럼 하나가 완료되면 다음 순서에 따라 진행되는게 아니라 각각을 콜백으로 실행하기 때문이다.

공식문서 자료

Array.prototype.forEach ( callbackfn [ , thisArg ] )
  • ECMAScript Language Specification에 보면 forEach는 “callbackfn should be a function that accepts three arguments. forEach calls callbackfn once for each element present in the array, in ascending order. callbackfn is called only for elements of the array which actually exist; it is not called for missing elements of the array.”라고 나와있다.
arr.forEach(callback(currentValue[, index[, array]]) {
  // execute something
}[, thisArg]);
  • MDN Web Docs에 보면 forEach는 “forEach() calls a provided callback function once for each element in an array in ascending order.”라고 나와있다.


해결방법

for-in 혹은 for-of 사용

for-in

  • 요청 코드
async function forInRequest () {
  for (const i in arr){
    const response = await axios.post("http://0.0.0.0:8080/test", {
        data: arr[i],
    });
    console.log("forInRequest server: " + response.data);
  }
}

forInRequest();
  • 응답 결과
node test.js
forInRequest server: 1
forInRequest server: 2
forInRequest server: 3
forInRequest server: 4
  • 서버 응답 결과
Time:1603631271374, reqeust from client : 1
127.0.0.1 - - [25/Oct/2020 22:07:51] "POST /test HTTP/1.1" 200 -
Time:1603631271383, reqeust from client : 2
127.0.0.1 - - [25/Oct/2020 22:07:51] "POST /test HTTP/1.1" 200 -
Time:1603631271386, reqeust from client : 3
127.0.0.1 - - [25/Oct/2020 22:07:51] "POST /test HTTP/1.1" 200 -
Time:1603631271388, reqeust from client : 4
127.0.0.1 - - [25/Oct/2020 22:07:51] "POST /test HTTP/1.1" 200 -

for-of

  • 요청 코드
async function forOfRequest () {
  for (const i of arr){
    const response = await axios.post("http://0.0.0.0:8080/test", {
        data: i,
    });
    console.log("forOfRequest server: " + response.data);
  }
}

forOfRequest();
  • 응답 결과
node test.js
forOfRequest server: 1
forOfRequest server: 2
forOfRequest server: 3
forOfRequest server: 4
  • 서버 응답 결과
Time:1603631202328, reqeust from client : 1
127.0.0.1 - - [25/Oct/2020 22:06:42] "POST /test HTTP/1.1" 200 -
Time:1603631202335, reqeust from client : 2
127.0.0.1 - - [25/Oct/2020 22:06:42] "POST /test HTTP/1.1" 200 -
Time:1603631202338, reqeust from client : 3
127.0.0.1 - - [25/Oct/2020 22:06:42] "POST /test HTTP/1.1" 200 -
Time:1603631202342, reqeust from client : 4
127.0.0.1 - - [25/Oct/2020 22:06:42] "POST /test HTTP/1.1" 200 -


참고자료

When I use async await in javascript, there was problem that results were not in sequential order. Find out reason and solution for it.


Environment and Prerequisite

  • Javascript
  • async and await
  • Python
  • Flask


Before In

  • Briefly explain about async await and test environment.

async await

async

  • Asynchronous function define keyword
  • Add it to use below await.

await

  • It waits Promise object and can be used in async function.
  • Wait until Promise becomes resolve or reject.
  • In simple terms, it is a keyword which waits until response of asynchronous request comes.

Examples

function timer(x) {
  return new Promise(resolve => {
    setTimeout(() => {
      resolve(x);
    }, 2000);
  });
}

async function test() {
  var x = await timer(10); // wait 2000 milliseconds
  console.log(x); // 10
}

test();


Test Environment

  • Send client’s id and server returns it
  • Server is written in python but there is no problem with understanding because we only focus on client code.

Test Structure

Server Code

from flask import Flask, request
import json
import time

app = Flask(__name__)

host_addr = "0.0.0.0"
port_num = "8080"

@app.route("/test", methods = ['POST'])
def test():
    print("Time:" + str(int(round(time.time() * 1000))) + ", reqeust from client : " + str(request.json['data']))
    return str(request.json['data'])

if __name__ == "__main__":
    app.run(host = host_addr, port = port_num)


Issue

  • Response does not come sequentially when we use forEach..

Request Code

arr = [1, 2, 3, 4];

arr.forEach(async (i) => {
  const response = await axios.post("http://0.0.0.0:8080/test", {
      data: i,
  });
  console.log("forEachRequestUsingAsyncAwait server response: " + response.data);
});

Response

  • Responses are not sequential.
$ node test.js
forEachRequestUsingAsyncAwait server response: 1
forEachRequestUsingAsyncAwait server response: 4
forEachRequestUsingAsyncAwait server response: 3
forEachRequestUsingAsyncAwait server response: 2

Server Log

  • Requests are not sequential.
Time:1602409922301, reqeust from client : 1
127.0.0.1 - - [11/Oct/2020 18:52:02] "POST /test HTTP/1.1" 200 -
Time:1602409922302, reqeust from client : 4
127.0.0.1 - - [11/Oct/2020 18:52:02] "POST /test HTTP/1.1" 200 -
Time:1602409922303, reqeust from client : 3
127.0.0.1 - - [11/Oct/2020 18:52:02] "POST /test HTTP/1.1" 200 -
Time:1602409922303, reqeust from client : 2
127.0.0.1 - - [11/Oct/2020 18:52:02] "POST /test HTTP/1.1" 200 -


Reason

Reason

  • forEach calls a provided callback function once for each element in an array in ascending order. Callback runs in sequential but it does not wait until one is finished.
  • That is because it runs not like iteration which runs next step when one step is finished. It runs each callback.

Documents

Array.prototype.forEach ( callbackfn [ , thisArg ] )
  • According to ECMAScript Language Specification, it says “callbackfn should be a function that accepts three arguments. forEach calls callbackfn once for each element present in the array, in ascending order. callbackfn is called only for elements of the array which actually exist; it is not called for missing elements of the array.”
arr.forEach(callback(currentValue[, index[, array]]) {
  // execute something
}[, thisArg]);
  • According to MDN Web Docs, it says “forEach() calls a provided callback function once for each element in an array in ascending order.”


Solutions

Use for-in or for-of

for-in

  • Request code
async function forInRequest () {
  for (const i in arr){
    const response = await axios.post("http://0.0.0.0:8080/test", {
        data: arr[i],
    });
    console.log("forInRequest server: " + response.data);
  }
}

forInRequest();
  • Response result
node test.js
forInRequest server: 1
forInRequest server: 2
forInRequest server: 3
forInRequest server: 4
  • Server response result
Time:1603631271374, reqeust from client : 1
127.0.0.1 - - [25/Oct/2020 22:07:51] "POST /test HTTP/1.1" 200 -
Time:1603631271383, reqeust from client : 2
127.0.0.1 - - [25/Oct/2020 22:07:51] "POST /test HTTP/1.1" 200 -
Time:1603631271386, reqeust from client : 3
127.0.0.1 - - [25/Oct/2020 22:07:51] "POST /test HTTP/1.1" 200 -
Time:1603631271388, reqeust from client : 4
127.0.0.1 - - [25/Oct/2020 22:07:51] "POST /test HTTP/1.1" 200 -

for-of

  • Request code
async function forOfRequest () {
  for (const i of arr){
    const response = await axios.post("http://0.0.0.0:8080/test", {
        data: i,
    });
    console.log("forOfRequest server: " + response.data);
  }
}

forOfRequest();
  • Response result
node test.js
forOfRequest server: 1
forOfRequest server: 2
forOfRequest server: 3
forOfRequest server: 4
  • Server response result
Time:1603631202328, reqeust from client : 1
127.0.0.1 - - [25/Oct/2020 22:06:42] "POST /test HTTP/1.1" 200 -
Time:1603631202335, reqeust from client : 2
127.0.0.1 - - [25/Oct/2020 22:06:42] "POST /test HTTP/1.1" 200 -
Time:1603631202338, reqeust from client : 3
127.0.0.1 - - [25/Oct/2020 22:06:42] "POST /test HTTP/1.1" 200 -
Time:1603631202342, reqeust from client : 4
127.0.0.1 - - [25/Oct/2020 22:06:42] "POST /test HTTP/1.1" 200 -


Reference

“SyntaxError: Non-ASCII character …, but no encoding declared”와 같은 이슈가 생겼을 때 해결 방법을 알아보자


환경

  • Python


문제와 해결 방법

문제 상황

  • 파이썬 코드를 실행했는데 아래와 같은 오류를 보여준다.
python /home/twpower/MailReminder/reminder.py
...
SyntaxError: Non-ASCII character '\xeb' in file /home/twpower/MailReminder/reminder.py on line 12, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details


원인

  • 실행중인 코드에 ASCII가 아닌 문자가 들어있는데 인코딩을 명시해주지 않아서 생기는 오류


해결 방법

  • 파이썬 파일 제일 상단에 # -*- coding: utf-8 -*-를 추가한다.
# -*- coding: utf-8 -*-
...


참고자료