[Airflow] Bash Operator 의 마지막 echo 를 xcom 으로 넘기기

 

 

Bash Operator 에서 실행한 Shell Script 에서 가장 마지막에 echo(stdout) 로 출력된 것을

Airflow 의 xcom 으로 넘길 수 있음

 

DAG 아래처럼 작성

import airflow from airflow import DAG from airflow.utils import timezone from airflow.operators.bash import BashOperator from datetime import datetime, timedelta import pendulum local_tz = pendulum.timezone('Asia/Seoul') default_args = {     'start_date': datetime(2022, 6, 25, 00, tzinfo=local_tz),     'schedule_interval': '00 00 * * *',     'owner': 'eyeballs', } with DAG(dag_id='passing_echo',         default_args=default_args,         catchup=False) as dag:     BASE_TIME = "{{data_interval_end.in_timezone('Asia/Seoul').strftime('%Y%m%d')}}"     making_echo = BashOperator(             task_id = "making_echo",             bash_command=f'echo today is {BASE_TIME}',             do_xcom_push=True,         )     catching_echo = BashOperator(             task_id = "catching_echo",             bash_command="echo {{ti.xcom_pull(task_ids='making_echo')}} and its beautiful",         )     making_echo >> catching_echo

 

로그는 아래와 같이 나옴

 

< making_echo 로그 > [2022-09-28, 07:31:18 UTC] {subprocess.py:74} INFO - Running command: ['bash', '-c', 'echo today is 20220928'] [2022-09-28, 07:31:18 UTC] {subprocess.py:85} INFO - Output: [2022-09-28, 07:31:18 UTC] {subprocess.py:89} INFO - today is 20220928 [2022-09-28, 07:31:18 UTC] {subprocess.py:93} INFO - Command exited with return code 0

 

< catching_echo 로그 > [2022-09-28, 07:31:20 UTC] {subprocess.py:74} INFO - Running command: ['bash', '-c', 'echo today is 20220928 is today'] [2022-09-28, 07:31:20 UTC] {subprocess.py:85} INFO - Output: [2022-09-28, 07:31:20 UTC] {subprocess.py:89} INFO - today is 20220928 and its beautiful [2022-09-28, 07:31:20 UTC] {subprocess.py:93} INFO - Command exited with return code 0