我正在嘗試擁有一個主匕首,它將根據(jù)我的需要創(chuàng)建更多的匕首。我在氣流.cfg dags_folder中有以下python文件。此代碼在數(shù)據(jù)庫中創(chuàng)建主 dag。此主 dag 應(yīng)讀取文本文件,并應(yīng)為文本文件中的每一行創(chuàng)建 dag。但是,在主 dag 中創(chuàng)建的 dag 不會添加到數(shù)據(jù)庫中。創(chuàng)建它的正確方法是什么?版本詳細信息:蟒蛇版本:3.7阿帕奇氣流版本:1.10.8import datetime as dtfrom airflow import DAGfrom airflow.operators.bash_operator import BashOperatorfrom airflow.operators.python_operator import PythonOperatorroot_dir = "/home/user/TestSpace/airflow_check/res"print("\n\n ===> \n Dag generator")default_args = { 'owner': 'airflow', 'start_date': dt.datetime(2020, 3, 22, 00, 00, 00), 'concurrency': 1, 'retries': 0}def greet(_name): message = "Greetings {} at UTC: {} Local: {}\n".format(_name, dt.datetime.utcnow(), dt.datetime.now()) f = open("{}/greetings.txt".format(root_dir), "a+") print("\n\n =====> {}\n\n".format(message)) f.write(message) f.close()def create_dag(dag_name): with DAG(dag_name, default_args=default_args, schedule_interval='*/2 * * * *', catchup=False ) as i_dag: i_opr_greet = PythonOperator(task_id='greet', python_callable=greet, op_args=["{}_{}".format("greet", dag_name)]) i_echo_op = BashOperator(task_id='echo', bash_command='echo `date`') i_opr_greet >> i_echo_op return i_dagdef create_all_dags(): all_lines = [] f = open("{}/../dag_names.txt".format(root_dir), "r") for x in f: all_lines.append(str(x)) f.close() for line in all_lines: print("Dag creation for {}".format(line)) globals()[line] = create_dag(line)with DAG('master_dag', default_args=default_args, schedule_interval='*/1 * * * *', catchup=False ) as dag: echo_op = BashOperator(task_id='echo', bash_command='echo `date`') create_op = PythonOperator(task_id='create_dag', python_callable=create_all_dags) echo_op >> create_op
添加回答
舉報
0/150
提交
取消