In the article it does like this: class MyFirstPlugin(AirflowPlugin): ModuleNotFoundError: No module named 'operators.mytest_plugin' Note: The Plugins mechanism still must be used for plugins that makeĪs an aside, the errors I was getting before this (that are now resolved): ModuleNotFoundError: No module named 'mytest_hook' However it might not work for you if you want changes to the webserver UI: This worked for me and was much simpler than trying to subclass AirflowPlugin. My_operator.py: from airflow.models import BaseOperator Mytask = MyTestOperator(task_id='MyTest Task', dag=dag) Working snippets from my files are shown below.įrom operators.mytest_operator import MyTestOperator Instead from operators.mytest_operator import MyTestOperator is the way to go (note the alignment to from directory/file.py import Class in my setup above). This means that doing from _operator import MyTestOperator probably isn't going to work. When Airflow is running, it will add dags/, plugins/, and config/ to PATH All the _init_.py files are empty (unlike some equally confusing approaches of putting Plugin code in some of them).įor the imports needed, consider how Airflow actually uses the plugins directory: With this approach, all the code for my operator and hook live entirely in their respective files - and there's no confusing plugin file. └── mytest_operator.py (contains class MyTestOperator) │ └── mytest_hook.py (contains class MyTestHook) So instead of messing around with the Plugins API I followed Astronomer's approach, setting up Airflow as shown below. ![]() Airflow is evenĬonsidering deprecating using the Plugins mechanism for hooks and The issue and leads to confusion for many people. According to the Airflow documentation, they can beĪdded using Airflow’s Plugins mechanism. ![]() There is however some confusion on the best way to After struggling with the Airflow documentation and trying some of the answers here without success, I found this approach from astronomer.io.Īs they point out, building an Airflow Plugin can be confusing and perhaps not the best way to add hooks and operators going forward.Ĭustom hooks and operators are a powerful way to extend Airflow to meet your needs.
0 Comments
Leave a Reply. |
Details
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |