Could you please make video on best practices for kubernetesexecuter on K8s and how we can run parallel pods?
@ricardodelzealandia62902 жыл бұрын
Thanks for this. My question - the official apache airflow helm chart and the docker compose file splits airflow into multiple pieces using the same container image. Which instance of this image cares about the dag and it's related files, given every instance has the dags folder?
@dani2500d7 ай бұрын
Hey, Awesome webinar! Thank you! I do have one question about the best practices of structuring a DAG. So, It is better to put the task implementations (python operators) into a separate file. If my tasks require a lot of imports, is it better to import inside every task (method) or is it fine to import it all on the top level of the tasks file?
@Astronomer
7 ай бұрын
I honestly usually just add them all to the top of DAG file as Taskflow tasks, instead of pulling in from files, but if you're using the same python functions in multiple DAG's, the import method is probably best for you!
@mosa36 Жыл бұрын
Thank you for this video! I was wondering how you would see all the variables available or add new variables?
@Astronomer
Жыл бұрын
Check out the Airflow registry, it shows all the variables for each operator! registry.astronomer.io/
@razer9234 Жыл бұрын
Where can we get the presentation slides?
@rafalkuc72172 жыл бұрын
In 18:24 You are presenting DAGs Dashoard. Please noticed DAG called airflow2_good_example has schedule_imterval = '@daily', but Last run and Next run are showing the same date (2021-11-29, 00:00:00). Do You know when we can expect improvements? Its so confusing for end users.
@yevgenym9204Ай бұрын
@Astronomer Please share a direct link to the CLI library you mention (for proper files strcuture) kzread.info/dash/bejne/rIqupLipksrdYaQ.htmlsi=HiJa9Afi-53yLZOG&t=873
@Astronomer
Ай бұрын
You can find documentation on the Astro CLI, including download instructions, here: docs.astronomer.io/astro/cli/overview
@Cal97g7 ай бұрын
Why can't americans say words? eye-dempotency?
@Slayeh
7 ай бұрын
How do you say it? I and everyone else I work with (in London) say it like that
Пікірлер: 12
Could you please make video on best practices for kubernetesexecuter on K8s and how we can run parallel pods?
Thanks for this. My question - the official apache airflow helm chart and the docker compose file splits airflow into multiple pieces using the same container image. Which instance of this image cares about the dag and it's related files, given every instance has the dags folder?
Hey, Awesome webinar! Thank you! I do have one question about the best practices of structuring a DAG. So, It is better to put the task implementations (python operators) into a separate file. If my tasks require a lot of imports, is it better to import inside every task (method) or is it fine to import it all on the top level of the tasks file?
@Astronomer
7 ай бұрын
I honestly usually just add them all to the top of DAG file as Taskflow tasks, instead of pulling in from files, but if you're using the same python functions in multiple DAG's, the import method is probably best for you!
Thank you for this video! I was wondering how you would see all the variables available or add new variables?
@Astronomer
Жыл бұрын
Check out the Airflow registry, it shows all the variables for each operator! registry.astronomer.io/
Where can we get the presentation slides?
In 18:24 You are presenting DAGs Dashoard. Please noticed DAG called airflow2_good_example has schedule_imterval = '@daily', but Last run and Next run are showing the same date (2021-11-29, 00:00:00). Do You know when we can expect improvements? Its so confusing for end users.
@Astronomer Please share a direct link to the CLI library you mention (for proper files strcuture) kzread.info/dash/bejne/rIqupLipksrdYaQ.htmlsi=HiJa9Afi-53yLZOG&t=873
@Astronomer
Ай бұрын
You can find documentation on the Astro CLI, including download instructions, here: docs.astronomer.io/astro/cli/overview
Why can't americans say words? eye-dempotency?
@Slayeh
7 ай бұрын
How do you say it? I and everyone else I work with (in London) say it like that