What's new in Apache Airflow 2.8?

Ғылым және технология

What's new in Apache Airflow 2.8?
Join me on Jan 11th, 2024 to discover Airflow 2.8 new features in PRACTICE 👇
www.astronomer.io/events/webi...
Here are the significant changes:
➡️ New XComs tab!
💻 Access data shared by tasks directly from the Grid view
➡️ Task context logging
💻 A task in failure? Access messages from Airflow components (Scheduler, Executor) to the task logs to know what happened all at a single place
➡️ New listeners to datasets
💻 Take actions on dataset creations and updates
➡️ New Airflow ObjectStore API
💻 Seamlessly transfer data between different object storage systems with a unified interface. No more transfer operators
➡️ New prev_start_date_success
💻 Take actions based on the data interval of the previous successful DAG run
➡️ New BranchPythonVirtualEnvOperator
💻 Choose between different tasks without dependency conflicts
➡️ New Decorators
💻 For external and virtual env branch operators
➡️ New clear_number attribute
💻 Track the number of times a DAG run has been cleared (restarted)
➡️ Allow override of navbar text color
💻 Use NAVBAR_TEXT_COLOR to change the navbar text color and identify environments
➡️ Description of pools!
💻 Never forget what a pool does with the description field
And more!
As usual, BIG shoutout to all contributors and the vibrant community!
We’re getting closer to 3.0 🥹
Join me on Jan 11th, 2024 to discover Airflow 2.8 new features in PRACTICE 👇
www.astronomer.io/events/webi...
🏆 BECOME A PRO: www.udemy.com/course/the-comp...
👍 Smash the like button to become an Airflow Super Hero!
❤️ Subscribe to my channel to become a master of Airflow
🚨 My Patreon: / marclamberti
Ready?
Let's go!

Пікірлер: 6

  • @fbravoc9748
    @fbravoc97483 ай бұрын

    amazing video!! What if I need to transfer data from a s3 Bucket to a RDS Postgres

  • @donnillorussia
    @donnillorussia5 ай бұрын

    Airflow is evolving so fast... Almost every major update introduces features that can tranform the very logic of your DAGs. There are no "best practices" anymore, cause there is always smth new. Same tasks can be accomplished in so many different ways...

  • @0kazaki
    @0kazaki6 ай бұрын

    Thank you

  • @p.s_kev.in96
    @p.s_kev.in966 ай бұрын

    I was struggling with writing pandas dataframe as CSV to GCS few days ago , the ObjectStoragePath is a blessing , also the connection gcs is the service account path that we use for local development using astro cli?

  • @ruhult20
    @ruhult206 ай бұрын

    First view first comment

  • @MarcLamberti

    @MarcLamberti

    6 ай бұрын

    ❤️

Келесі