Scheduling in Airflow

Ойын-сауық

The flexibility and freedom that Airflow offers you is incredible, but to really take advantage of it you need to master some concepts first, one of which has just been released in Airflow 2.2 By the end of the webinar, you will be able to define schedule intervals that you thought were impossible before.

Пікірлер: 6

  • @maximilianrausch5193
    @maximilianrausch51932 жыл бұрын

    Is it possible to use time tables to schedule tasks at the start of the interval now instead of the end ?

  • @1UniverseGames
    @1UniverseGames2 жыл бұрын

    I have a question, can you show me like I have a DAG trained model which done in deep learning technique. Now how can I integrate my model into Airflow for dag job scheduling. Can you show a demo using such model.

  • @ayushikhanna1094
    @ayushikhanna10943 ай бұрын

    Is there any option available in airflow ui to auto trigger.

  • @cloudlover9186
    @cloudlover91865 ай бұрын

    I am running in to below problem , will it be acheived by time table concepts . i have same dag which should satisfy the below schedule intervals schedule interval = '30 1,4,7,10,13,16,19,22 * * *' & '00 3,6,12,15,18,21,00 * * *' , Please help and guide.

  • @Astronomer

    @Astronomer

    4 ай бұрын

    Would you mind sharing info on what type of scheduling interval you'd like to achieve? Not sure what it is based on that string unfortunately!

  • @cloudlover9186

    @cloudlover9186

    4 ай бұрын

    @@Astronomer Hi , we are in process of changing a daily schedule to 90 mins frequency dag and expectation of dag to run at 00:00, 01:30 , 03:00 so on and also another new dag which is of same 90 mins frequency should run at 00:20, 01:50, 03:10 etc.., point is if i have hard coded start date as future date for example today is 01/10 i will hard code as 01/11 (2024/01/11,00,00) any future change is not impacting the start date schedule , having said we have advised to research more not to hard code start date .FYI we are using timedelta(minutes=90) in schedule interval attribute. if we use current date logic , during deployment time ,(deployment time > start date time) it is executing immediatley , how we can over come this , please help.

Келесі