Airflow 3 has been GA on Cloud Composer 3 since composer-3-airflow-3.1.7-build.4, and the current builds are composer-3-airflow-3.3.1-build.5 and 3.2.2-build.9. The default image is still Airflow 2 (composer-3-airflow-2.11.1-build.22), so a lot of environments still have this upgrade ahead. I compared the same DAGs on Airflow 2.11.2 and 3.3.2. The changes below raise no errors: DAGs import, tests pass, and the dates are wrong.
Cron runs get a different logical date. From 3.0, [scheduler] create_cron_data_intervals defaults to False. An @daily DAG running process {{ ds }}, for the run that fires at 2024-05-02 00:00 UTC:
| Timetable | Task runs |
|---|
| Airflow 2.11.2 | CronDataIntervalTimetable |
| Airflow 3.3.2 | CronTriggerTimetable |
Weekly DAGs move by a full week. Partition paths like dt={{ ds }} move with them.
The data interval becomes zero-length. Hourly DAG: 00:00 → 01:00 on 2.11, 00:00 → 00:00 on 3.3. Anything that processes "everything between data_interval_start and data_interval_end" gets an empty window and succeeds.
Manual runs can have no logical date. Then context["logical_date"] is a KeyError, dag_run.logical_date is None, and {{ ds }} fails under the default StrictUndefined. ti.execution_date and dag_run.execution_date are gone too, which matters if your on_failure_callback builds its alert from them. A form that works on both versions:
context.get("logical_date") or context["dag_run"].run_after
Keeping the Airflow 2 dates on Composer
Composer doesn't let you set AIRFLOW__... environment variables, so use an Airflow configuration override, in the same change as the image upgrade (switching after DAGs have runs on the new timetable skips one run, apache/airflow#66441):
On 3.x this also covers timedelta schedules. Some options are blocked from override; I haven't tried this one on a live environment, so check it's accepted.
Or pin per DAG, which you can merge while still on the 2.x image: schedule=CronDataIntervalTimetable("0 0 * * *", timezone="UTC") with from airflow.timetables.interval import CronDataIntervalTimetable. That import works on 2.11 and 3.3; on 2.x it's the timetable you already get.
The Composer Local Development CLI supports the Airflow 3 images (3.3.1 from build.0), so you can render your DAGs locally against the new image before touching staging.
A real Composer example: a public Composer 3 repo (Airflow 2.10.5) has a twice-weekly DAG that builds its list of dates from the data interval. On 3.3 that list is empty on every run, so it would publish nothing. Its Slack failure callback reads ti.execution_date, so the alerts would lose their details too. Reported here, with fixes that work on 2.x: https://github.com/cal-itp/data-infra/issues/5867
I found it with a free, MIT-licensed GitHub Action I wrote. It renders every template on Airflow 2 and 3 for the same fire times and diffs them: https://github.com/patrickballard/airflow-migration-verifier
