ATLAS Releasestrategy 4 Flushstrategy

Andi Lamprecht ·2026-05-11· 1 min read· Accepted

Release Strategy - Flush Strategy

Context

Flush run is a distinctive approach to pipeline execution, offering an alternative mode that can be triggered when needed. In this mode, the pipeline operates by processing the entire dataset rather than focusing solely on changes, which results in the complete replacement of the output data rather than merging it with existing data.

This method is particularly useful in situations where a fresh start or a clean slate is required, ensuring that the output is entirely updated and free from any remnants of previous runs.

CI/CD Release flow should ensure triggering that for the pipelines that are affected by software changes.

Following Scenarios may appear for the changes related to github repository:

1. Change in /dags directory - affected pipeline should be triggered with flush = true
2. change in /dags/spark_jobs/common directory :

• new file/s addes - no flush required
• any file modified or deleted - flush run required for all pipelines

1. change in /dags/spark_jobs/dags directory - flush run required for affected pipeline