Bundled Deployment Approach

Andi Lamprecht ·2026-05-11· 4 min read· Accepted

Title

Context

The current environment management strategy requires immediate change due to operational constraints and the need to improve security posture.

1. Environment Availability and Conflict: The existing staging environment has been taken over by the Blueflite team, and it is now considered their stable environment. Any changes deployed there would impact their work. Similarly, the production environment is stable, tested, and FAA approved, preventing direct deployments. The team requires a new staging environment for testing components following work completion (e.g., click to fly MC removal), which is necessary for flight team testing and FAA approval preparation.
2. Challenge in Patching Stable Versions: There is an existing issue where applying security patches to older, stable versions conflicts with the latest changes on the trunk or in the sandbox environment, so can not be tested individually. Production currently lacks crucial security updates.
3. Lack of Deployment Consistency: We require a mechanism to ensure consistency across environments by grouping a defined set of services with specific, tested versions.

Decision

We will adopt a structured bundling approach and implement a support branching model across repositories to manage service versions and deployment stability.

1. Bundling Implementation: Services will be grouped into Bundles, where the Bundle Repo defines the specific service applications and their versions that are tested together.

2. Support Branching Model: Support branches will be created and maintained in both Service Repositories and Bundle Repositories to represent stable versions deployed on stable environments.

   1. Versioning on Bundle Repo: Version updates will follow semantic versioning rules: major updates occur only on trunk branches, while minor and patch updates can occur on the newly established support branches.
   2. When major or minor tags are created for Bundle, corresponding support branches will be created automatically for Bundle repo and for Service repos.

.png)

4. Security Patch Flow: Support branches will be scanned by automated dependency management tools. This includes leveraging Renovate to suggest fixes and using the Bundle Repo flow to recollect these changes and prepare Security PRs on a regular cadence.

Consequences

What Becomes Easier

Environment Consistency: The bundle concept ensures consistency across environments by having a set of services with specific versions tested together, guaranteeing that what is tested in one environment is identical on another.

Security and Patching: Maintaining support branches allows for the independent addition and testing of security patches and other changes for stable versions without interference from the latest trunk changes.

Isolated Testing: The new approach allows the team to spin up a replica of an existing environment with just the updated component, enabling isolated testing of specific changes.

Traceability: Changes to the bundle repository only occur during releases to staging or production, providing clear definition of service versions composing a bundle for release testing.

What Becomes More Difficult (or requires new process)

Process Overhead: New workflows are required for security patches and bug fixes, including distinguishing between work tickets and release tickets that handle approvals and CCB sign-off for deployment.

Alternatives Considered

1. Alternative: Deploying Services Independently.

   1. Reason for Rejection: The current system of deploying services individually is not viable because it fails to ensure consistency across environments and leads to conflicts when trying to apply security fixes to stable, older versions.

2. Alternative: Spin up new Staging by repeating same approach

   1. Reason for Rejection: This approach will require time and effort to connect all dependencies and configuration together, and to set up additional deployment pipelines in each repository. However, it doesn’t contribute to faster environment creation in the future and leaves a challenge in testing individual changes against specific versions of our system.

Formal Impact

Services that are already coupled inside of Versioned Bundle

• apollo-frontend
• geodata
• missions
• inventory
• pathfinding

Steps to achieve New Staging:

1. Add flightlogs-processor in Bundle
2. Add environment-controller in Bundle
3. Cover terraform piece for avatar-controller in Terraform Bundle
4. Install 2nd Nginx in shared cluster for mTLS traffic
5. Deploy env-controller on dev and stg
6. Create New Staging Environment with Bundle
7. Troubleshooting

Steps to achieve Security Patches Flow:

1. Flow to create and maintain support branches for stable releases in Service and Bundle repos
2. Automated security scanning on trunk and support branches
3. “Renovate” to suggest security fixes
4. Flow on Bundle repo to regularly collect security fixes and prepare Security PR