Files
qt6-migration-pipeline/prompts/split-external-dependencies-ci-cd.prompt.md
T
2026-11-04 15:43:50 +00:00

158 lines
12 KiB
Markdown

# Reproducible External Dependency and Container Pipelines
This is an implementation task. Work autonomously, keep changes focused, and do not modify unrelated files. The goal is to inventory every external resource used by the Docker build script and create independently versioned, reproducible CI/CD pipelines for those resources and the build container.
## Fixed scope
Work only on branch:
`feature/regression-arcgis300-symbology-migration`
If this branch does not exist in a checkout, create it from that repository's current default branch before editing. Do not silently substitute another branch.
Use these repository files as the source of truth:
- Build/install behavior: `./qt-migration-legatus/qt6-after/qt6-migrated-legatus/new_docker_build.sh`
- Prompt organization and section style: `./prompts/verify-legatus-qt6-migration.prompt.md`
- Container Dockerfile baseline: the Qt6/Ubuntu 26.04 Dockerfile supplied in the task context
The supplied Dockerfile `./prompts/Dockerfile-ubuntu26-qt6_anpassung_tp.dockerfile` is a reference attachment, not an executable host-side build script. Do not attempt to execute it directly on the Copilot host.
## Required investigation
Before changing files:
1. Read the complete `new_docker_build.sh` and trace every resource it installs or downloads, including resources generated through variables, helper functions, Dockerfile heredocs, `apt`, `pip`/`aqtinstall`, Git clones, and runtime downloads.
2. Verify actual versions, tags/branches/commits, URLs, checksums/signatures, build flags, licenses, and transitive dependencies from repository evidence and upstream metadata. Do not infer a pinned version from a shallow clone or a moving branch such as `master`.
3. Inventory all resources in a table and classify each as:
- OS/base image or Dockerfile-provided resource
- Ubuntu/Nexus APT binary package
- upstream binary/artifact (including Qt installed with `aqtinstall`)
- built from source
- proprietary/runtime-mounted or otherwise unavailable
4. Record the current installation method, version/pin status, license/distribution constraints, and proposed artifact/package and pipeline for each resource. Include Log4Qt, Grantlee, KDReports, GoogleTest/GoogleMock, Qt and every other resource found; do not limit the inventory to these examples.
5. Inspect existing repository folder conventions, CI conventions, Dockerfiles, package repository configuration, and available signing/publishing mechanisms before choosing paths or inventing configuration.
If the attachment or an upstream resource cannot be examined, state the exact limitation and do not claim a complete inventory.
## Dependency folders and packaging
Keep downloaded, non-source-built dependencies in separate, clearly named folders by origin and package, following existing repository conventions where possible. If no suitable convention exists, use a layout equivalent to:
- `external-dependencies/downloads/apt/<package>/`
- `external-dependencies/downloads/upstream/<package>/`
- `external-dependencies/source/<package>/` only for source-build inputs
Do not put source-build inputs in the binary-download folders. Store versioned source archives or immutable Git references, checksums, license notices, and provenance manifests with the corresponding package.
Create one independently versioned pipeline per external package/resource, with its own uniquely named Dockerfile and Jenkinsfile. Each pipeline must:
- produce a version-pinned `.deb` (or, where technically or legally impossible, a documented equivalent artifact without pretending it is a `.deb`);
- publish to the organization's `external_rpms` package repository using existing repository conventions;
- make package versions, source revisions, build arguments, checksums, and resulting artifact names explicit;
- support rebuilding one dependency without rebuilding unrelated dependencies;
- emit an SBOM and provenance/build manifest, and preserve the source/license notices;
- pin all transitive build inputs where practical and fail on checksum, signature, or version mismatches;
- avoid embedding credentials in files, logs, images, or artifacts; use configured Jenkins credentials and existing approved repository endpoints.
Qt is the explicit exception to compiling upstream code: install the specified Qt version/modules as upstream Qt binaries using `aqtinstall` from Qt online sources. Do not build Qt from source. Make the resulting Qt binaries and their provenance available for approved air-gapped use, using the repository's permitted packaging/artifact mechanism. Check Qt and module redistribution terms first; if redistribution into `.deb` or the internal repository is not permitted or cannot be verified, stop that packaging step and report the exact constraint and an approved alternative needed. Do not bypass licensing.
For Ubuntu APT packages that are not built from source, preserve the original pinned `.deb` and metadata in the separate package download folder; do not recompile or silently repackage upstream binaries. Use the existing approved Ubuntu/Nexus source configuration and publish/mirror artifacts according to the existing `external_rpms` repository format. If the requested per-package `.deb`/pipeline model conflicts with the package's origin, license, or repository format, document the conflict and do not fabricate an artifact.
## Build container and Jenkins pipelines
Create a separate build-container Dockerfile and Jenkinsfile, distinct from the per-dependency pipelines. Follow the supplied Dockerfile as the baseline while applying these requirements:
- Use Ubuntu `26.04` only. Prefer the approved internal Harbor mirror for the base image; do not silently fall back to another Ubuntu release.
- Preserve the Qt version/modules and project-specific environment needed by `new_docker_build.sh`, but consume the produced internal packages/artifacts instead of rebuilding or downloading unpinned dependencies in the container.
- Configure the Ubuntu/Nexus APT sources using the repository's approved pattern. Do not use an untrusted key or disable signature verification.
- Set `ARG no_proxy` so it contains the exact required entry `__.__sl.defence-elec.de`; do not replace it with a broader or differently spelled domain. Preserve required local/internal exclusions from the supplied baseline as appropriate.
- Use the supplied proxy values only as build-time configuration; do not bake credentials or secrets into the image.
- Keep proprietary RTI and ArcGIS SDKs mounted at runtime when required; do not commit, package, or copy proprietary SDK/license files into the image.
- Set Log4Qt telnet logging to `OFF` explicitly in its build configuration.
- Pin Grantlee and every other source-built dependency to a verified immutable version/tag/commit; never use an unpinned `master` branch.
- Configure DNS to `10.199.101.35` and `10.199.101.158` only when DNS resolution fails in the container, using the supported Docker/Jenkins configuration rather than hard-coding a host-wide DNS change.
- Create the container Jenkins pipeline following this structure and adapt the repository URL, branch, and paths to the actual checkout. Use the requested feature branch, not `main`, unless repository policy requires a documented equivalent:
```groovy
node {
def app
stage('Clone Repo') {
git branch: 'feature/regression-arcgis300-symbology-migration', url: '<repository-url>'
}
stage('Build Container') {
app = docker.build(
'buildcontainer/ubuntu_26_04_qt6',
'-f Dockerfile-ubuntu26-qt6.dockerfile .'
)
}
stage('Push Container') {
docker.withRegistry('https://harbor.sl.defence-elec.de', 'harbor-robot') {
app.push("${env.BUILD_ID}")
app.push('latest')
}
}
}
```
Use existing credential IDs and registry policy; do not add credentials to source. Do not push an image unless the user/repository's normal CI authorization permits it.
## `new_docker_build.sh` integration
The current script must be inspected for the actual state of its options; do not assume an option exists merely because it is requested here. Add `--docker-mount-root PATH` if it is absent, with `/root` as the default. The option must:
- be parsed consistently with the script's existing options and documented in `--help`;
- accept and validate a container mount destination, defaulting to `/root`;
- be passed to the container invocation and used for the intended dependency/artifact workspace mount without changing unrelated mounts;
- preserve current script behavior when omitted;
- fail explicitly on invalid input or a missing required host directory instead of silently falling back.
Use the option with its default `/root` in the documented build/pipeline invocation. Add or update focused tests for argument parsing and mount construction if the repository has a test mechanism for this script.
## Reproducibility, supply chain, and safety
- Pin the Ubuntu base-image digest where the approved registry supports it, and record the upstream image and digest.
- Record exact source URL, immutable version/revision, retrieval timestamp, checksum/signature verification, license, build toolchain, build flags, and output checksum for each artifact.
- Generate an SBOM for each package and the final container; use existing tools/formats if configured, otherwise choose a standard format and document the choice.
- Ensure the build can use the internal artifact/package repository without depending on public internet access after approved artifacts have been mirrored.
- Do not silently continue after failed downloads, package installation, signature checks, builds, SBOM generation, or publishing.
- Keep repository credentials and signing keys in Jenkins secrets; never print them or commit them.
- Do not execute builds on the Copilot host. Run validation only in the supported Docker/Jenkins environment. If that environment is unavailable, perform static checks and report the exact unexecuted validations.
## Verification
1. Verify the inventory covers every external install/download path in the complete script, including both prebuilt-image and runtime-install paths.
2. Verify each package has an isolated version/pipeline definition, unique Dockerfile/Jenkinsfile names, pinned inputs, and a documented output.
3. Verify APT downloads and upstream binary artifacts are in separate per-package folders and source-build inputs are kept separate.
4. Verify the container build consumes the produced packages, uses Ubuntu 26.04, includes the required `no_proxy` token, and disables Log4Qt telnet logging.
5. Verify the script's `--docker-mount-root` default, help text, argument validation, and resulting Docker mount.
6. Run available static validation, shell checks, Dockerfile/Jenkinsfile validation, package/SBOM checks, and pipeline tests in the supported environment. Do not report a pipeline as tested if it was only inspected.
7. Verify no secrets, proprietary SDKs, licenses, or unapproved binaries were added to the repository.
## Success criteria
The task is complete only when:
- every external resource used by the script is inventoried and classified;
- each package/resource has a separate, reproducible, versioned CI/CD pipeline and its own Dockerfile/Jenkinsfile, or a specific documented license/technical blocker;
- deliverable packages/artifacts and SBOM/provenance are suitable for controlled internal and approved air-gapped distribution;
- the build-container pipeline is independent and follows the supplied Ubuntu 26.04 Qt6 template;
- `new_docker_build.sh` supports `--docker-mount-root` with default `/root` and the option is verified;
- all checks actually run are reported accurately, with unavailable checks and blockers stated plainly.
## Final report
Provide a factual report containing:
- status: `COMPLETE`, `PARTIALLY COMPLETE`, or `BLOCKED`;
- branch and changed files;
- the dependency inventory, including version/pin and source/build method;
- each package's pipeline, Dockerfile/Jenkinsfile, output artifact, repository destination, SBOM/provenance status, and verification result;
- build-container image name, Ubuntu base/digest, Jenkins pipeline result, and push status;
- `--docker-mount-root` behavior and test result;
- Qt license/distribution handling and any unresolved approval;
- commands/checks run, results, and exact skipped checks or blockers;
- remaining supply-chain, reproducibility, or air-gap risks.
Do not claim full reproducibility, air-gapped readiness, successful publishing, or complete validation without concrete evidence.