qt6-migration-pipeline/external-dependencies/README.md

# External Dependencies CI/CD Pipelines

This directory contains independently versioned, reproducible CI/CD pipelines for each external dependency used in the Legatus Qt6 migration project.

## Overview

Following Ubuntu 26.04 standards, each dependency has:
- **Dockerfile**: Reproducible build environment
- **Jenkinsfile**: CI/CD pipeline with SBOM generation
- **README.md**: Usage documentation
- **SBOM artifacts**: CycloneDX format, SLSA v1.0 provenance, license compliance

## Directory Structure

```
external-dependencies/
├── DEPENDENCIES-INVENTORY.md          # Complete dependency inventory
├── scripts/
│   └── generate-sbom.sh               # Reusable SBOM generation script
├── ubuntu-26.04-build-container/      # Main build container (all dependencies)
├── qt6-framework/                     # Qt6 6.10.2 pipeline
├── kdreports/                         # KDReports 2.3.0 pipeline
├── log4qt/                            # Log4Qt 2.0.0 pipeline
├── grantlee/                          # Grantlee 5.3.1 pipeline
├── googletest/                        # GoogleTest 1.15.2 pipeline
└── postgresql/                        # PostgreSQL 16 pipeline
```

## Dependency Graph

```
ubuntu-26.04-build-container (base)
├── qt6-framework (Qt6 6.10.2)
│   └── kdreports (KDReports 2.3.0)
├── log4qt (Log4Qt 2.0.0)
├── grantlee (Grantlee 5.3.1 with Qt6)
├── googletest (GoogleTest 1.15.2)
└── postgresql (PostgreSQL 16)
```

## Usage

### Building Individual Packages

Each package can be built independently:

```bash
cd external-dependencies/log4qt
docker build -t legatus-log4qt:2.0.0 .
```

### Running Jenkins Pipelines

1. Import the Jenkinsfile into your Jenkins instance
2. Configure environment variables:
   - `GIT_URL`: Repository URL
   - `GIT_COMMIT`: Commit SHA
   - `BUILD_NUMBER`: Jenkins build number
   - `HARBOR_REGISTRY`: Container registry URL
   - `HARBOR_CREDENTIALS_ID`: Jenkins credentials ID

3. Trigger the build

### SBOM Generation

SBOMs are automatically generated by the Jenkins pipeline. To generate manually:

```bash
cd external-dependencies/scripts
./generate-sbom.sh <package-name> <version> <output-dir>
```

Example:
```bash
./generate-sbom.sh log4qt 2.0.0 ./sbom-output
```

This generates:
- `<package>-sbom.json`: CycloneDX SBOM
- `<package>-provenance.json`: SLSA v1.0 provenance
- `<package>-licenses.txt`: License compliance report

## Air-Gapped Environment Support

For air-gapped deployment, offline dependency packages are prepared and transferred via secure media:

### Offline Cache Structure

```
external-dependencies/offline-cache/
├── qt6/6.10.2/gcc_64/          # Qt 6.10.2 installation (2.2GB)
├── src/                        # Source tarballs
│   ├── KDReports-2.3.0.tar.gz
│   ├── Log4Qt-2.0.0.tar.gz
│   └── grantlee-*.tar.gz
├── deb/                        # Ubuntu .deb packages (49 packages, 51MB)
└── scripts/                    # Installation scripts
    ├── install-offline.sh      # Offline installation script
    └── README.md               # Package inventory
```

### Download Scripts

**download-all-dependencies.sh**: Downloads all dependencies for offline use
```bash
cd external-dependencies/scripts
./download-all-dependencies.sh [output-directory]
```

**download-deb-packages.sh**: Downloads Ubuntu .deb packages
```bash
./download-deb-packages.sh [output-directory]
```

**verify-offline-cache.sh**: Verifies offline cache completeness
```bash
./verify-offline-cache.sh
```

### Transfer to Air-Gapped Environment

1. Run download scripts on internet-connected machine
2. Copy `offline-cache/` directory to secure media (encrypted USB drive)
3. Transfer to air-gapped environment
4. Mount or copy to Jenkins build server
5. Use offline installation scripts in Dockerfiles

### Important Notes

- **Offline cache is excluded from git** (2.3GB, security best practices)
- Transfer via secure media only
- Verify checksums after transfer
- Keep offline cache updated with dependency version changes
- See `offline-cache/README.md` for detailed usage

## Version Pinning Strategy

All dependencies use **exact version pinning**:
- No floating versions (e.g., `>=1.0.0`)
- No `latest` tags
- Semantic versioning with major.minor.patch
- Git commit SHA for source-based builds

Example:
```dockerfile
ENV QT_VERSION=6.10.2
ENV KDREPORTS_VERSION=2.3.0
ENV LOG4QT_VERSION=2.0.0
```

## Security Considerations

### Proprietary Dependencies

RTI Connext DDS and ArcGIS Runtime SDK are **NOT** included in these pipelines:
- They are proprietary and require commercial licenses
- They must be mounted at runtime from the host
- They are excluded from Docker images and SBOMs

See `DEPENDENCIES-INVENTORY.md` for license requirements.

### License Files

**DO NOT COMMIT LICENSE FILES TO GIT**

The following files are in `.gitignore`:
- `rti_license.dat`: RTI Connext DDS license
- `ArcGIS_Maps_SDK_Qt_linux_300.0.0.zip`: ArcGIS installer

These must be provided at runtime via secure mounting or environment variables.

## Harbor Registry

All built images are pushed to the internal Harbor registry:

```
harbor.sl.defence-elec.de/legatus/<package-name>:<version>
```

Example:
```
harbor.sl.defence-elec.de/legatus/log4qt:2.0.0
```

## Ubuntu 26.04 Base

All containers use Ubuntu 26.04 (Noble Numbat) as the base OS for:
- Consistency across all packages
- Security updates and LTS support
- Compatibility with Qt6 requirements

## Build Container

The main build container (`ubuntu-26.04-build-container`) includes all dependencies and is used for:
- Full Legatus builds
- Integration testing
- Release builds

Usage:
```bash
cd external-dependencies/ubuntu-26.04-build-container
docker build -t legatus-build:ubuntu-26.04 .
```

## SBOM Requirements

All SBOMs follow these standards:

### CycloneDX 1.6
- Component metadata
- License information
- Dependency graph
- PURL identifiers

### SLSA v1.0 Provenance
- Build definition
- External parameters (source, commit, dockerfile)
- Run details (builder ID, invocation, timestamp)

### License Compliance
- Package license identification
- Dependency licenses
- Compliance status
- Review requirements

## Maintenance

### Updating Dependencies

1. Update version in Dockerfile
2. Update version in Jenkinsfile
3. Update version in README.md
4. Run pipeline to verify build
5. Update DEPENDENCIES-INVENTORY.md

### Adding New Dependencies

1. Create directory: `external-dependencies/<package-name>/`
2. Copy template from existing package
3. Update Dockerfile with installation steps
4. Update Jenkinsfile with package metadata
5. Add to DEPENDENCIES-INVENTORY.md
6. Add to dependency graph in README.md

## Troubleshooting

### Build Failures

1. Check Docker build logs
2. Verify network access to package repositories
3. Check for version conflicts
4. Review Jenkins pipeline console output

### SBOM Generation Issues

1. Ensure `generate-sbom.sh` is executable
2. Check environment variables (GIT_URL, GIT_COMMIT, BUILD_NUMBER)
3. Verify output directory permissions

## References

- [DEPENDENCIES-INVENTORY.md](./DEPENDENCIES-INVENTORY.md): Complete dependency list
- [Qt6 Migration Verification Report](../QT6_MIGRATION_VERIFICATION_REPORT.md)
- [Docker Build Script](../qt-migration-legatus/qt6-after/qt6-migrated-legatus/new_docker_build.sh)