qt6-migration-pipeline/external-dependencies/README.md

# External Dependencies CI/CD Pipelines

This directory contains independently versioned, reproducible CI/CD pipelines for each external dependency used in the Legatus Qt6 migration project.

## Overview

Following Ubuntu 26.04 standards, each dependency has:
- **Dockerfile**: Reproducible build environment
- **Jenkinsfile**: CI/CD pipeline with SBOM generation
- **README.md**: Usage documentation
- **SBOM artifacts**: CycloneDX format, SLSA v1.0 provenance, license compliance

## Directory Structure

```
external-dependencies/
├── DEPENDENCIES-INVENTORY.md          # Complete dependency inventory
├── scripts/
│   └── generate-sbom.sh               # Reusable SBOM generation script
├── ubuntu-26.04-build-container/      # Main build container (all dependencies)
├── qt6-framework/                     # Qt6 6.10.2 pipeline
├── kdreports/                         # KDReports 2.3.0 pipeline
├── log4qt/                            # Log4Qt 2.0.0 pipeline
├── grantlee/                          # Grantlee 5.3.1 pipeline
├── googletest/                        # GoogleTest 1.15.2 pipeline
└── postgresql/                        # PostgreSQL 16 pipeline
```

## Dependency Graph

```
ubuntu-26.04-build-container (base)
├── qt6-framework (Qt6 6.10.2)
│   └── kdreports (KDReports 2.3.0)
├── log4qt (Log4Qt 2.0.0)
├── grantlee (Grantlee 5.3.1 with Qt6)
├── googletest (GoogleTest 1.15.2)
└── postgresql (PostgreSQL 16)
```

## Usage

### Building Individual Packages

Each package can be built independently:

```bash
cd external-dependencies/log4qt
docker build -t legatus-log4qt:2.0.0 .
```

### Running Jenkins Pipelines

1. Import the Jenkinsfile into your Jenkins instance
2. Configure environment variables:
   - `GIT_URL`: Repository URL
   - `GIT_COMMIT`: Commit SHA
   - `BUILD_NUMBER`: Jenkins build number
   - `HARBOR_REGISTRY`: Container registry URL
   - `HARBOR_CREDENTIALS_ID`: Jenkins credentials ID

3. Trigger the build

### SBOM Generation

SBOMs are automatically generated by the Jenkins pipeline. To generate manually:

```bash
cd external-dependencies/scripts
./generate-sbom.sh <package-name> <version> <output-dir>
```

Example:
```bash
./generate-sbom.sh log4qt 2.0.0 ./sbom-output
```

This generates:
- `<package>-sbom.json`: CycloneDX SBOM
- `<package>-provenance.json`: SLSA v1.0 provenance
- `<package>-licenses.txt`: License compliance report

## External Dependencies (Proprietary/Custom)

The following 8 external dependencies use a **DEB package-based deployment model** from the internal SCM repository:

### Dependencies List

| Dependency | Version | DEB Package | License Type |
|------------|---------|-------------|--------------|
| AVS | Custom | `libavs-dev` | Contract-based (lifetime) |
| ArcGIS Runtime SDK | 300.0.0 | `libarcgis-maps-sdk-qt-dev` | Embedded ini-file licenses |
| BuildingModels | Custom | `libbuildingmodels-dev` | Internal |
| GeographicLib | Custom | `libgeographiclib-dev` | MIT |
| Schusstafel | Custom | `libschusstafel-dev` | Internal |
| dtEntity | Custom | `libdtentity-dev` | Open Source |
| RTI Connext DDS | 7.7.0 | `librti-connext-dds-7.7.0-dev` | Commercial (license file) |
| tmsif | Custom | `libtmsif-dev` | Contract-based (lifetime) |

### DEB Package Repository

All DEB packages are stored in the internal SCM repository:
- **URL:** `https://internal-repo.sl.defence-elec.de/#browse/browse:external-rpms:general`
- **APT Repository:** `http://remote-repo.sl.defence-elec.de:8081/repository/ubuntu`
- **Signing Key:** `http://10.198.80.75/rme/deb_and_rpm_signage_key.pub`

### Dockerfile Pattern

All 8 Dockerfiles follow this pattern:

```dockerfile
# Install <dependency> from internal SCM repository
# DEB package: <deb-package-name>
# Repository: https://internal-repo.sl.defence-elec.de/#browse/browse:external-rpms:general

# License handling: <license-type>
# <license-details>

RUN apt-get update && apt-get install -y \
    <deb-package-name> \
    && rm -rf /var/lib/apt/lists/*
```

### License Handling

**RTI Connext DDS (7.7.0):**
- License file: `rti_license.dat` (provided at build time)
- Environment variable: `RTI_LICENSE_FILE=/opt/rti_connext_dds-7.7.0/rti_license.dat`
- Commercial license with FlexLM

**ArcGIS Runtime SDK (300.0.0):**
- Licenses embedded in 19 .ini files
- Location: `Lts/Viewer/SituationMap/Map/*.ini`
- Map-specific one-line license strings

**AVS & tmsif:**
- Contract-based licensing (lifetime licenses)
- No license files required
- Customer has physical license contracts

**BuildingModels, GeographicLib, Schusstafel, dtEntity:**
- Internal or open source
- No license files required

### Jenkinsfile Pattern

All 8 Jenkinsfiles include:

```groovy
stage('Prepare License Files') {
    steps {
        // License handling: <license-type>
        // <license-details>
        
        // DEB package installed from internal SCM repository
        // Repository: https://internal-repo.sl.defence-elec.de/#browse/browse:external-rpms:general
        // Package: <deb-package-name>
        sh '''
            apt-get update
            apt-get install -y <deb-package-name>
        '''
    }
}
```

### DEB Package Build Pipelines

Each dependency has a DEB package build directory:

```
external-dependencies/deb-packages/
├── avs/                  # AVS DEB package build
├── arcgis/               # ArcGIS DEB package build
├── buildingmodels/       # BuildingModels DEB package build
├── geographiclib/        # GeographicLib DEB package build
├── schusstafel/          # Schusstafel DEB package build
├── dtentity/             # dtEntity DEB package build
├── rti-connext-dds/      # RTI Connext DDS DEB package build
└── tmsif/                # tmsif DEB package build
```

Each directory contains:
- `debian/` directory with control files
- Build scripts for package creation
- README with packaging instructions

### Installation in Air-Gapped Environment

1. DEB packages are built and uploaded to internal SCM repository
2. CI/CD pipelines configure APT to use internal repository
3. Dockerfiles install packages via `apt-get install`
4. SBOM generation tracks all installed packages

## Air-Gapped Environment Support

For air-gapped deployment, offline dependency packages are prepared and transferred via secure media:

### Offline Cache Structure

```
external-dependencies/offline-cache/
├── qt6/6.10.2/gcc_64/          # Qt 6.10.2 installation (2.2GB)
├── src/                        # Source tarballs
│   ├── KDReports-2.3.0.tar.gz
│   ├── Log4Qt-2.0.0.tar.gz
│   └── grantlee-*.tar.gz
├── deb/                        # Ubuntu .deb packages (49 packages, 51MB)
└── scripts/                    # Installation scripts
    ├── install-offline.sh      # Offline installation script
    └── README.md               # Package inventory
```

### Download Scripts

**download-all-dependencies.sh**: Downloads all dependencies for offline use
```bash
cd external-dependencies/scripts
./download-all-dependencies.sh [output-directory]
```

**download-deb-packages.sh**: Downloads Ubuntu .deb packages
```bash
./download-deb-packages.sh [output-directory]
```

**verify-offline-cache.sh**: Verifies offline cache completeness
```bash
./verify-offline-cache.sh
```

### Transfer to Air-Gapped Environment

1. Run download scripts on internet-connected machine
2. Copy `offline-cache/` directory to secure media (encrypted USB drive)
3. Transfer to air-gapped environment
4. Mount or copy to Jenkins build server
5. Use offline installation scripts in Dockerfiles

### Important Notes

- **Offline cache is excluded from git** (2.3GB, security best practices)
- Transfer via secure media only
- Verify checksums after transfer
- Keep offline cache updated with dependency version changes
- See `offline-cache/README.md` for detailed usage

## Version Pinning Strategy

All dependencies use **exact version pinning**:
- No floating versions (e.g., `>=1.0.0`)
- No `latest` tags
- Semantic versioning with major.minor.patch
- Git commit SHA for source-based builds

Example:
```dockerfile
ENV QT_VERSION=6.10.2
ENV KDREPORTS_VERSION=2.3.0
ENV LOG4QT_VERSION=2.0.0
```

## Security Considerations

### Proprietary Dependencies

RTI Connext DDS and ArcGIS Runtime SDK are **NOT** included in these pipelines:
- They are proprietary and require commercial licenses
- They must be mounted at runtime from the host
- They are excluded from Docker images and SBOMs

See `DEPENDENCIES-INVENTORY.md` for license requirements.

### License Files

**DO NOT COMMIT LICENSE FILES TO GIT**

The following files are in `.gitignore`:
- `rti_license.dat`: RTI Connext DDS license
- `ArcGIS_Maps_SDK_Qt_linux_300.0.0.zip`: ArcGIS installer

These must be provided at runtime via secure mounting or environment variables.

## Harbor Registry

All built images are pushed to the internal Harbor registry:

```
harbor.sl.defence-elec.de/legatus/<package-name>:<version>
```

Example:
```
harbor.sl.defence-elec.de/legatus/log4qt:2.0.0
```

## Ubuntu 26.04 Base

All containers use Ubuntu 26.04 (Noble Numbat) as the base OS for:
- Consistency across all packages
- Security updates and LTS support
- Compatibility with Qt6 requirements

## Build Container

The main build container (`ubuntu-26.04-build-container`) includes all dependencies and is used for:
- Full Legatus builds
- Integration testing
- Release builds

Usage:
```bash
cd external-dependencies/ubuntu-26.04-build-container
docker build -t legatus-build:ubuntu-26.04 .
```

## SBOM Requirements

All SBOMs follow these standards:

### CycloneDX 1.6
- Component metadata
- License information
- Dependency graph
- PURL identifiers

### SLSA v1.0 Provenance
- Build definition
- External parameters (source, commit, dockerfile)
- Run details (builder ID, invocation, timestamp)

### License Compliance
- Package license identification
- Dependency licenses
- Compliance status
- Review requirements

## Maintenance

### Updating Dependencies

1. Update version in Dockerfile
2. Update version in Jenkinsfile
3. Update version in README.md
4. Run pipeline to verify build
5. Update DEPENDENCIES-INVENTORY.md

### Adding New Dependencies

1. Create directory: `external-dependencies/<package-name>/`
2. Copy template from existing package
3. Update Dockerfile with installation steps
4. Update Jenkinsfile with package metadata
5. Add to DEPENDENCIES-INVENTORY.md
6. Add to dependency graph in README.md

## Troubleshooting

### Build Failures

1. Check Docker build logs
2. Verify network access to package repositories
3. Check for version conflicts
4. Review Jenkins pipeline console output

### SBOM Generation Issues

1. Ensure `generate-sbom.sh` is executable
2. Check environment variables (GIT_URL, GIT_COMMIT, BUILD_NUMBER)
3. Verify output directory permissions

## References

- [DEPENDENCIES-INVENTORY.md](./DEPENDENCIES-INVENTORY.md): Complete dependency list
- [Qt6 Migration Verification Report](../QT6_MIGRATION_VERIFICATION_REPORT.md)
- [Docker Build Script](../qt-migration-legatus/qt6-after/qt6-migrated-legatus/new_docker_build.sh)