- Add ENABLE_COVERAGE CMake option with gcov flags - Create collect-coverage.sh script for LCOV workflow - Add Qt6 test files with QTest framework - Update Jenkinsfile with Coverage stage (main/master/coverage.*) - Update CI Docker image with lcov and genhtml packages - Add .gitignore entries for coverage artifacts - Add COVERAGE.md documentation Coverage workflow: 1. Configure with -DENABLE_COVERAGE=ON 2. Build and run tests to generate .gcda/.gcno files 3. Collect coverage with lcov 4. Filter out Qt/system libraries 5. Generate HTML report and lcov.info for Jenkins Jenkins publishes coverage/lcov.info using Coverage Plugin. Coverage stage only runs on protected branches. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Entire-Checkpoint: 7783c7698334 Entire-Checkpoint: 70341e29996b Signed-off-by: twi-devel <torsten.wierschin@life-maps.com>
145 lines
5.1 KiB
Markdown
145 lines
5.1 KiB
Markdown
# Add LCOV Coverage for a Qt5-to-Qt6 Migration Project
|
|
|
|
Work in the supplied Qt5-to-Qt6 migration repository. Inspect the actual repository layout, CMake targets, tests, and `Jenkinsfile` before making changes. Do not assume target names or paths.
|
|
|
|
## Goal
|
|
|
|
Add reliable LCOV coverage generation for the Qt6 project so Jenkins Coverage Plugin can consume:
|
|
|
|
```text
|
|
coverage/lcov.info
|
|
```
|
|
|
|
## Repository-specific Jenkins CI integration
|
|
|
|
The repository contains Jenkins-related CI support files in `./ci/`. Before changing the build or pipeline, inspect and understand all of the following files:
|
|
|
|
```text
|
|
ci/allowed_signers
|
|
ci/check_pr_policies.py
|
|
ci/Dockerfile.ci
|
|
```
|
|
|
|
Analyze how these files are used by Jenkins, including the CI image, installed tools and versions, signing or trust configuration, pull-request policy checks, workspace-relative paths, user permissions, and any commands or environment variables that affect builds and tests. Adapt the coverage implementation to the existing conventions instead of bypassing them. In particular:
|
|
|
|
- ensure `lcov`, `genhtml`, Qt6, CMake, CTest, and any required Python/runtime dependencies are available in the Jenkins CI image or document the required image changes;
|
|
- preserve and integrate existing policy, signing, and verification checks;
|
|
- use the existing Jenkins container/tooling invocation and workspace layout;
|
|
- do not weaken, remove, or bypass checks from `ci/check_pr_policies.py` or trust rules from `ci/allowed_signers`;
|
|
- update `ci/Dockerfile.ci` and related CI configuration when needed so the coverage stages work reproducibly;
|
|
- verify that coverage artifacts remain workspace-relative and are available to later Jenkins stages;
|
|
- document any CI-file changes and explain how they support the Jenkins coverage workflow.
|
|
|
|
If any of these files are absent, inaccessible, or not actually referenced by the current Jenkins setup, report that explicitly and do not invent their contents or usage.
|
|
|
|
## Requirements
|
|
|
|
- Do not modify the Qt5 baseline under `qt5-before`.
|
|
- Preserve normal non-coverage builds.
|
|
- Add an optional CMake coverage mode, disabled by default, for GCC and Clang where supported.
|
|
- Use debug symbols and suitable low-optimization coverage flags.
|
|
- Ensure Qt6 tests are registered with CTest and run with:
|
|
|
|
```bash
|
|
ctest --test-dir <build-dir> --output-on-failure
|
|
```
|
|
|
|
- Add an executable script, preferably:
|
|
|
|
```text
|
|
scripts/collect-coverage.sh
|
|
```
|
|
|
|
The script must:
|
|
|
|
- accept an optional build directory, defaulting to `build/coverage`;
|
|
- check that `lcov` is installed;
|
|
- optionally require `genhtml` when generating HTML;
|
|
- create `coverage/lcov.info`;
|
|
- optionally create `coverage/html/index.html`;
|
|
- include only project source files;
|
|
- exclude Qt, system, compiler, build, and external dependency paths;
|
|
- use safe error handling such as `set -euo pipefail`;
|
|
- fail on missing, empty, or invalid coverage data;
|
|
- validate that the tracefile contains valid LCOV records.
|
|
|
|
- Update `Jenkinsfile` with explicit stages for:
|
|
|
|
1. configuring the coverage build;
|
|
2. compiling;
|
|
3. running CTest;
|
|
4. collecting LCOV data;
|
|
5. validating and archiving coverage artifacts;
|
|
6. publishing coverage with the Jenkins Coverage Plugin.
|
|
|
|
- Configure the plugin to read exactly:
|
|
|
|
```text
|
|
coverage/lcov.info
|
|
```
|
|
|
|
Prefer `recordCoverage` with the LCOV parser when compatible with the existing Jenkins setup. Inspect the current `Jenkinsfile` and follow its conventions.
|
|
|
|
- The Jenkins build must fail when:
|
|
- a test fails;
|
|
- LCOV generation fails;
|
|
- `coverage/lcov.info` is missing, empty, or invalid.
|
|
|
|
- Use workspace-relative paths only.
|
|
- Preserve generated coverage output outside source control; update `.gitignore` if necessary.
|
|
- Do not commit secrets or fake coverage results.
|
|
- Update the documentation with:
|
|
- required Qt6, CMake, CTest, `lcov`, and `genhtml` versions;
|
|
- Debian/Ubuntu installation instructions;
|
|
- local build, test, and coverage commands;
|
|
- Jenkins Coverage Plugin requirements;
|
|
- artifact locations;
|
|
- the role of `ci/allowed_signers`, `ci/check_pr_policies.py`, and `ci/Dockerfile.ci` in the Jenkins workflow.
|
|
|
|
Document and verify this workflow:
|
|
|
|
```bash
|
|
rm -rf build/coverage coverage
|
|
|
|
cmake -S qt6-after/qt6-migrated-benchmark \
|
|
-B build/coverage \
|
|
-DCMAKE_BUILD_TYPE=Debug \
|
|
-DENABLE_COVERAGE=ON
|
|
|
|
cmake --build build/coverage
|
|
|
|
ctest --test-dir build/coverage \
|
|
--output-on-failure
|
|
|
|
./scripts/collect-coverage.sh build/coverage
|
|
```
|
|
|
|
Expected files:
|
|
|
|
```text
|
|
coverage/lcov.info
|
|
coverage/html/index.html
|
|
```
|
|
|
|
Verify, where possible:
|
|
|
|
- normal configuration and build;
|
|
- coverage configuration and build;
|
|
- CTest execution;
|
|
- LCOV generation;
|
|
- LCOV validation;
|
|
- HTML report generation;
|
|
- ShellCheck for new shell scripts;
|
|
- the Jenkins CI image and `./ci/` policy/signing checks relevant to the coverage stages.
|
|
|
|
At the end, report:
|
|
|
|
- changed files, including any files under `ci/`;
|
|
- coverage-enabled CMake targets;
|
|
- exact LCOV and HTML report paths;
|
|
- Jenkins publication stage;
|
|
- how the `./ci/` files were analyzed and adapted;
|
|
- successful checks;
|
|
- checks skipped because dependencies were unavailable.
|
|
|