feat: Add LCOV coverage support for Jenkins CI
- Add ENABLE_COVERAGE CMake option with gcov flags - Create collect-coverage.sh script for LCOV workflow - Add Qt6 test files with QTest framework - Update Jenkinsfile with Coverage stage (main/master/coverage.*) - Update CI Docker image with lcov and genhtml packages - Add .gitignore entries for coverage artifacts - Add COVERAGE.md documentation Coverage workflow: 1. Configure with -DENABLE_COVERAGE=ON 2. Build and run tests to generate .gcda/.gcno files 3. Collect coverage with lcov 4. Filter out Qt/system libraries 5. Generate HTML report and lcov.info for Jenkins Jenkins publishes coverage/lcov.info using Coverage Plugin. Coverage stage only runs on protected branches. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Entire-Checkpoint: 7783c7698334 Entire-Checkpoint: 70341e29996b Signed-off-by: twi-devel <torsten.wierschin@life-maps.com>
This commit is contained in:
19 files changed
+734
-235
No files matched your search
@@ -0,0 +1,241 @@
|
||||
# Code Coverage with LCOV
|
||||
|
||||
This document describes how to generate and view code coverage reports for the Qt6 migrated benchmark project.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### Local Development
|
||||
|
||||
Install the required tools:
|
||||
|
||||
```bash
|
||||
# Ubuntu/Debian
|
||||
sudo apt-get install lcov genhtml
|
||||
|
||||
# Fedora/RHEL
|
||||
sudo dnf install lcov
|
||||
|
||||
# macOS (with Homebrew)
|
||||
brew install lcov
|
||||
```
|
||||
|
||||
### CI/CD Environment
|
||||
|
||||
The CI Docker image (`lifemaps/ci-cpp:latest`) includes `lcov` and `genhtml`. If you're building a custom CI image, ensure these packages are installed:
|
||||
|
||||
```dockerfile
|
||||
RUN apt-get install -y --no-install-recommends \
|
||||
lcov \
|
||||
genhtml
|
||||
```
|
||||
|
||||
## Generating Coverage Reports
|
||||
|
||||
### Local Coverage
|
||||
|
||||
1. **Navigate to the project directory:**
|
||||
```bash
|
||||
cd qt6-after/qt6-migrated-benchmark
|
||||
```
|
||||
|
||||
2. **Run the coverage collection script:**
|
||||
```bash
|
||||
./scripts/collect-coverage.sh
|
||||
```
|
||||
|
||||
Or specify a custom build directory:
|
||||
```bash
|
||||
./scripts/collect-coverage.sh /path/to/build-dir
|
||||
```
|
||||
|
||||
3. **View the HTML report:**
|
||||
```bash
|
||||
xdg-open coverage/html/index.html # Linux
|
||||
open coverage/html/index.html # macOS
|
||||
```
|
||||
|
||||
### Manual Coverage Generation
|
||||
|
||||
If you prefer manual control:
|
||||
|
||||
```bash
|
||||
# 1. Configure with coverage enabled
|
||||
cmake -S qt6-after/qt6-migrated-benchmark -B build-coverage -G Ninja \
|
||||
-DCMAKE_BUILD_TYPE=Debug \
|
||||
-DENABLE_COVERAGE=ON
|
||||
|
||||
# 2. Build the project
|
||||
cmake --build build-coverage
|
||||
|
||||
# 3. Run tests to generate coverage data
|
||||
ctest --test-dir build-coverage --output-on-failure
|
||||
|
||||
# 4. Collect coverage data
|
||||
lcov --directory build-coverage \
|
||||
--capture \
|
||||
--output-file coverage.info \
|
||||
--ignore-errors mismatch \
|
||||
--no-external
|
||||
|
||||
# 5. Filter out system and Qt libraries
|
||||
lcov --remove coverage.info \
|
||||
'/usr/*' \
|
||||
'/opt/*' \
|
||||
'*/Qt6/*' \
|
||||
'*/tests/*' \
|
||||
--output-file coverage.filtered.info \
|
||||
--ignore-errors mismatch
|
||||
|
||||
# 6. Generate HTML report
|
||||
genhtml coverage.filtered.info \
|
||||
--output-directory coverage/html \
|
||||
--branch-coverage \
|
||||
--ignore-errors mismatch \
|
||||
--title "Qt6 Migration Benchmark Coverage"
|
||||
```
|
||||
|
||||
## CI/CD Integration
|
||||
|
||||
### Jenkins Pipeline
|
||||
|
||||
The Jenkinsfile automatically runs coverage collection on `main`, `master`, or branches matching `coverage.*`. The coverage report is published using the Jenkins Coverage Plugin.
|
||||
|
||||
**Coverage Stage Configuration:**
|
||||
- Runs only on protected branches
|
||||
- Expects `coverage/lcov.info` at workspace root
|
||||
- Publishes coverage trends across builds
|
||||
|
||||
**Viewing Coverage in Jenkins:**
|
||||
1. Navigate to your job in Jenkins
|
||||
2. Click on a build number
|
||||
3. Find the "Coverage Report" section in the sidebar
|
||||
4. Click to view detailed coverage statistics
|
||||
|
||||
### Coverage Report Location
|
||||
|
||||
After the Jenkins build completes:
|
||||
- **LCOV data:** `coverage/lcov.info` (relative to workspace root)
|
||||
- **HTML report:** Available in Jenkins UI via Coverage Plugin
|
||||
|
||||
## CMake Configuration
|
||||
|
||||
Coverage is controlled by the `ENABLE_COVERAGE` CMake option:
|
||||
|
||||
```cmake
|
||||
option(ENABLE_COVERAGE "Enable code coverage with gcov/lcov" OFF)
|
||||
```
|
||||
|
||||
### Coverage Flags
|
||||
|
||||
When enabled, the following flags are applied:
|
||||
|
||||
**Compile flags:**
|
||||
- `-g` - Debug symbols
|
||||
- `-O0` - No optimization
|
||||
- `--coverage` - Enable coverage instrumentation
|
||||
- `-fprofile-arcs` - Generate arc profiling data
|
||||
- `-ftest-coverage` - Generate coverage data
|
||||
|
||||
**Link flags:**
|
||||
- `--coverage` - Link with coverage libraries
|
||||
- `-fprofile-arcs` - Link arc profiling support
|
||||
|
||||
## Test Requirements
|
||||
|
||||
Coverage reports are only as good as your test suite. Ensure you have:
|
||||
|
||||
1. **Unit tests** for individual functions and methods
|
||||
2. **Integration tests** for component interactions
|
||||
3. **Edge case coverage** for boundary conditions
|
||||
4. **Error path coverage** for exception handling
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
ctest --test-dir build-coverage --output-on-failure
|
||||
|
||||
# Run specific test
|
||||
ctest --test-dir build-coverage -R TestMainWindow --output-on-failure
|
||||
|
||||
# Run with verbose output
|
||||
ctest --test-dir build-coverage -V
|
||||
```
|
||||
|
||||
## Interpreting Coverage Results
|
||||
|
||||
### Coverage Metrics
|
||||
|
||||
- **Lines:** Percentage of executable lines executed
|
||||
- **Functions:** Percentage of functions called
|
||||
- **Branches:** Percentage of decision branches taken (if enabled)
|
||||
|
||||
### Coverage Targets
|
||||
|
||||
Aim for:
|
||||
- **>80%** line coverage for production code
|
||||
- **>70%** branch coverage for critical paths
|
||||
- **100%** coverage for safety-critical modules
|
||||
|
||||
### Coverage Report Structure
|
||||
|
||||
The HTML report includes:
|
||||
- **Summary table:** Overall coverage statistics
|
||||
- **File list:** Per-file coverage breakdown
|
||||
- **Source view:** Line-by-line coverage with highlighting
|
||||
- Green: Executed lines
|
||||
- Red: Never executed lines
|
||||
- Yellow: Partially executed branches
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### No Coverage Data Generated
|
||||
|
||||
**Problem:** `lcov` reports "no valid records found"
|
||||
|
||||
**Solutions:**
|
||||
1. Ensure tests actually run and complete successfully
|
||||
2. Verify `ENABLE_COVERAGE=ON` was set during CMake configuration
|
||||
3. Check that `.gcda` files are generated in the build directory
|
||||
4. Ensure the same compiler is used for build and coverage collection
|
||||
|
||||
### Mismatch Errors
|
||||
|
||||
**Problem:** `lcov` reports "version mismatch"
|
||||
|
||||
**Solutions:**
|
||||
1. Clean the build directory: `rm -rf build-coverage`
|
||||
2. Reconfigure and rebuild with coverage enabled
|
||||
3. Ensure consistent compiler versions
|
||||
|
||||
### Low Coverage Numbers
|
||||
|
||||
**Problem:** Coverage is lower than expected
|
||||
|
||||
**Solutions:**
|
||||
1. Verify all tests are being executed (check CTest output)
|
||||
2. Check if code is conditionally compiled out
|
||||
3. Ensure test data exercises all code paths
|
||||
4. Review filtered paths in `collect-coverage.sh`
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Run coverage regularly** - Integrate into CI/CD pipeline
|
||||
2. **Track trends** - Monitor coverage over time, not just absolute values
|
||||
3. **Focus on critical paths** - Prioritize coverage of business logic
|
||||
4. **Don't chase 100%** - Aim for meaningful coverage, not vanity metrics
|
||||
5. **Review uncovered code** - Identify dead code or missing tests
|
||||
|
||||
## Related Files
|
||||
|
||||
- `CMakeLists.txt` - CMake configuration with coverage option
|
||||
- `scripts/collect-coverage.sh` - Automated coverage collection script
|
||||
- `Jenkinsfile` - CI/CD pipeline with coverage stage
|
||||
- `ci/Dockerfile.ci` - Docker image with coverage tools
|
||||
|
||||
## References
|
||||
|
||||
- [LCOV Documentation](http://ltp.sourceforge.net/coverage/lcov.php)
|
||||
- [CMake Testing with CTest](https://cmake.org/cmake/help/latest/manual/ctest.1.html)
|
||||
- [GCC Coverage Instrumentation](https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html)
|
||||
- [Jenkins Coverage Plugin](https://plugins.jenkins.io/code-coverage-api/)
|
||||
Reference in new issue
Block a user