qt-migration-copilot-demo/qt6-after/qt6-migrated-benchmark/COVERAGE.md

Code Coverage with LCOV

This document describes how to generate and view code coverage reports for the Qt6 migrated benchmark project.

Prerequisites

Local Development

Install the required tools:

# Ubuntu/Debian
sudo apt-get install lcov genhtml

# Fedora/RHEL
sudo dnf install lcov

# macOS (with Homebrew)
brew install lcov

CI/CD Environment

The CI Docker image (lifemaps/ci-cpp:latest) includes lcov and genhtml. If you're building a custom CI image, ensure these packages are installed:

RUN apt-get install -y --no-install-recommends \
    lcov \
    genhtml

Generating Coverage Reports

Local Coverage

1. Navigate to the project directory:

   cd qt6-after/qt6-migrated-benchmark
   

2. Run the coverage collection script:

   ./scripts/collect-coverage.sh
   

   Or specify a custom build directory:

   ./scripts/collect-coverage.sh /path/to/build-dir
   

3. View the HTML report:

   xdg-open coverage/html/index.html  # Linux
   open coverage/html/index.html      # macOS
   

Manual Coverage Generation

If you prefer manual control:

# 1. Configure with coverage enabled
cmake -S qt6-after/qt6-migrated-benchmark -B build-coverage -G Ninja \
    -DCMAKE_BUILD_TYPE=Debug \
    -DENABLE_COVERAGE=ON

# 2. Build the project
cmake --build build-coverage

# 3. Run tests to generate coverage data
ctest --test-dir build-coverage --output-on-failure

# 4. Collect coverage data
lcov --directory build-coverage \
     --capture \
     --output-file coverage.info \
     --ignore-errors mismatch \
     --no-external

# 5. Filter out system and Qt libraries
lcov --remove coverage.info \
     '/usr/*' \
     '/opt/*' \
     '*/Qt6/*' \
     '*/tests/*' \
     --output-file coverage.filtered.info \
     --ignore-errors mismatch

# 6. Generate HTML report
genhtml coverage.filtered.info \
          --output-directory coverage/html \
          --branch-coverage \
          --ignore-errors mismatch \
          --title "Qt6 Migration Benchmark Coverage"

CI/CD Integration

Jenkins Pipeline

The Jenkinsfile automatically runs coverage collection on main, master, or branches matching coverage.*. The coverage report is published using the Jenkins Coverage Plugin.

Coverage Stage Configuration:

• Runs only on protected branches
• Expects coverage/lcov.info at workspace root
• Publishes coverage trends across builds

Viewing Coverage in Jenkins:

1. Navigate to your job in Jenkins
2. Click on a build number
3. Find the "Coverage Report" section in the sidebar
4. Click to view detailed coverage statistics

Coverage Report Location

After the Jenkins build completes:

• LCOV data: coverage/lcov.info (relative to workspace root)
• HTML report: Available in Jenkins UI via Coverage Plugin

CMake Configuration

Coverage is controlled by the ENABLE_COVERAGE CMake option:

option(ENABLE_COVERAGE "Enable code coverage with gcov/lcov" OFF)

Coverage Flags

When enabled, the following flags are applied:

Compile flags:

• -g - Debug symbols
• -O0 - No optimization
• --coverage - Enable coverage instrumentation
• -fprofile-arcs - Generate arc profiling data
• -ftest-coverage - Generate coverage data

Link flags:

• --coverage - Link with coverage libraries
• -fprofile-arcs - Link arc profiling support

Test Requirements

Coverage reports are only as good as your test suite. Ensure you have:

1. Unit tests for individual functions and methods
2. Integration tests for component interactions
3. Edge case coverage for boundary conditions
4. Error path coverage for exception handling

Running Tests

# Run all tests
ctest --test-dir build-coverage --output-on-failure

# Run specific test
ctest --test-dir build-coverage -R TestMainWindow --output-on-failure

# Run with verbose output
ctest --test-dir build-coverage -V

Interpreting Coverage Results

Coverage Metrics

• Lines: Percentage of executable lines executed
• Functions: Percentage of functions called
• Branches: Percentage of decision branches taken (if enabled)

Coverage Targets

Aim for:

• >80% line coverage for production code
• >70% branch coverage for critical paths
• 100% coverage for safety-critical modules

Coverage Report Structure

The HTML report includes:

• Summary table: Overall coverage statistics
• File list: Per-file coverage breakdown
• Source view: Line-by-line coverage with highlighting
  • Green: Executed lines
  • Red: Never executed lines
  • Yellow: Partially executed branches

Troubleshooting

No Coverage Data Generated

Problem: lcov reports "no valid records found"

Solutions:

1. Ensure tests actually run and complete successfully
2. Verify ENABLE_COVERAGE=ON was set during CMake configuration
3. Check that .gcda files are generated in the build directory
4. Ensure the same compiler is used for build and coverage collection

Mismatch Errors

Problem: lcov reports "version mismatch"

Solutions:

1. Clean the build directory: rm -rf build-coverage
2. Reconfigure and rebuild with coverage enabled
3. Ensure consistent compiler versions

Low Coverage Numbers

Problem: Coverage is lower than expected

Solutions:

1. Verify all tests are being executed (check CTest output)
2. Check if code is conditionally compiled out
3. Ensure test data exercises all code paths
4. Review filtered paths in collect-coverage.sh

Best Practices

1. Run coverage regularly - Integrate into CI/CD pipeline
2. Track trends - Monitor coverage over time, not just absolute values
3. Focus on critical paths - Prioritize coverage of business logic
4. Don't chase 100% - Aim for meaningful coverage, not vanity metrics
5. Review uncovered code - Identify dead code or missing tests

Related Files

• CMakeLists.txt - CMake configuration with coverage option
• scripts/collect-coverage.sh - Automated coverage collection script
• Jenkinsfile - CI/CD pipeline with coverage stage
• ci/Dockerfile.ci - Docker image with coverage tools

References

• LCOV Documentation
• CMake Testing with CTest
• GCC Coverage Instrumentation
• Jenkins Coverage Plugin