qt-migration-copilot-demo/qt6-after/qt6-migrated-benchmark/COVERAGE.md

# Code Coverage with LCOV

This document describes how to generate and view code coverage reports for the Qt6 migrated benchmark project.

## Prerequisites

### Local Development

Install the required tools:

```bash
# Ubuntu/Debian
sudo apt-get install lcov genhtml

# Fedora/RHEL
sudo dnf install lcov

# macOS (with Homebrew)
brew install lcov
```

### CI/CD Environment

The CI Docker image (`lifemaps/ci-cpp:latest`) includes `lcov` and `genhtml`. If you're building a custom CI image, ensure these packages are installed:

```dockerfile
RUN apt-get install -y --no-install-recommends \
    lcov \
    genhtml
```

## Generating Coverage Reports

### Local Coverage

1. **Navigate to the project directory:**
   ```bash
   cd qt6-after/qt6-migrated-benchmark
   ```

2. **Run the coverage collection script:**
   ```bash
   ./scripts/collect-coverage.sh
   ```

   Or specify a custom build directory:
   ```bash
   ./scripts/collect-coverage.sh /path/to/build-dir
   ```

3. **View the HTML report:**
   ```bash
   xdg-open coverage/html/index.html  # Linux
   open coverage/html/index.html      # macOS
   ```

### Manual Coverage Generation

If you prefer manual control:

```bash
# 1. Configure with coverage enabled
cmake -S qt6-after/qt6-migrated-benchmark -B build-coverage -G Ninja \
    -DCMAKE_BUILD_TYPE=Debug \
    -DENABLE_COVERAGE=ON

# 2. Build the project
cmake --build build-coverage

# 3. Run tests to generate coverage data
ctest --test-dir build-coverage --output-on-failure

# 4. Collect coverage data
lcov --directory build-coverage \
     --capture \
     --output-file coverage.info \
     --ignore-errors mismatch \
     --no-external

# 5. Filter out system and Qt libraries
lcov --remove coverage.info \
     '/usr/*' \
     '/opt/*' \
     '*/Qt6/*' \
     '*/tests/*' \
     --output-file coverage.filtered.info \
     --ignore-errors mismatch

# 6. Generate HTML report
genhtml coverage.filtered.info \
          --output-directory coverage/html \
          --branch-coverage \
          --ignore-errors mismatch \
          --title "Qt6 Migration Benchmark Coverage"
```

## CI/CD Integration

### Jenkins Pipeline

The Jenkinsfile automatically runs coverage collection on `main`, `master`, or branches matching `coverage.*`. The coverage report is published using the Jenkins Coverage Plugin.

**Coverage Stage Configuration:**
- Runs only on protected branches
- Expects `coverage/lcov.info` at workspace root
- Publishes coverage trends across builds

**Viewing Coverage in Jenkins:**
1. Navigate to your job in Jenkins
2. Click on a build number
3. Find the "Coverage Report" section in the sidebar
4. Click to view detailed coverage statistics

### Coverage Report Location

After the Jenkins build completes:
- **LCOV data:** `coverage/lcov.info` (relative to workspace root)
- **HTML report:** Available in Jenkins UI via Coverage Plugin

## CMake Configuration

Coverage is controlled by the `ENABLE_COVERAGE` CMake option:

```cmake
option(ENABLE_COVERAGE "Enable code coverage with gcov/lcov" OFF)
```

### Coverage Flags

When enabled, the following flags are applied:

**Compile flags:**
- `-g` - Debug symbols
- `-O0` - No optimization
- `--coverage` - Enable coverage instrumentation
- `-fprofile-arcs` - Generate arc profiling data
- `-ftest-coverage` - Generate coverage data

**Link flags:**
- `--coverage` - Link with coverage libraries
- `-fprofile-arcs` - Link arc profiling support

## Test Requirements

Coverage reports are only as good as your test suite. Ensure you have:

1. **Unit tests** for individual functions and methods
2. **Integration tests** for component interactions
3. **Edge case coverage** for boundary conditions
4. **Error path coverage** for exception handling

### Running Tests

```bash
# Run all tests
ctest --test-dir build-coverage --output-on-failure

# Run specific test
ctest --test-dir build-coverage -R TestMainWindow --output-on-failure

# Run with verbose output
ctest --test-dir build-coverage -V
```

## Interpreting Coverage Results

### Coverage Metrics

- **Lines:** Percentage of executable lines executed
- **Functions:** Percentage of functions called
- **Branches:** Percentage of decision branches taken (if enabled)

### Coverage Targets

Aim for:
- **>80%** line coverage for production code
- **>70%** branch coverage for critical paths
- **100%** coverage for safety-critical modules

### Coverage Report Structure

The HTML report includes:
- **Summary table:** Overall coverage statistics
- **File list:** Per-file coverage breakdown
- **Source view:** Line-by-line coverage with highlighting
  - Green: Executed lines
  - Red: Never executed lines
  - Yellow: Partially executed branches

## Troubleshooting

### No Coverage Data Generated

**Problem:** `lcov` reports "no valid records found"

**Solutions:**
1. Ensure tests actually run and complete successfully
2. Verify `ENABLE_COVERAGE=ON` was set during CMake configuration
3. Check that `.gcda` files are generated in the build directory
4. Ensure the same compiler is used for build and coverage collection

### Mismatch Errors

**Problem:** `lcov` reports "version mismatch"

**Solutions:**
1. Clean the build directory: `rm -rf build-coverage`
2. Reconfigure and rebuild with coverage enabled
3. Ensure consistent compiler versions

### Low Coverage Numbers

**Problem:** Coverage is lower than expected

**Solutions:**
1. Verify all tests are being executed (check CTest output)
2. Check if code is conditionally compiled out
3. Ensure test data exercises all code paths
4. Review filtered paths in `collect-coverage.sh`

## Best Practices

1. **Run coverage regularly** - Integrate into CI/CD pipeline
2. **Track trends** - Monitor coverage over time, not just absolute values
3. **Focus on critical paths** - Prioritize coverage of business logic
4. **Don't chase 100%** - Aim for meaningful coverage, not vanity metrics
5. **Review uncovered code** - Identify dead code or missing tests

## Related Files

- `CMakeLists.txt` - CMake configuration with coverage option
- `scripts/collect-coverage.sh` - Automated coverage collection script
- `Jenkinsfile` - CI/CD pipeline with coverage stage
- `ci/Dockerfile.ci` - Docker image with coverage tools

## References

- [LCOV Documentation](http://ltp.sourceforge.net/coverage/lcov.php)
- [CMake Testing with CTest](https://cmake.org/cmake/help/latest/manual/ctest.1.html)
- [GCC Coverage Instrumentation](https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html)
- [Jenkins Coverage Plugin](https://plugins.jenkins.io/code-coverage-api/)