Testing Coverage Guide

Complete guide to testing practices, coverage measurement, and quality standards for Armature applications.

Overview
Testing Standards
Running Tests
Measuring Coverage
Types of Tests
Writing Quality Tests
Coverage Goals
Testing Utilities
Best Practices
CI/CD Integration
Examples

Overview

Armature follows rigorous testing standards to ensure reliability, maintainability, and confidence in code changes. This guide covers testing practices, coverage measurement, and quality standards.

Testing Philosophy

High Coverage: Target 85% code coverage across the codebase
Quality over Quantity: Meaningful tests that verify behavior
Fast Feedback: Quick test execution for rapid development
Test Pyramid: Balanced mix of unit, integration, and E2E tests
Behavior Testing: Test what code does, not how it does it

Testing Standards

Coverage Targets

Component	Target	Priority
Core Library	90%+	Critical
Submodules	85%+	High
Examples	50%+	Medium
Documentation Examples	100%	High

Quality Metrics

✅ All public APIs have tests
✅ All error paths covered
✅ Edge cases tested
✅ No flaky tests
✅ Fast execution (< 30s for full suite)
✅ Clear, descriptive test names
✅ Isolated tests (no shared state)

Running Tests

Basic Commands

# Run all tests
cargo test

# Run tests for specific package
cargo test --package armature-core

# Run tests with output
cargo test -- --nocapture

# Run specific test
cargo test test_name

# Run tests in parallel (default)
cargo test

# Run tests serially (for debugging)
cargo test -- --test-threads=1

Running by Type

# Unit tests only
cargo test --lib

# Integration tests only
cargo test --test '*'

# Doc tests only
cargo test --doc

# Specific module tests
cargo test --package armature-core --lib container

Workspace Testing

# All packages
cargo test --workspace

# All packages with all features
cargo test --workspace --all-features

# Exclude examples
cargo test --workspace --lib --bins

Measuring Coverage

Using Tarpaulin (Recommended)

Install tarpaulin:

cargo install cargo-tarpaulin

Generate coverage report:

# HTML report
cargo tarpaulin --out Html --output-dir coverage

# Terminal output
cargo tarpaulin --out Stdout

# XML for CI
cargo tarpaulin --out Xml

# Workspace coverage
cargo tarpaulin --workspace --all-features

# Exclude tests directory
cargo tarpaulin --workspace --exclude-files 'tests/*'

View HTML report:

open coverage/index.html  # macOS
xdg-open coverage/index.html  # Linux

Using llvm-cov

Install llvm-cov:

rustup component add llvm-tools-preview
cargo install cargo-llvm-cov

Generate coverage:

# HTML report
cargo llvm-cov --html --open

# Workspace coverage
cargo llvm-cov --workspace --all-features

# JSON report
cargo llvm-cov --json --output-path coverage.json

# Text summary
cargo llvm-cov --summary-only

Coverage Configuration

Create .tarpaulin.toml:

[build]
workspace = true

[report]
out = ["Html", "Xml"]
output-dir = "coverage"

[coverage]
exclude-files = [
    "tests/*",
    "examples/*",
    "target/*",
]

[run]
all-features = true

Types of Tests

1. Unit Tests

Test individual functions, methods, and modules in isolation.

Location: src/ files with #[cfg(test)] modules

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_user_creation() {
        let user = User::new("Alice");
        assert_eq!(user.name, "Alice");
    }

    #[test]
    fn test_validation_failure() {
        let result = User::validate("");
        assert!(result.is_err());
    }

    #[test]
    #[should_panic(expected = "Invalid ID")]
    fn test_panic_on_invalid_id() {
        User::from_id(-1);
    }
}

2. Integration Tests

Test multiple components working together.

Location: tests/ directory

// tests/api_integration.rs
use armature_framework::prelude::*;
use armature_testing::*;

#[tokio::test]
async fn test_full_request_flow() {
    let app = TestAppBuilder::new()
        .add_module::<AppModule>()
        .build();

    let client = app.client();

    let response = client.get("/users/1").await;

    assert_http_ok(&response);
    assert_json_contains(&response, "name", "Alice");
}

3. Documentation Tests

Test examples in documentation comments.

/// Create a new user.
///
/// # Examples
///
/// ```
/// use armature_framework::User;
///
/// let user = User::new("Alice");
/// assert_eq!(user.name, "Alice");
/// ```
pub fn new(name: &str) -> Self {
    // ...
}

Run doc tests:

cargo test --doc

4. Benchmark Tests (Optional)

Measure performance.

#[cfg(test)]
mod benches {
    use super::*;
    use std::time::Instant;

    #[test]
    fn benchmark_user_creation() {
        let start = Instant::now();

        for _ in 0..10_000 {
            let _ = User::new("Alice");
        }

        let duration = start.elapsed();
        assert!(duration.as_millis() < 100, "Too slow: {:?}", duration);
    }
}

Writing Quality Tests

Test Naming

Use descriptive names that explain what is being tested:

Good:

#[test]
fn test_user_login_succeeds_with_valid_credentials() { }

#[test]
fn test_user_login_fails_with_invalid_password() { }

#[test]
fn test_cache_expires_after_ttl() { }

Bad:

#[test]
fn test1() { }

#[test]
fn test_login() { }  // Too vague

#[test]
fn test_cache() { }  // What about cache?

AAA Pattern

Structure tests with Arrange-Act-Assert:

#[test]
fn test_user_service_creates_user() {
    // Arrange
    let service = UserService::new();
    let user_data = UserData {
        name: "Alice".to_string(),
        email: "alice@example.com".to_string(),
    };

    // Act
    let user = service.create_user(user_data).unwrap();

    // Assert
    assert_eq!(user.name, "Alice");
    assert_eq!(user.email, "alice@example.com");
    assert!(user.id > 0);
}

Test Isolation

Each test should be independent:

Good:

#[test]
fn test_user_creation() {
    let db = create_test_db();  // Fresh DB for this test
    let service = UserService::new(db);

    let user = service.create_user("Alice").unwrap();
    assert_eq!(user.name, "Alice");
}

#[test]
fn test_user_deletion() {
    let db = create_test_db();  // Fresh DB for this test
    let service = UserService::new(db);

    let user = service.create_user("Bob").unwrap();
    service.delete_user(user.id).unwrap();
    assert!(service.get_user(user.id).is_none());
}

Bad:

// ❌ Shared state between tests
static mut GLOBAL_USER: Option<User> = None;

#[test]
fn test_1_create() {
    unsafe { GLOBAL_USER = Some(create_user()); }
}

#[test]
fn test_2_update() {  // Depends on test_1
    unsafe { update_user(GLOBAL_USER.as_ref().unwrap()); }
}

Testing Async Code

Use #[tokio::test] for async tests:

use tokio;

#[tokio::test]
async fn test_async_operation() {
    let service = AsyncService::new();

    let result = service.fetch_data().await;

    assert!(result.is_ok());
}

#[tokio::test]
async fn test_concurrent_requests() {
    let service = Arc::new(AsyncService::new());

    let handles: Vec<_> = (0..10)
        .map(|i| {
            let service = Arc::clone(&service);
            tokio::spawn(async move {
                service.process(i).await
            })
        })
        .collect();

    for handle in handles {
        assert!(handle.await.unwrap().is_ok());
    }
}

Testing Error Cases

Always test error paths:

#[test]
fn test_user_validation_empty_name() {
    let result = User::validate("");
    assert!(result.is_err());
    assert_eq!(
        result.unwrap_err().to_string(),
        "Name cannot be empty"
    );
}

#[test]
fn test_database_connection_failure() {
    let result = Database::connect("invalid://url");
    match result {
        Err(DatabaseError::ConnectionFailed(_)) => {}, // ✅ Expected
        _ => panic!("Expected ConnectionFailed error"),
    }
}

#[test]
#[should_panic(expected = "User not found")]
fn test_panic_on_missing_user() {
    let service = UserService::new();
    service.get_user_or_panic(999);  // Non-existent ID
}

Coverage Goals

Minimum Requirements

Before Merging PR:

✅ Overall coverage: 85%+
✅ New code coverage: 90%+
✅ Critical paths: 100%
✅ All public APIs tested
✅ No decrease in existing coverage

Priority Areas (100% Coverage)

Error Handling
- All error types
- Error conversions
- Error messages
Public APIs
- All public functions
- All public methods
- All traits
Security-Critical Code
- Authentication
- Authorization
- Encryption/Decryption
- Input validation
Data Integrity
- Database operations
- Cache operations
- File I/O

Acceptable Lower Coverage

Private implementation details: 70%+
Examples: 50%+
Generated code: Excluded
External integrations: Mocked

Testing Utilities

Armature provides comprehensive testing utilities in armature-testing.

Test Application Builder

use armature_testing::*;

#[tokio::test]
async fn test_with_test_app() {
    let app = TestAppBuilder::new()
        .add_module::<UserModule>()
        .add_module::<AuthModule>()
        .build();

    let client = app.client();

    let response = client.get("/users").await;
    assert_http_ok(&response);
}

Mock Services

use armature_testing::*;

#[test]
fn test_with_mock_service() {
    let mock_db = MockService::new("Database")
        .with_response("query", Ok("data"))
        .with_response("save", Err("Connection failed"));

    let service = UserService::new(mock_db);

    // Test with mocked responses
    assert!(service.get_data().is_ok());
    assert!(service.save_data().is_err());
}

Spy Pattern

use armature_testing::*;

#[test]
fn test_service_interaction() {
    let real_service = UserService::new();
    let spy = Spy::new(real_service);

    // Use the service
    spy.inner().create_user("Alice");
    spy.inner().create_user("Bob");

    // Verify interactions
    assert_eq!(spy.call_count(), 2);
    assert!(spy.was_called("create_user"));
}

Assertions

use armature_testing::*;

#[tokio::test]
async fn test_assertions() {
    let response = client.get("/api/user/1").await;

    // HTTP assertions
    assert_http_ok(&response);
    assert_http_status(&response, 200);

    // JSON assertions
    assert_json_contains(&response, "name", "Alice");
    assert_json_path(&response, "$.user.email", "alice@example.com");

    // Header assertions
    assert_header(&response, "Content-Type", "application/json");
}

Best Practices

1. Test One Thing

Each test should verify one specific behavior:

Good:

#[test]
fn test_user_name_validation_accepts_valid_names() {
    assert!(User::validate_name("Alice").is_ok());
}

#[test]
fn test_user_name_validation_rejects_empty_names() {
    assert!(User::validate_name("").is_err());
}

#[test]
fn test_user_name_validation_rejects_too_long_names() {
    let long_name = "a".repeat(300);
    assert!(User::validate_name(&long_name).is_err());
}

Bad:

#[test]
fn test_user_validation() {
    // ❌ Testing multiple behaviors
    assert!(User::validate_name("Alice").is_ok());
    assert!(User::validate_name("").is_err());
    assert!(User::validate_email("alice@example.com").is_ok());
    assert!(User::validate_age(25).is_ok());
}

2. Use Helper Functions

Extract common setup into helper functions:

#[cfg(test)]
mod tests {
    use super::*;

    fn create_test_user(name: &str) -> User {
        User {
            id: 1,
            name: name.to_string(),
            email: format!("{}@example.com", name.to_lowercase()),
            created_at: Utc::now(),
        }
    }

    fn create_test_db() -> TestDatabase {
        TestDatabase::new().with_migrations()
    }

    #[test]
    fn test_user_operations() {
        let db = create_test_db();
        let user = create_test_user("Alice");

        // Test logic...
    }
}

3. Test Edge Cases

Don't just test the happy path:

#[test]
fn test_pagination_edge_cases() {
    let service = UserService::new();

    // Empty list
    assert_eq!(service.paginate(0, 10).len(), 0);

    // First page
    assert_eq!(service.paginate(1, 10).len(), 10);

    // Last page (partial)
    assert_eq!(service.paginate(5, 10).len(), 7);

    // Beyond last page
    assert_eq!(service.paginate(100, 10).len(), 0);

    // Invalid page size
    assert!(service.paginate(1, 0).is_empty());
    assert!(service.paginate(1, -1).is_empty());
}

4. Avoid Test Interdependence

Tests should not depend on execution order:

Good:

#[test]
fn test_a() {
    let data = setup_test_data();
    // Test with data
}

#[test]
fn test_b() {
    let data = setup_test_data();  // Independent setup
    // Test with data
}

Bad:

static mut SHARED_STATE: Option<Data> = None;

#[test]
fn test_a() {
    unsafe { SHARED_STATE = Some(create_data()); }
}

#[test]
fn test_b() {
    // ❌ Depends on test_a running first
    unsafe { use_data(SHARED_STATE.as_ref().unwrap()); }
}

5. Use Descriptive Assertions

Make failures easy to understand:

Good:

assert_eq!(
    user.age, 25,
    "User age should be 25, but was {}",
    user.age
);

assert!(
    result.is_ok(),
    "Expected successful result, got error: {:?}",
    result.err()
);

Bad:

assert!(user.age == 25);  // ❌ No context on failure
assert!(result.is_ok());   // ❌ No error information

6. Clean Up Resources

Ensure tests clean up after themselves:

#[test]
fn test_file_operations() {
    let temp_file = "test_output.txt";

    // Test code...
    write_file(temp_file, "data");

    // Clean up
    let _ = std::fs::remove_file(temp_file);
}

// Better: Use RAII
#[test]
fn test_file_operations_safe() {
    let _temp_file = TempFile::new("test_output.txt");  // Auto-cleanup on drop

    // Test code...
}

CI/CD Integration

GitHub Actions

Create .github/workflows/test.yml:

name: Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Install Rust
        uses: actions-rs/toolchain@v1
        with:
          toolchain: stable
          override: true

      - name: Run tests
        run: cargo test --workspace --all-features

      - name: Install tarpaulin
        run: cargo install cargo-tarpaulin

      - name: Generate coverage
        run: cargo tarpaulin --workspace --all-features --out Xml

      - name: Upload coverage to Codecov
        uses: codecov/codecov-action@v3
        with:
          files: ./cobertura.xml
          fail_ci_if_error: true

      - name: Check coverage threshold
        run: |
          coverage=$(cargo tarpaulin --workspace --all-features --out Stdout | grep -oP '\d+\.\d+(?=%)')
          if (( $(echo "$coverage < 85.0" | bc -l) )); then
            echo "Coverage $coverage% is below 85% threshold"
            exit 1
          fi

Coverage Badges

Add to README.md:

[![codecov](https://codecov.io/gh/username/armature/branch/main/graph/badge.svg)](https://codecov.io/gh/username/armature)

Pre-commit Hook

Create .git/hooks/pre-commit:

#!/bin/bash
set -e

echo "Running tests..."
cargo test --workspace

echo "Checking coverage..."
cargo tarpaulin --workspace --all-features | grep -q '85\.'

echo "✅ All checks passed"

Examples

Complete Test Module

#[cfg(test)]
mod tests {
    use super::*;

    // Helper functions
    fn setup() -> TestEnvironment {
        TestEnvironment::new()
    }

    fn create_test_user(name: &str) -> User {
        User::new(name)
    }

    // Unit tests
    #[test]
    fn test_user_creation() {
        let user = create_test_user("Alice");
        assert_eq!(user.name, "Alice");
    }

    #[test]
    fn test_user_validation_empty_name() {
        let result = User::validate("");
        assert!(result.is_err());
    }

    // Async tests
    #[tokio::test]
    async fn test_async_user_fetch() {
        let service = UserService::new();
        let user = service.fetch_user(1).await.unwrap();
        assert_eq!(user.id, 1);
    }

    // Error tests
    #[test]
    #[should_panic(expected = "Invalid ID")]
    fn test_panic_on_invalid_id() {
        User::from_id(-1);
    }

    // Integration tests
    #[tokio::test]
    async fn test_full_user_flow() {
        let env = setup();

        // Create
        let user = env.service.create_user("Alice").await.unwrap();

        // Read
        let fetched = env.service.get_user(user.id).await.unwrap();
        assert_eq!(fetched.name, "Alice");

        // Update
        env.service.update_user(user.id, "Alice Updated").await.unwrap();

        // Delete
        env.service.delete_user(user.id).await.unwrap();
        assert!(env.service.get_user(user.id).await.is_none());
    }
}

Summary

Key Takeaways:

✅ Target 85% coverage across the codebase
✅ Write meaningful tests that verify behavior
✅ Test error paths as thoroughly as happy paths
✅ Keep tests isolated and independent
✅ Use descriptive names and clear assertions
✅ Measure coverage regularly with tarpaulin or llvm-cov
✅ Integrate testing into CI/CD pipeline
✅ Follow the test pyramid: Many unit tests, some integration tests, few E2E tests

Testing Checklist:

All public APIs have tests
All error cases covered
Edge cases tested
Async code properly tested
No test interdependence
Clear, descriptive test names
Coverage meets 85% threshold
CI/CD pipeline includes tests
Documentation examples compile
No flaky tests

Resources:

Rust Testing Book
cargo-tarpaulin
cargo-llvm-cov
armature-testing - Built-in testing utilities

Quality testing is not about achieving 100% coverage—it's about having confidence that your code works correctly in all scenarios. Focus on testing behavior, edge cases, and error handling to build robust, reliable applications.

Testing Coverage Guide

Table of Contents

Overview

Testing Philosophy

Testing Standards

Coverage Targets

Quality Metrics

Running Tests

Basic Commands

Running by Type

Workspace Testing

Measuring Coverage

Using Tarpaulin (Recommended)

Using llvm-cov

Coverage Configuration

Types of Tests

1. Unit Tests

2. Integration Tests

3. Documentation Tests

4. Benchmark Tests (Optional)

Writing Quality Tests

Test Naming

AAA Pattern

Test Isolation

Testing Async Code

Testing Error Cases

Coverage Goals

Minimum Requirements

Priority Areas (100% Coverage)

Acceptable Lower Coverage

Testing Utilities

Test Application Builder

Mock Services

Spy Pattern

Assertions

Best Practices

1. Test One Thing

2. Use Helper Functions

3. Test Edge Cases

4. Avoid Test Interdependence

5. Use Descriptive Assertions

6. Clean Up Resources

CI/CD Integration

GitHub Actions

Coverage Badges

Pre-commit Hook

Examples

Complete Test Module

Summary