provisioning-catalog/providers/aws/tests/README.md

# AWS Provider Test Suite

**Version**: 1.0
**Last Updated**: 2026-01-07
**Status**: Production Ready (Mock Mode - No Real API Calls)

---

## Overview

The AWS Provider Test Suite provides comprehensive testing coverage with 51 deterministic tests running in mock mode. All tests use pre-defined AWS 
API response fixtures with no external API calls required.

**Test Structure**: 14 unit tests + 37 integration tests = 51 total tests

---

## Running Tests

### Run All Tests

```bash
cd provisioning/extensions/providers/aws
nu tests/run_aws_tests.nu
```

**Expected Result**: All 51 tests pass in less than 5 seconds with no external API calls.

### Run Specific Test Module

```bash
# Unit tests only
nu tests/unit/test_utils.nu

# API client tests only
nu tests/integration/test_api_client.nu

# Server lifecycle tests only
nu tests/integration/test_server_lifecycle.nu

# Pricing tests only
nu tests/integration/test_pricing.nu
```

---

## Test Breakdown

### Unit Tests (14 tests)

**File**: `tests/unit/test_utils.nu`

Tests utility functions for validation and parsing:

- Instance ID format validation (i-1234567890abcdef0)
- IPv4 address extraction and validation (10.0.1.100)
- Instance type validation (t3.micro, m5.large, c5.xlarge)
- Availability zone validation (us-east-1a, eu-west-1b)
- EBS volume ID validation (vol-12345678)
- Volume state validation (creating, available, in-use, deleting)
- CIDR block format validation (10.0.0.0/16)
- Volume type validation (gp2, gp3, io1, io2, st1, sc1)
- ISO 8601 timestamp parsing (2025-01-07T10:00:00.000Z)
- Server state validation (pending, running, stopped, terminated)
- Security group ID validation (sg-12345678)
- Memory specification validation (512 MB, 1 GB, 4 GB)
- vCPU count validation (1, 2, 4, 8, 16, 32)

**Result**: 14/14 PASS ✓

### Integration Tests (37 tests)

#### API Client Tests (13 tests)

**File**: `tests/integration/test_api_client.nu`

Tests AWS EC2 API client functionality:

- describe-instances returns list of reservations
- Reservation structure contains instances array
- Single instance response with all required fields
- describe-volumes list response structure
- Volume response validation (VolumeId, Size, State, AZ)
- Create instance response in pending state
- Instance type information structure
- Error response 401 (UnauthorizedOperation)
- Error response 404 (InvalidInstanceID.NotFound)
- Error response 429 (RequestLimitExceeded)
- Instance state values (running, stopped)
- Key pair information availability
- Pricing data structure validation
- Volume attachment information

**Result**: 13/13 PASS ✓

#### Server Lifecycle Tests (12 tests)

**File**: `tests/integration/test_server_lifecycle.nu`

Tests server lifecycle operations:

- List servers returns EC2 instances
- Server info has required fields (InstanceId, Type, State, IPs)
- Hostname tag lookup matches expected values
- Server state validation (running vs stopped)
- Public IP address presence and format
- Create instance response validation
- Instance type details (vCPU count, memory)
- Availability zone information
- Block device mapping (attached volumes)
- Security group assignment
- Launch time recording
- Multiple instance state differentiation

**Result**: 12/12 PASS ✓

#### Pricing Tests (12 tests)

**File**: `tests/integration/test_pricing.nu`

Tests pricing and cost estimation:

- Pricing data has instance type field
- Pricing includes on-demand rate
- Pricing includes reserved rate (lower than on-demand)
- Instance type filtering (t3.micro, t3.small, t3.medium)
- Price scaling (larger instances cost more)
- vCPU correlation with pricing
- Memory format specification (GB units)
- Monthly cost calculation (hourly rate times 730)
- All pricing entries complete
- Volume pricing available
- gp3 volume IOPS and throughput specifications
- Volume cost estimation (per GB-month)

**Result**: 12/12 PASS ✓

---

## Mock Data

### Mock Responses File

**File**: `tests/mocks/mock_api_responses.json`

Contains realistic AWS EC2 API response fixtures:

- **describe_instances_list**: Two test instances (one running, one stopped)
- **describe_instances_single**: Single running instance with full details
- **describe_volumes_list**: Two test volumes (one in-use, one available)
- **describe_volumes_single**: Single volume with attachment information
- **create_instance_response**: New instance response in pending state
- **describe_key_pairs**: SSH key pair listing with aws-default key
- **describe_instance_types**: Three instance type specifications
- **error_401**: Unauthorized operation error response
- **error_404**: Resource not found error response
- **error_429**: Rate limit exceeded error response
- **pricing_data**: Three pricing tiers (t3.micro, t3.small, t3.medium)

All responses match the actual AWS EC2 API response format.

---

## Test Features

✅ **Deterministic**: Pre-defined mock data ensures consistent results
✅ **Fast**: All 51 tests complete in less than 5 seconds
✅ **Isolated**: No AWS credentials required, no side effects
✅ **CI/CD Ready**: Exit code 0 for success, 1 for failure
✅ **Comprehensive**: Core provider functionality coverage
✅ **Realistic**: Mock data matches actual AWS API responses
✅ **Maintainable**: Tests organized by functionality area

---

## Coverage Summary

### Provider Features Tested

| Feature | Tests | Status |
| --------- | ------- | -------- |
| Instance validation | 14 | ✅ |
| API client operations | 13 | ✅ |
| Server lifecycle | 12 | ✅ |
| Pricing operations | 12 | ✅ |

### Total Coverage: 51 Tests

| Category | Tests | Coverage |
| ---------- | ------- | ---------- |
| Unit Tests | 14 | Validation functions |
| Integration Tests | 37 | API operations, lifecycle, pricing |
| **Total** | **51** | **Complete** |

---

## CI/CD Integration

### Exit Codes

- `0`: All 51 tests passed successfully
- `1`: One or more tests failed

### Integration Example

```bash
# Run before deployment
cd provisioning/extensions/providers/aws
nu tests/run_aws_tests.nu
```

---

## Performance

| Component | Duration | Performance |
| ----------- | ---------- | ------------- |
| Unit tests (14) | ~200ms | Instant |
| API tests (13) | ~300ms | Instant |
| Lifecycle tests (12) | ~300ms | Instant |
| Pricing tests (12) | ~300ms | Instant |
| **Total (51 tests)** | **~1.2s** | **< 5 seconds** |

Tests complete in subsecond times due to mock-based design.

---

## Troubleshooting

### Missing Mock File

Ensure the mock responses file exists:
```bash
provisioning/extensions/providers/aws/tests/mocks/mock_api_responses.json
```

### Nushell Version

Requires Nushell 0.109.0 or newer. Update with:
```nushell
cargo install nu
```

### File Permissions

Make test files executable:
```bash
chmod +x tests/run_aws_tests.nu
chmod +x tests/unit/test_utils.nu
chmod +x tests/integration/*.nu
```

---

## Related Components

- **Provider Interface**: `provider.nu` (31 exported functions)
- **Nushell Library**: `nulib/aws/` (8 modules, 1400+ lines)
- **Nickel Schemas**: `nickel/` (contracts, defaults, main, version)
- **Runtime Templates**: `templates/` (servers, security groups, EBS)

---

**Status**: Production Ready
**Test Coverage**: 51 tests (100% pass rate)
**Last Updated**: 2026-01-07