252 lines
5.3 KiB
Markdown
252 lines
5.3 KiB
Markdown
|
|
# Taskserv Validation & Testing System
|
||
|
|
|
||
|
|
**Version**: 1.0.0
|
||
|
|
**Status**: Production Ready
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Quick Start
|
||
|
|
|
||
|
|
### Validate Before Deployment
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Complete validation (recommended)
|
||
|
|
provisioning taskserv validate kubernetes
|
||
|
|
|
||
|
|
# Check mode (dry-run)
|
||
|
|
provisioning taskserv create kubernetes --check
|
||
|
|
|
||
|
|
# Sandbox testing
|
||
|
|
provisioning taskserv test kubernetes --runtime docker
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Available Commands
|
||
|
|
|
||
|
|
| Command | Description |
|
||
|
|
|---------|-------------|
|
||
|
|
| `taskserv validate <name>` | Multi-level validation (KCL, templates, scripts, dependencies) |
|
||
|
|
| `taskserv check-deps <name>` | Check dependencies against infrastructure |
|
||
|
|
| `taskserv create <name> --check` | Dry-run with preview (no actual deployment) |
|
||
|
|
| `taskserv test <name>` | Test in sandbox container |
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Validation Levels
|
||
|
|
|
||
|
|
### 1. **Static Validation**
|
||
|
|
- ✅ KCL schema syntax
|
||
|
|
- ✅ Jinja2 template syntax
|
||
|
|
- ✅ Shell script validation (shellcheck)
|
||
|
|
- ⚡ **Fast** - No infrastructure needed
|
||
|
|
|
||
|
|
### 2. **Dependency Validation**
|
||
|
|
- ✅ Required dependencies available
|
||
|
|
- ✅ Conflict detection
|
||
|
|
- ✅ Resource requirements
|
||
|
|
- ✅ Health check configuration
|
||
|
|
|
||
|
|
### 3. **Check Mode (Dry-Run)**
|
||
|
|
- ✅ All static validations
|
||
|
|
- ✅ Dependency checking
|
||
|
|
- ✅ Configuration preview
|
||
|
|
- ✅ Prerequisites verification
|
||
|
|
- 🚫 **No actual deployment**
|
||
|
|
|
||
|
|
### 4. **Sandbox Testing**
|
||
|
|
- ✅ Package prerequisites
|
||
|
|
- ✅ Configuration validity
|
||
|
|
- ✅ Script execution
|
||
|
|
- ✅ Health check simulation
|
||
|
|
- 🐳 Requires Docker/Podman
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Examples
|
||
|
|
|
||
|
|
### Complete Validation Workflow
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# 1. Static validation
|
||
|
|
provisioning taskserv validate kubernetes --level static
|
||
|
|
|
||
|
|
# 2. Check dependencies
|
||
|
|
provisioning taskserv check-deps kubernetes --infra my-project
|
||
|
|
|
||
|
|
# 3. Dry-run with preview
|
||
|
|
provisioning taskserv create kubernetes --check -v
|
||
|
|
|
||
|
|
# 4. Sandbox testing
|
||
|
|
provisioning taskserv test kubernetes --runtime docker
|
||
|
|
|
||
|
|
# 5. Deploy (after all checks pass)
|
||
|
|
provisioning taskserv create kubernetes
|
||
|
|
```
|
||
|
|
|
||
|
|
### Quick Validation
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# All validations in one command
|
||
|
|
provisioning taskserv validate kubernetes --level all -v
|
||
|
|
```
|
||
|
|
|
||
|
|
### CI/CD Integration
|
||
|
|
|
||
|
|
```yaml
|
||
|
|
# .gitlab-ci.yml
|
||
|
|
validate:
|
||
|
|
script:
|
||
|
|
- provisioning taskserv validate kubernetes --out json
|
||
|
|
- provisioning taskserv check-deps kubernetes --infra prod
|
||
|
|
|
||
|
|
test:
|
||
|
|
script:
|
||
|
|
- provisioning taskserv test kubernetes --runtime docker
|
||
|
|
|
||
|
|
deploy:
|
||
|
|
script:
|
||
|
|
- provisioning taskserv create kubernetes
|
||
|
|
only:
|
||
|
|
- main
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Module Structure
|
||
|
|
|
||
|
|
```
|
||
|
|
taskservs/
|
||
|
|
├── validate.nu # Main validation framework
|
||
|
|
├── deps_validator.nu # Dependency validation
|
||
|
|
├── check_mode.nu # Enhanced check mode
|
||
|
|
├── test.nu # Sandbox testing
|
||
|
|
├── create.nu # Taskserv creation
|
||
|
|
├── delete.nu # Taskserv deletion
|
||
|
|
├── update.nu # Taskserv updates
|
||
|
|
├── generate.nu # Configuration generation
|
||
|
|
├── handlers.nu # Core handlers
|
||
|
|
├── run.nu # Execution engine
|
||
|
|
├── utils.nu # Utilities
|
||
|
|
├── ops.nu # Operations
|
||
|
|
└── mod.nu # Module exports
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Key Features
|
||
|
|
|
||
|
|
### ✅ **Multi-Level Validation**
|
||
|
|
- Static, dependency, prerequisite, and health check validation
|
||
|
|
- Fail-fast with clear error messages
|
||
|
|
- Verbose mode for detailed output
|
||
|
|
|
||
|
|
### 🎯 **Enhanced Check Mode**
|
||
|
|
- Comprehensive dry-run before deployment
|
||
|
|
- Configuration preview
|
||
|
|
- File listing
|
||
|
|
- No SSH required in check mode
|
||
|
|
|
||
|
|
### 🐳 **Sandbox Testing**
|
||
|
|
- Isolated container environment
|
||
|
|
- Docker and Podman support
|
||
|
|
- Keep containers for debugging
|
||
|
|
- Multiple test scenarios
|
||
|
|
|
||
|
|
### 📊 **Multiple Output Formats**
|
||
|
|
- Text (default, human-readable)
|
||
|
|
- JSON (for automation)
|
||
|
|
- YAML (for configuration)
|
||
|
|
|
||
|
|
### 🔗 **Dependency Management**
|
||
|
|
- Automatic dependency detection
|
||
|
|
- Conflict resolution
|
||
|
|
- Resource requirement validation
|
||
|
|
- Health check configuration
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Dependencies
|
||
|
|
|
||
|
|
### Required
|
||
|
|
- Nushell 0.107.1+
|
||
|
|
- KCL 0.11.3+
|
||
|
|
|
||
|
|
### Optional
|
||
|
|
- `shellcheck` - Enhanced script validation
|
||
|
|
- `docker` or `podman` - Sandbox testing
|
||
|
|
- `glow` or `bat` - Better markdown rendering
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Documentation
|
||
|
|
|
||
|
|
- [Complete Validation Guide](../../../docs/user/taskserv-validation-guide.md)
|
||
|
|
- [KCL Schema Patterns](../../../.claude/kcl_idiomatic_patterns.md)
|
||
|
|
- [Taskserv Development](../../../docs/development/taskserv-development.md)
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Troubleshooting
|
||
|
|
|
||
|
|
### Validation Errors
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Check KCL syntax
|
||
|
|
kcl fmt <file>.k
|
||
|
|
|
||
|
|
# Validate dependencies manually
|
||
|
|
kcl run extensions/taskservs/<name>/kcl/dependencies.k
|
||
|
|
|
||
|
|
# Run with verbose output
|
||
|
|
provisioning taskserv validate <name> -v
|
||
|
|
```
|
||
|
|
|
||
|
|
### Sandbox Testing Issues
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Check Docker is running
|
||
|
|
docker ps
|
||
|
|
|
||
|
|
# Keep container for debugging
|
||
|
|
provisioning taskserv test <name> --keep
|
||
|
|
|
||
|
|
# Connect to container
|
||
|
|
docker exec -it taskserv-test-<name> bash
|
||
|
|
```
|
||
|
|
|
||
|
|
### shellcheck not found
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# macOS
|
||
|
|
brew install shellcheck
|
||
|
|
|
||
|
|
# Ubuntu/Debian
|
||
|
|
apt install shellcheck
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Contributing
|
||
|
|
|
||
|
|
When adding new validation checks:
|
||
|
|
|
||
|
|
1. Add validation function in appropriate module
|
||
|
|
2. Update `validate.nu` to call new check
|
||
|
|
3. Add tests in `tests/` directory
|
||
|
|
4. Update documentation
|
||
|
|
5. Add examples in this README
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Version History
|
||
|
|
|
||
|
|
| Version | Date | Changes |
|
||
|
|
|---------|------|---------|
|
||
|
|
| 1.0.0 | 2025-10-06 | Initial validation and testing system |
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
**Maintained By**: Infrastructure Team
|
||
|
|
**Status**: ✅ Production Ready
|