syntaxis/docs/advanced-installation.md

# Advanced Installation Guide

This guide covers advanced scenarios: migrating between presets, customizing installations, troubleshooting, and managing multiple environments.

**Table of Contents:**
- [Migrating Between Presets](#migrating-between-presets)
- [Custom Presets](#custom-presets)
- [Environment Variables](#environment-variables)
- [Troubleshooting](#troubleshooting)
- [Multi-Environment Management](#multi-environment-management)
- [CI/CD Integration](#cicd-integration)
- [Service Management](#service-management)
- [Performance Tuning](#performance-tuning)
- [Security Hardening](#security-hardening)

---

## Migrating Between Presets

### Upgrade Path: Simple → Advanced

You can upgrade your installation from a simpler preset to a more advanced one:

```bash
# Start with minimal (CLI only)
just install-minimal

# Later, upgrade to local (add TUI)
just install-local

# Then upgrade to dev (add services)
just install-dev

# Finally, deploy to staging/production
just install-staging
just install-production
```

### Migration Checklist

**Before migrating:**

```bash
# 1. Backup current configuration
mkdir -p ~/.config/syntaxis/backups/$(date +%Y-%m-%d)
cp -r ~/.config/syntaxis/* ~/.config/syntaxis/backups/$(date +%Y-%m-%d)/

# 2. Check current status
just scripts-status
just detect-provctl

# 3. Export project data if needed
syntaxis-cli export --output projects-backup.json
```

**During migration:**

```bash
# 1. Generate new configuration
just install-generate staging > /tmp/staging-config.toml

# 2. Review the configuration
cat /tmp/staging-config.toml

# 3. Perform the migration
just install-config /tmp/staging-config.toml --verbose

# 4. Verify installation
just scripts-status
just test-provisioning
```

**After migration:**

```bash
# 1. Validate services (if applicable)
just services list staging

# 2. Run health checks
curl http://localhost:3000/health  # API
curl http://localhost:8080         # Dashboard

# 3. Import project data
syntaxis-cli import --input projects-backup.json
```

### Rollback Procedure

If migration fails:

```bash
# 1. Find backup timestamp
ls ~/.config/syntaxis/backups/

# 2. Restore from backup
BACKUP_DATE="2025-11-19-10-30"
cp -r ~/.config/syntaxis/backups/$BACKUP_DATE/* ~/.config/syntaxis/

# 3. Restart services if applicable
just services stop syntaxis-api
just services start syntaxis-api

# 4. Verify rollback
just scripts-status
```

---

## Custom Presets

### Creating Custom Presets

Edit `configs/installation.toml` and add your preset:

```toml
[preset.my-custom]
name = "My Custom Setup"
description = "Custom development environment"
database_backend = "surrealdb"
surrealdb_mode = "server"
auto_start_services = true
provctl_recommended = true

[preset.my-custom.services]
syntaxis_cli = { enabled = true, auto_start = false }
syntaxis_tui = { enabled = true, auto_start = false }
syntaxis_api = { enabled = true, auto_start = true, port = 3001 }
dashboard = { enabled = true, auto_start = true, port = 8081 }
surrealdb = { enabled = true, auto_start = true, port = 8001 }
nats = { enabled = true, auto_start = true, port = 4222 }
```

### Using Custom Presets

```bash
# List your custom preset
just install-list

# Install with custom preset
just install my-custom

# Or using the script directly
nu scripts/provisioning/install-with-presets.nu --preset my-custom
```

### Preset Configuration Priority

Configuration is resolved in this order:

1. **Command-line arguments** (highest priority)
2. **Environment variables**
3. **Configuration file** (--config)
4. **Default location** (configs/installation.toml)
5. **Preset defaults** (lowest priority)

---

## Environment Variables

### Configuration via Environment

```bash
# Override database backend
export SYNTAXIS_DB_BACKEND=surrealdb
export SYNTAXIS_DB_URL=ws://localhost:8000

# Override service ports
export SYNTAXIS_API_PORT=3001
export SYNTAXIS_DASHBOARD_PORT=8081

# Control provisioning behavior
export SYNTAXIS_PROVCTL_MODE=required  # auto, required, disabled
export SYNTAXIS_PROVISIONING_VERBOSE=true

# Database credentials
export SURREALDB_USERNAME=admin
export SURREALDB_PASSWORD=secure-password

# Installation paths
export SYNTAXIS_CONFIG_DIR=~/.config/syntaxis
export SYNTAXIS_DATA_DIR=~/.local/share/syntaxis
```

### Using Environment in Installation

```bash
# Set environment variables before installation
export SYNTAXIS_DB_BACKEND=surrealdb
export SYNTAXIS_API_PORT=3001

# Then run installation
just install-dev
```

---

## Troubleshooting

### Issue: Port Already in Use

**Symptoms:**
```
Error: Address already in use (os error 48)
Failed to bind to port 3000
```

**Solution:**

```bash
# Find what's using the port
lsof -i :3000

# Option 1: Kill the process
kill -9 <PID>

# Option 2: Use different port (custom preset)
# Edit configs/installation.toml:
# [preset.custom.services]
# syntaxis_api = { port = 3001 }

# Then reinstall
just install-config custom.toml
```

### Issue: provctl Not Found

**Symptoms:**
```
✗ provctl is not available
Install provctl from: https://github.com/Akasha/provctl
```

**Solution:**

```bash
# Option 1: Install provctl
cargo install --git https://github.com/Akasha/provctl.git

# Option 2: Use without provctl (manual setup)
just install-dev --provctl disabled

# Option 3: Check if provctl is in PATH
which provctl
export PATH=$PATH:/Users/username/.cargo/bin
```

### Issue: Database Connection Failed

**Symptoms:**
```
Error connecting to database
SurrealDB connection refused
SQLite file locked
```

**Solution:**

```bash
# For SurrealDB - verify it's running
just services status surrealdb

# Start SurrealDB manually
surreal start --bind 127.0.0.1:8000 memory

# For SQLite - check file permissions
ls -la ~/.local/share/core/workspace.db
chmod 644 ~/.local/share/core/workspace.db

# Verify database configuration
cat ~/.config/syntaxis/database.toml
```

### Issue: Service Won't Start

**Symptoms:**
```
✗ Failed to start surrealdb
Service status: stopped
Health check failed
```

**Solution:**

```bash
# Check service logs
just services logs surrealdb

# Check service definition
nu scripts/provisioning/provctl-services.nu validate

# Try manual startup
surreal start --bind 127.0.0.1:8000 memory &

# Monitor health
while true; do
  curl -s http://127.0.0.1:8000 && echo "✓ Up" || echo "✗ Down"
  sleep 2
done
```

### Issue: Installation Hangs or Slow

**Symptoms:**
- Installation takes very long time
- Process seems stuck
- No output for extended period

**Solution:**

```bash
# 1. Check system resources
top -l 5 | head -30

# 2. Run in verbose mode for debugging
just install-dev --verbose

# 3. Skip certain components
# Edit custom preset to disable unnecessary services

# 4. Check network connectivity
ping github.com
curl https://github.com

# 5. Check disk space
df -h
```

### Issue: Permission Denied

**Symptoms:**
```
Permission denied: ~/.config/syntaxis
Cannot write to ~/.local/share/syntaxis
```

**Solution:**

```bash
# Check current permissions
ls -la ~/.config/syntaxis
ls -la ~/.local/share

# Fix permissions
chmod -R u+rwX ~/.config/syntaxis
chmod -R u+rwX ~/.local/share/syntaxis

# Ensure ownership
chown -R $USER:$USER ~/.config/syntaxis
chown -R $USER:$USER ~/.local/share/syntaxis
```

### Issue: Configuration Parse Error

**Symptoms:**
```
Failed to parse configuration file
Invalid TOML syntax
```

**Solution:**

```bash
# Validate TOML syntax
if command -v toml-cli &> /dev/null; then
    toml-cli validate configs/installation.toml
fi

# Or use nu to parse
nu -c "open configs/installation.toml | print"

# Check for common issues
# - Missing quotes around strings
# - Invalid TOML table syntax
# - Unclosed brackets/braces

# Restore from backup if corrupted
cp -r ~/.config/syntaxis/archives/*/configs/* ~/.config/syntaxis/
```

---

## Multi-Environment Management

### Development, Staging, Production Setup

```bash
# Create separate configurations for each environment
mkdir -p ~/.config/syntaxis/{dev,staging,prod}

# Generate configs for each
just install-generate dev > ~/.config/syntaxis/dev/install.toml
just install-generate staging > ~/.config/syntaxis/staging/install.toml
just install-generate production > ~/.config/syntaxis/prod/install.toml

# Switch between environments
function switch-env() {
    ENV=$1
    cp ~/.config/syntaxis/$ENV/install.toml configs/installation.toml
    echo "✓ Switched to $ENV environment"
}

# Usage
switch-env dev
switch-env staging
switch-env prod
```

### Environment-Specific Services

```bash
# Development: minimal services
[preset.dev-local]
database_backend = "sqlite"

# Staging: Docker services with monitoring
[preset.dev-staging]
database_backend = "surrealdb"
services.prometheus = { enabled = true }

# Production: Kubernetes
[preset.dev-prod]
database_backend = "surrealdb"
services.prometheus = { enabled = true }
services.grafana = { enabled = true }
```

---

## CI/CD Integration

### GitHub Actions

Run installation tests in CI:

```yaml
# .github/workflows/install-test.yml
name: Installation Tests
on: [push, pull_request]

jobs:
  test-presets:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: nu tests/provisioning/test-all.nu
```

### GitLab CI

```yaml
# .gitlab-ci.yml
test-installation:
  stage: test
  script:
    - apt-get update && apt-get install -y nushell
    - nu tests/provisioning/test-all.nu
```

### Local Pre-commit

```bash
#!/bin/bash
# .git/hooks/pre-commit

echo "Running provisioning tests..."
nu tests/provisioning/test-all.nu || exit 1

echo "Running provisioning validation..."
just test-provisioning || exit 1
```

---

## Service Management

### Starting Services

```bash
# Start all services for a preset
just services deploy dev

# Start specific service
just services start surrealdb
just services start nats

# Or manually
surreal start --bind 127.0.0.1:8000 memory &
nats-server &
syntaxis-api &
```

### Monitoring Services

```bash
# Check all service status
just services status

# Monitor logs
just services logs surrealdb
just services logs syntaxis-api

# Health checks
curl http://localhost:3000/health
curl http://localhost:8000  # SurrealDB
```

### Stopping Services

```bash
# Stop all services
just services stop

# Stop specific service
just services stop surrealdb
just services stop nats

# Kill hanging processes
killall surreal nats-server syntaxis-api
```

---

## Performance Tuning

### Database Optimization

**SQLite:**
```toml
[database.sqlite]
pragma_synchronous = "NORMAL"      # Faster writes
pragma_cache_size = 2000           # 2000 pages cache
wal_mode = true                    # Write-ahead logging
pragma_journal_mode = "WAL"
pragma_temp_store = "MEMORY"       # Memory temp tables
```

**SurrealDB:**
```toml
[database.surrealdb.server]
max_connections = 20               # Connection pool size
timeout_secs = 60                  # Timeout setting
```

### Service Optimization

```toml
[services.syntaxis-api]
environment = {
    RUST_LOG = "info",             # Reduce debug logs
    TOKIO_WORKER_THREADS = "4"     # Tune threads
}
```

### Memory Management

```bash
# Monitor memory usage
watch -n 1 'ps aux | grep syntaxis'

# Limit service memory (via provctl)
# Edit service definitions in configs/provisioning/services/

# Use resource limits
[services.syntaxis-api]
resources = {
    memory_limit_mb = 512
    cpu_limit = 2
}
```

---

## Security Hardening

### TLS/HTTPS Setup

```toml
[database.surrealdb.server]
tls_enabled = true
tls_ca_cert = "/path/to/ca.pem"
tls_client_cert = "/path/to/client.pem"
tls_client_key = "/path/to/client.key"
```

### Authentication

```bash
# Enable SurrealDB authentication
export SURREALDB_USERNAME=admin
export SURREALDB_PASSWORD=$(openssl rand -base64 32)

# Protect configuration files
chmod 600 ~/.config/syntaxis/database.toml
chmod 600 ~/.config/syntaxis/api/*.toml
```

### Firewall Configuration

```bash
# Restrict access to services (macOS)
sudo ipfw add allow tcp from 127.0.0.1 to 127.0.0.1 3000 in

# Or (Linux with ufw)
sudo ufw allow from 127.0.0.1 to 127.0.0.1 port 3000
```

### Environment-Specific Security

```toml
# Development: permissive
[preset.dev]
auth_enabled = false
debug_mode = true

# Production: strict
[preset.production]
auth_enabled = true
debug_mode = false
tls_required = true
```

---

## Advanced Examples

### Blue-Green Deployment

```bash
# Prepare new environment
just install-generate production > /tmp/prod-new.toml
# Edit /tmp/prod-new.toml to customize

# Deploy to new environment
just install-config /tmp/prod-new.toml

# Verify new environment
curl http://localhost:3000/health

# Switch traffic (via load balancer)
# Update DNS/routing

# Keep old environment as fallback
```

### Canary Deployment

```bash
# Deploy to canary environment (10% traffic)
just install-config canary-preset.toml

# Monitor metrics
open http://localhost:9090  # Prometheus

# Gradually increase traffic
# 10% → 25% → 50% → 100%

# Rollback if issues
just install-config stable-preset.toml
```

### Multi-Tenant Setup

```toml
[preset.multitenant]
database_backend = "surrealdb"

[preset.multitenant.database.surrealdb]
# Separate databases per tenant
namespaces = ["tenant-1", "tenant-2", "tenant-3"]

[preset.multitenant.services]
# Separate API instances per tenant
api_tenant_1 = { port = 3001 }
api_tenant_2 = { port = 3002 }
api_tenant_3 = { port = 3003 }
```

---

## Getting Help

### Diagnostic Commands

```bash
# Comprehensive diagnostic
cat << 'EOF' > diagnose.nu
print "System Information:"
print (uname -a)
print ""
print "Rust Toolchain:"
print (rustc --version)
print (cargo --version)
print ""
print "NuShell:"
print (nu --version)
print ""
print "Installed Binaries:"
syntaxis-cli --version
syntaxis-tui --version
syntaxis-api --version || true
print ""
print "Database Status:"
just services status
print ""
print "provctl Status:"
just detect-provctl --verbose
EOF

nu diagnose.nu
```

### Support Resources

- **Documentation**: `docs/INSTALLATION_CONTEXTS.md`
- **Troubleshooting**: This guide
- **Configuration**: `CLAUDE.md`
- **GitHub Issues**: https://github.com/Akasha/syntaxis/issues
- **Configuration Files**:
  - `configs/installation.toml` - Main preset definitions
  - `configs/database-*.toml` - Database configurations
  - `configs/provisioning/services/*.toml` - Service definitions

---

**Last Updated:** 2025-11-19
**Related Files:** `docs/INSTALLATION_CONTEXTS.md`, `CLAUDE.md`, `configs/installation.toml`