provisioning-platform/typedialog/README.md

# Platform Services TypeDialog Configuration System

Interactive configuration system for 8 platform services using TypeDialog nickel-roundtrip workflow. Edit configurations via web interface, generate type-safe Nickel, and validate against schemas.

## Quick Start

Configure any service with a single command:

```bash
# Configure a service (opens web UI automatically)
cd platform/typedialog/scripts
./configure-orchestrator.sh    # HTTP server on 9090
./configure-control-center.sh  # HTTP server on 8080
./configure-mcp-server.sh      # MCP stdio interface
./configure-vault-service.sh   # Secrets management
./configure-catalog-registry.sh
./configure-rag.sh
./configure-ai-service.sh
./configure-provisioning-daemon.sh
```

Web interface launches on `http://localhost:8080` with interactive form pre-populated from existing configuration.

## Workflow

```
Existing Nickel Config
        ↓ (TypeDialog reads nickel_path values)
Form Pre-populated with Current Values
        ↓ (User edits in web UI)
Updated Form Data
        ↓ (Tera template renders with new values)
Generated Nickel Config
        ↓ (TypeDialog validates)
Type-checked + Validated Config
        ↓ (User can export to TOML or run service)
Service Execution
```

## Services

| Service | Port | Purpose |
|---------|------|---------|
| Orchestrator | 9090 | Workflow orchestration |
| Control Center | 8080 | Policy/RBAC management |
| MCP Server | stdio | Model Context Protocol |
| Vault Service | 8200 | Secrets management |
| Catalog Registry | 8082 | Extension management |
| RAG | (custom) | Retrieval-Augmented Generation |
| AI Service | 8082 | AI integration |
| Provisioning Daemon | (background) | Background task daemon |

## Directory Structure

```
platform/
├── schemas/                    # Nickel type definitions
│   ├── orchestrator.ncl
│   ├── control-center.ncl
│   ├── mcp-server.ncl
│   ├── vault-service.ncl
│   ├── catalog-registry.ncl
│   ├── rag.ncl
│   ├── ai-service.ncl
│   └── provisioning-daemon.ncl
│
├── defaults/                   # Solo mode default values
│   ├── orchestrator.ncl
│   ├── control-center.ncl
│   ├── mcp-server.ncl
│   ├── vault-service.ncl
│   ├── catalog-registry.ncl
│   ├── rag.ncl
│   ├── ai-service.ncl
│   └── provisioning-daemon.ncl
│
├── templates/                  # Tera templates for generation
│   ├── orchestrator.ncl.j2
│   ├── control-center.ncl.j2
│   ├── mcp-server.ncl.j2
│   ├── vault-service.ncl.j2
│   ├── catalog-registry.ncl.j2
│   ├── rag.ncl.j2
│   ├── ai-service.ncl.j2
│   └── provisioning-daemon.ncl.j2
│
└── typedialog/
    ├── forms/                  # TypeDialog form definitions
    │   ├── orchestrator-form.toml
    │   ├── control-center-form.toml
    │   ├── mcp-server-form.toml
    │   ├── vault-service-form.toml
    │   ├── catalog-registry-form.toml
    │   ├── rag-form.toml
    │   ├── ai-service-form.toml
    │   └── provisioning-daemon-form.toml
    │
    ├── scripts/                # Configuration scripts
    │   ├── configure-orchestrator.sh
    │   ├── configure-control-center.sh
    │   ├── configure-mcp-server.sh
    │   ├── configure-vault-service.sh
    │   ├── configure-catalog-registry.sh
    │   ├── configure-rag.sh
    │   ├── configure-ai-service.sh
    │   └── configure-provisioning-daemon.sh
    │
    └── README.md              # This file
```

## How It Works

### 1. Schema Definition

Each service has a **type definition** in `platform/schemas/{service}.ncl`. This defines the structure and validates configuration.

Example from `orchestrator.ncl`:

```nickel
{
  OrchestratorSettings = {
    enabled | Bool,
    name | String,
    server | ServerConfig,
    queue | QueueConfig,
    # ... other sections
  },

  ServerConfig = {
    host | String,
    port | Number,
    workers | Number,
    # ...
  },
}
```

### 2. Default Values

Each service has **solo mode defaults** in `platform/defaults/{service}.ncl`:

```nickel
{
  orchestrator | schema.OrchestratorConfig = {
    enabled = true,
    server = {
      host = "127.0.0.1",
      port = 9090,
      workers = 2,
      # ...
    },
  },
}
```

### 3. Form Definition

Each service has an **interactive form** in `platform/typedialog/forms/{service}-form.toml`:

```toml
[[elements]]
type = "text"
name = "server_host"
prompt = "Bind Host"
default = "127.0.0.1"
nickel_path = ["orchestrator", "server", "host"]

[[elements]]
type = "custom"
custom_type = "Number"
name = "server_port"
prompt = "Port"
default = 9090
nickel_path = ["orchestrator", "server", "port"]
```

Key: `nickel_path` maps form fields to Nickel structure for roundtrip workflow.

### 4. Template Rendering

Each service has a **Tera template** in `platform/templates/{service}.ncl.j2`:

```jinja2
{
  orchestrator = {
    enabled = {{ server_enabled }},
    server = {
      host = "{{ server_host }}",
      port = {{ server_port }},
      workers = {{ server_workers }},
    },
  },
}
```

TypeDialog injects form values, template renders valid Nickel.

### 5. Configuration Script

Each service has a **shell script** that orchestrates the workflow:

```bash
./configure-orchestrator.sh
```

This script:
1. Creates user config directory: `~/Library/Application Support/provisioning/platform/`
2. Initializes from defaults if missing
3. Backs up current config
4. Launches TypeDialog web backend
5. Opens browser to form
6. Validates generated Nickel
7. Shows next steps

### 6. User Configs

Generated configs stored at: `~/Library/Application Support/provisioning/platform/{service}.ncl`

Each service discovers its config automatically via `--config-dir` flag:

```bash
cargo run -p orchestrator -- --config-dir ~/Library/Application\ Support/provisioning/platform
```

## Using the System

### Configure a Service

```bash
# Navigate to scripts directory
cd platform/typedialog/scripts

# Run configuration script
./configure-orchestrator.sh

# Browser opens automatically to form
# Edit values in web UI
# Submit form
# Nickel config generated and validated
```

### View Current Configuration

```bash
cat ~/Library/Application\ Support/provisioning/platform/orchestrator.ncl
```

### Export to TOML

```bash
nickel export --format toml \
  ~/Library/Application\ Support/provisioning/platform/orchestrator.ncl \
  > orchestrator.toml
```

### Run Service

```bash
cd provisioning/platform
cargo run -p orchestrator -- --config-dir ~/Library/Application\ Support/provisioning/platform
```

### Review Backups

Automatic backups created with timestamp:

```bash
ls ~/Library/Application\ Support/provisioning/platform/orchestrator.ncl.backup.*
```

Restore from backup:

```bash
cp ~/Library/Application\ Support/provisioning/platform/orchestrator.ncl.backup.20250124_143022 \
   ~/Library/Application\ Support/provisioning/platform/orchestrator.ncl
```

## Features

### Validation

- **Schema validation**: Nickel typecheck ensures correctness
- **Runtime validation**: Service deserializes config before starting
- **Form validation**: Real-time validation in web UI

### Roundtrip Workflow

- **Pre-populated forms**: Load existing config, form shows current values
- **Incremental editing**: Change only what you need
- **Change tracking**: See what changed in diff viewer
- **Automatic backup**: Previous config preserved with timestamp

### Configuration Discovery

Services auto-discover configs via:
1. CLI flag: `--config-dir`
2. Environment: `SERVICE_CONFIG_DIR`
3. Defaults: Solo mode values

## Command Reference

### All Configuration Scripts

```bash
./configure-orchestrator.sh       # Orchestrator (port 9090)
./configure-control-center.sh     # Control Center (port 8080)
./configure-mcp-server.sh         # MCP Server (stdio)
./configure-vault-service.sh      # Vault Service (port 8200)
./configure-catalog-registry.sh # Catalog Registry (port 8082)
./configure-rag.sh                # RAG Service
./configure-ai-service.sh         # AI Service (port 8082)
./configure-provisioning-daemon.sh # Provisioning Daemon
```

### TypeDialog CLI Commands

Launch CLI backend instead of web:

```bash
typedialog nickel-roundtrip \
  --input orchestrator.ncl \
  --form orchestrator-form.toml \
  --output orchestrator.ncl \
  --ncl-template orchestrator.ncl.j2 \
  --backend cli
```

Launch TUI backend:

```bash
typedialog nickel-roundtrip \
  --input orchestrator.ncl \
  --form orchestrator-form.toml \
  --output orchestrator.ncl \
  --ncl-template orchestrator.ncl.j2 \
  --backend tui
```

### Verification Commands

```bash
# Check all schemas typecheck
for f in platform/schemas/*.ncl; do
  nickel typecheck "$f"
done

# Check all defaults validate
for f in platform/defaults/*.ncl; do
  nickel typecheck "$f"
  nickel export "$f" > /dev/null
done

# Verify generated configs
for f in ~/Library/Application\ Support/provisioning/platform/*.ncl; do
  nickel typecheck "$f"
done
```

## Architecture

### Schema-Driven Design

Configuration defined by **source of truth**: Rust struct types (not examples or documentation)

```
Rust Struct (source of truth)
    ↓
Nickel Schema (type validation)
    ↓
Default Values (solo mode)
    ↓
TypeDialog Form (user interface)
    ↓
Tera Template (code generation)
    ↓
User Config (deployed)
```

### Type Safety Throughout

- **Schemas**: Nickel types for validation
- **Templates**: Tera syntax ensures valid output
- **Forms**: Field types match Nickel types
- **Defaults**: Validated against schemas
- **User Configs**: Typecheck before service starts

### Separation of Concerns

| Layer | Purpose | Location |
|-------|---------|----------|
| Type Definition | Validation rules | `schemas/` |
| Defaults | Initial values | `defaults/` |
| Template | Code generation | `templates/` |
| Form | User interface | `typedialog/forms/` |
| Script | Workflow orchestration | `typedialog/scripts/` |

## Examples

### Configure Orchestrator (Complete Example)

```bash
# 1. Launch configuration
cd platform/typedialog/scripts
./configure-orchestrator.sh

# 2. Browser opens with form
# 3. Edit fields:
#    - Server: Change workers from 2 to 4
#    - Queue: Change max_concurrent_tasks from 2 to 8
#    - Logging: Change level from info to debug
#
# 4. Submit form
# 5. TypeDialog generates Nickel:

# Generated config at:
# ~/Library/Application Support/provisioning/platform/orchestrator.ncl

# 6. Export to TOML
nickel export --format toml \
  ~/Library/Application\ Support/provisioning/platform/orchestrator.ncl \
  > orchestrator.toml

# 7. Run service with new config
cd /Users/Akasha/project-provisioning/provisioning/platform
cargo run -p orchestrator -- \
  --config-dir ~/Library/Application\ Support/provisioning/platform
```

### Multi-Service Configuration

```bash
# Configure all services
cd platform/typedialog/scripts

./configure-orchestrator.sh
./configure-control-center.sh
./configure-vault-service.sh
./configure-catalog-registry.sh
./configure-rag.sh
./configure-ai-service.sh
./configure-mcp-server.sh
./configure-provisioning-daemon.sh

# Verify all configs
for f in ~/Library/Application\ Support/provisioning/platform/*.ncl; do
  nickel typecheck "$f"
done

# Export all to TOML
mkdir -p ~/Library/Application\ Support/provisioning/toml
for f in ~/Library/Application\ Support/provisioning/platform/*.ncl; do
  base=$(basename "$f" .ncl)
  nickel export --format toml "$f" \
    > ~/Library/Application\ Support/provisioning/toml/$base.toml
done
```

## Troubleshooting

### TypeDialog web backend won't start

```bash
# Check if port 8080 is in use
lsof -i :8080

# Use different port
PORT=9000 typedialog-web nickel-roundtrip ...
```

### Nickel typecheck fails

Check schema matches actual Nickel syntax:

```bash
nickel typecheck platform/schemas/orchestrator.ncl
nickel typecheck platform/defaults/orchestrator.ncl
```

### Form validation errors

Verify `nickel_path` in form matches Nickel structure:

```bash
# Extract field names from form
grep 'nickel_path' platform/typedialog/forms/orchestrator-form.toml

# Compare with Nickel structure
cat platform/defaults/orchestrator.ncl
```

### Service can't deserialize config

1. Verify config validates: `nickel typecheck config.ncl`
2. Export and check TOML: `nickel export --format toml config.ncl`
3. Check Rust struct deserializes: Run service with verbose logging

## Reference

### TypeDialog Roundtrip

Based on: `/Users/Akasha/Development/typedialog/examples/08-nickel-roundtrip`

Pattern:
```bash
typedialog nickel-roundtrip \
  --input config.ncl           # Current config (read values)
  --form form.toml             # Form with nickel_path mappings
  --output config.ncl          # Updated config (write values)
  --ncl-template config.ncl.j2 # Tera template for generation
```

### File Locations

- **Schemas**: `platform/schemas/{service}.ncl`
- **Defaults**: `platform/defaults/{service}.ncl`
- **Templates**: `platform/templates/{service}.ncl.j2`
- **Forms**: `platform/typedialog/forms/{service}-form.toml`
- **Scripts**: `platform/typedialog/scripts/configure-{service}.sh`
- **User Configs**: `~/Library/Application Support/provisioning/platform/{service}.ncl`

### Key Concepts

- **nickel_path**: Maps form field to Nickel structure path
- **Roundtrip**: Load → Edit → Generate → Validate cycle
- **Schema**: Type definition for validation
- **Template**: Tera template for Nickel generation
- **Form**: TypeDialog form definition with field metadata

## See Also

- [Nickel Language](https://nickel-lang.org/)
- [Tera Templates](https://tera.netlify.app/)
- [TypeDialog Documentation](https://github.com/typedialog/typedialog)
- [TypeDialog Roundtrip Example](file:///Users/Akasha/Development/typedialog/examples/08-nickel-roundtrip)