provisioning-platform/prov-ecosystem/crates/init-servs/INTEGRATION.md

# Init-Servs Crate - Comprehensive Integration Guide

## Overview

The Init-Servs crate provides a unified abstraction for managing services across different init systems. This comprehensive guide covers advanced integration patterns, platform-specific considerations, and best practices for cross-platform service management in production environments.

## Table of Contents

1. [Supported Init Systems](#supported-init-systems)
2. [Platform Detection Strategy](#platform-detection-strategy)
3. [Service File Generation](#service-file-generation)
4. [Configuration Patterns](#configuration-patterns)
5. [Init System Specifics](#init-system-specifics)
6. [Advanced Service Configuration](#advanced-service-configuration)
7. [Restart Policies](#restart-policies)
8. [Environment Management](#environment-management)
9. [Error Handling](#error-handling)
10. [Security Considerations](#security-considerations)
11. [Troubleshooting](#troubleshooting)

## Supported Init Systems

### systemd (Linux)

**Platforms**: Most modern Linux distributions (Ubuntu, Debian, CentOS, Fedora, etc.)

**Service Directory**: `/etc/systemd/system/` or `/usr/lib/systemd/system/`

**Key Features**:
- Unit-based service management
- Advanced restart policies
- Dependency ordering
- Resource limits
- Journal logging integration

**Service File Example**:
```ini
[Unit]
Description=My App Service
After=network-online.target

[Service]
Type=simple
User=myapp
Group=myapp
ExecStart=/usr/bin/my-app
ExecStop=/bin/kill $MAINPID
Restart=on-failure
RestartSec=10

Environment="RUST_LOG=info"
WorkingDirectory=/var/lib/myapp

[Install]
WantedBy=multi-user.target
```

**Configuration in Rust**:
```rust
use init_servs::{ServiceConfig, InitSystemType, InitConfig, InitSystem};

let config = InitConfig::new(InitSystemType::Systemd);
let init = InitSystem::new(config);

let service = ServiceConfig::new("myapp")
    .with_description("My App Service")
    .with_exec_start("/usr/bin/my-app")
    .with_user("myapp")
    .with_group("myapp")
    .with_working_directory("/var/lib/myapp")
    .with_environment("RUST_LOG", "info")
    .with_after(vec!["network-online.target".to_string()])
    .with_requires(vec!["network-online.target".to_string()]);

init.install_service(&service).await?;
```

### launchd (macOS)

**Platforms**: macOS (all versions)

**Service Directory**:
- User agents: `~/Library/LaunchAgents/`
- System daemons: `/Library/LaunchDaemons/`

**Key Features**:
- Property list (plist) format
- Launch-on-demand support
- KeepAlive policies
- Standard input/output redirection
- Environment variable support

**Service File Example**:
```xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" ...>
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.example.myapp</string>

    <key>ProgramArguments</key>
    <array>
        <string>/usr/local/bin/my-app</string>
    </array>

    <key>KeepAlive</key>
    <true/>

    <key>StandardOutPath</key>
    <string>/var/log/myapp/app.log</string>

    <key>StandardErrorPath</key>
    <string>/var/log/myapp/app.err</string>

    <key>EnvironmentVariables</key>
    <dict>
        <key>RUST_LOG</key>
        <string>info</string>
    </dict>
</dict>
</plist>
```

**Configuration in Rust**:
```rust
use init_servs::{ServiceConfig, InitSystemType, InitConfig, InitSystem};

let config = InitConfig::new(InitSystemType::Launchd);
let init = InitSystem::new(config);

let service = ServiceConfig::new("com.example.myapp")
    .with_description("My App Service")
    .with_exec_start("/usr/local/bin/my-app")
    .with_environment("RUST_LOG", "info")
    .with_working_directory("/var/lib/myapp");

init.install_service(&service).await?;
```

### OpenRC (Alpine Linux, Gentoo)

**Platforms**: Alpine Linux, Gentoo, and other distributions using OpenRC

**Service Directory**: `/etc/init.d/`

**Key Features**:
- Shell script-based services
- Dependency ordering via rc-service
- Simple configuration syntax
- Process supervision

**Service File Example**:
```bash
#!/sbin/openrc-run

description="My App Service"

command="/usr/bin/my-app"
command_user="myapp"
command_background="yes"

pidfile="/run/myapp/app.pid"
output_log="/var/log/myapp/app.log"
error_log="/var/log/myapp/app.err"

directory="/var/lib/myapp"

# Environment variables
export RUST_LOG=info

# Supervisor configuration
supervisor="supervise-daemon"
```

**Configuration in Rust**:
```rust
use init_servs::{ServiceConfig, InitSystemType, InitConfig, InitSystem};

let config = InitConfig::new(InitSystemType::OpenRC);
let init = InitSystem::new(config);

let service = ServiceConfig::new("myapp")
    .with_description("My App Service")
    .with_exec_start("/usr/bin/my-app")
    .with_user("myapp")
    .with_group("myapp")
    .with_working_directory("/var/lib/myapp")
    .with_environment("RUST_LOG", "info");

init.install_service(&service).await?;
```

### runit (Void Linux, Devuan)

**Platforms**: Void Linux, Devuan, and other runit-based systems

**Service Directory**: `/etc/sv/`

**Key Features**:
- Supervision suite for service management
- Run scripts (executable shell scripts)
- Process supervision by supervise daemon
- Signal handling

**Service File Structure**:
```
/etc/sv/myapp/
├── run           # Main service script
├── finish        # Cleanup script (optional)
├── log/
│   ├── run       # Logging script
│   └── main/     # Log directory
```

**Run Script Example**:
```bash
#!/bin/sh
exec 2>&1
exec chpst -u myapp -g myapp \
    env RUST_LOG=info \
    /usr/bin/my-app
```

**Configuration in Rust**:
```rust
use init_servs::{ServiceConfig, InitSystemType, InitConfig, InitSystem};

let config = InitConfig::new(InitSystemType::Runit);
let init = InitSystem::new(config);

let service = ServiceConfig::new("myapp")
    .with_description("My App Service")
    .with_exec_start("/usr/bin/my-app")
    .with_user("myapp")
    .with_group("myapp")
    .with_environment("RUST_LOG", "info");

init.install_service(&service).await?;
```

## Platform Detection Strategy

The init-servs crate automatically detects the appropriate init system:

### Linux Detection Priority

```
1. systemd (check: /run/systemd/system or ps output)
2. OpenRC (check: /sbin/openrc-run)
3. runit (check: /command/supervise or /sbin/runit)
4. sysvinit (fallback)
```

**Implementation**:
```rust
use init_servs::detect_init_system;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let init_type = detect_init_system().await?;
    println!("Detected: {}", init_type.name());
    // systemd, launchd, OpenRC, or runit
    Ok(())
}
```

### macOS Detection

On macOS, launchd is the only supported init system:

```rust
use init_servs::{InitSystemType, InitConfig, InitSystem};

#[cfg(target_os = "macos")]
fn create_init_system() -> InitSystem {
    let config = InitConfig::new(InitSystemType::Launchd);
    InitSystem::new(config)
}
```

### Cross-Platform Usage

```rust
use init_servs::{detect_init_system, InitSystem, InitConfig, ServiceConfig};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Auto-detect init system
    let init_type = detect_init_system().await?;

    // Create configuration for detected system
    let config = InitConfig::new(init_type);
    let init = InitSystem::new(config);

    // Define service (works across all platforms)
    let service = ServiceConfig::new("my-app")
        .with_description("My Application")
        .with_exec_start("/usr/bin/my-app")
        .with_user("appuser")
        .with_restart_policy(init_servs::RestartPolicy::OnFailure);

    // Install (generates correct format for detected init system)
    init.install_service(&service).await?;

    println!("Service installed using {}", init_type.name());
    Ok(())
}
```

## Service File Generation

### Auto-Generation from Rust Configuration

The init-servs crate automatically generates native service files:

```rust
use init_servs::{ServiceConfig, InitSystemType, InitConfig, InitSystem};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = InitConfig::new(InitSystemType::Systemd);
    let init = InitSystem::new(config);

    let service = ServiceConfig::new("tracker")
        .with_description("My App Service")
        .with_exec_start("/usr/bin/my-app")
        .with_user("myapp")
        .with_environment("RUST_LOG", "info");

    // Generates: /etc/systemd/system/tracker.service
    let generated_file = init.generate_service_file(&service).await?;
    println!("Generated: {}", generated_file);

    Ok(())
}
```

### Viewing Generated Content

```rust
use init_servs::{ServiceConfig, InitSystemType, InitConfig, InitSystem};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = InitConfig::new(InitSystemType::Systemd);
    let init = InitSystem::new(config);

    let service = ServiceConfig::new("tracker")
        .with_description("My App Service")
        .with_exec_start("/usr/bin/my-app")
        .with_user("myapp");

    // Get generated content without writing to disk
    let content = init.generate_service_content(&service)?;
    println!("{}", content);

    Ok(())
}
```

## Configuration Patterns

### Basic Service Configuration

```rust
let service = ServiceConfig::new("myservice")
    .with_description("My Service")
    .with_exec_start("/usr/bin/myservice");
```

### Configuration with All Options

```rust
let service = ServiceConfig::new("myservice")
    .with_description("My Service")
    .with_exec_start("/usr/bin/myservice")
    .with_exec_stop("/bin/kill $MAINPID")
    .with_user("myuser")
    .with_group("mygroup")
    .with_working_directory("/var/lib/myservice")
    .with_restart_policy(init_servs::RestartPolicy::OnFailure)
    .with_environment("LOG_LEVEL", "debug")
    .with_environment("CONFIG_PATH", "/etc/myservice.conf")
    .with_after(vec!["network-online.target".to_string()])
    .with_requires(vec!["postgresql.service".to_string()]);
```

### Environment-Driven Configuration

```rust
use init_servs::ServiceConfig;

fn create_service_from_env(name: &str) -> ServiceConfig {
    let mut service = ServiceConfig::new(name);

    // Load from environment
    if let Ok(exec_start) = std::env::var("EXEC_START") {
        service = service.with_exec_start(&exec_start);
    }

    if let Ok(user) = std::env::var("SERVICE_USER") {
        service = service.with_user(&user);
    }

    if let Ok(log_level) = std::env::var("LOG_LEVEL") {
        service = service.with_environment("RUST_LOG", &log_level);
    }

    service
}
```

### Template-Based Configuration

```rust
use init_servs::{ServiceConfig, ServiceTemplate};
use serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize)]
struct ServiceTemplate {
    name: String,
    description: String,
    exec_start: String,
    user: String,
    group: String,
    environment: std::collections::HashMap<String, String>,
}

fn create_service_from_template(template: ServiceTemplate) -> ServiceConfig {
    let mut service = ServiceConfig::new(&template.name)
        .with_description(&template.description)
        .with_exec_start(&template.exec_start)
        .with_user(&template.user)
        .with_group(&template.group);

    for (key, value) in template.environment {
        service = service.with_environment(&key, &value);
    }

    service
}
```

## Init System Specifics

### systemd-Specific Features

**Resource Limits**:
```rust
let service = ServiceConfig::new("tracker")
    // These map to systemd directives
    .with_exec_start("/usr/bin/my-app")
    .with_exec_stop("systemctl stop my-app");
    // Note: Full resource limit support would require extending ServiceConfig
```

**Socket Activation** (Advanced):
Create separate .socket and .service files for socket activation.

### launchd-Specific Features

**Launch-On-Demand**:
```rust
// launchd natively supports launch-on-demand
let service = ServiceConfig::new("com.example.myapp")
    .with_description("My App Service");
    // launchd will automatically start on first request
```

**Standard I/O Redirection**:
```rust
let service = ServiceConfig::new("tracker")
    .with_environment("LOG_PATH", "/var/log/tracker.log");
    // Maps to StandardOutPath in generated plist
```

### OpenRC-Specific Features

**Dependency Ordering**:
```rust
let service = ServiceConfig::new("tracker")
    .with_after(vec!["networking.service".to_string()])
    .with_requires(vec!["postgresql.service".to_string()]);
```

### runit-Specific Features

**Logging Service**:
For runit, logging is handled via a separate log/run script:
```bash
#!/bin/sh
exec svlogd -tt /var/log/tracker
```

The init-servs crate automatically creates this when needed.

## Advanced Service Configuration

### Services with Dependencies

```rust
use init_servs::{ServiceConfig, RestartPolicy};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = InitConfig::default();
    let init = InitSystem::new(config);

    // Database service
    let db_service = ServiceConfig::new("tracker-db")
        .with_description("Tracker Database")
        .with_exec_start("/usr/bin/postgresql");

    // API service depends on database
    let api_service = ServiceConfig::new("app-api")
        .with_description("App API")
        .with_exec_start("/usr/bin/my-app-api")
        .with_after(vec!["tracker-db.service".to_string()])
        .with_requires(vec!["tracker-db.service".to_string()]);

    init.install_service(&db_service).await?;
    init.install_service(&api_service).await?;

    Ok(())
}
```

### Multi-Instance Services

```rust
use init_servs::ServiceConfig;

fn create_instance_service(instance: u32) -> ServiceConfig {
    ServiceConfig::new(&format!("app-{}", instance))
        .with_description(&format!("App Instance {}", instance))
        .with_exec_start(&format!("/usr/bin/my-app --instance {}", instance))
        .with_user("myapp")
        .with_environment("INSTANCE_ID", &instance.to_string())
}
```

### Health Check Integration

```rust
let service = ServiceConfig::new("app")
    .with_description("My App Service")
    .with_exec_start("/usr/bin/my-app")
    // For systemd, add health check via ExecHealthCheck
    .with_exec_stop("/usr/bin/tracker-health-check");
```

## Restart Policies

### Policy Types

```rust
use init_servs::RestartPolicy;

// No restart on failure
let service = ServiceConfig::new("tracker")
    .with_restart_policy(RestartPolicy::No);

// Always restart
let service = ServiceConfig::new("tracker")
    .with_restart_policy(RestartPolicy::Always);

// Restart only on failure
let service = ServiceConfig::new("tracker")
    .with_restart_policy(RestartPolicy::OnFailure);

// Restart on abnormal termination
let service = ServiceConfig::new("tracker")
    .with_restart_policy(RestartPolicy::OnAbnormal);
```

### Restart Behavior by Init System

| Policy | systemd | launchd | OpenRC | runit |
|--------|---------|---------|--------|-------|
| No | `Restart=no` | Not supported | - | - |
| Always | `Restart=always` | `KeepAlive=true` | `respawn` | Automatic |
| OnFailure | `Restart=on-failure` | Limited support | `respawn` | Automatic |
| OnAbnormal | `Restart=on-abnormal` | Custom handler | - | - |

## Environment Management

### Setting Environment Variables

```rust
let service = ServiceConfig::new("tracker")
    .with_environment("RUST_LOG", "debug")
    .with_environment("TRACKER_PORT", "6969")
    .with_environment("DATABASE_URL", "sqlite:///var/lib/tracker.db")
    .with_environment("CONFIG_PATH", "/etc/tracker/config.toml");
```

### Loading from .env Files

```rust
use std::fs;

fn load_env_file(path: &str) -> std::collections::HashMap<String, String> {
    let content = fs::read_to_string(path).unwrap_or_default();
    content
        .lines()
        .filter(|line| !line.starts_with('#') && !line.is_empty())
        .map(|line| {
            let parts: Vec<&str> = line.split('=').collect();
            if parts.len() == 2 {
                (parts[0].to_string(), parts[1].to_string())
            } else {
                (String::new(), String::new())
            }
        })
        .collect()
}

let env_vars = load_env_file("/etc/tracker/.env");
let mut service = ServiceConfig::new("tracker");
for (key, value) in env_vars {
    service = service.with_environment(&key, &value);
}
```

### Secret Management Integration

```rust
use init_servs::ServiceConfig;

// Load secrets from vault or similar
async fn create_service_with_secrets(name: &str) -> ServiceConfig {
    let mut service = ServiceConfig::new(name);

    // Load from encrypt crate
    use encrypt::config::EncryptionConfig;

    let config = EncryptionConfig::default();
    // Load encrypted secrets and set as environment

    service
}
```

## Error Handling

### Common Error Scenarios

**Permission Denied**:
```rust
use init_servs::InitError;

match init.install_service(&service).await {
    Err(InitError::PermissionDenied) => {
        eprintln!("Permission denied. Run with sudo or proper permissions");
    }
    Err(e) => eprintln!("Error: {}", e),
    Ok(_) => println!("Service installed"),
}
```

**Service Already Exists**:
```rust
match init.install_service(&service).await {
    Err(InitError::ServiceExists) => {
        eprintln!("Service already exists. Remove it first: sudo systemctl stop tracker");
    }
    Err(e) => eprintln!("Error: {}", e),
    Ok(_) => println!("Service installed"),
}
```

**Init System Not Supported**:
```rust
use init_servs::detect_init_system;

match detect_init_system().await {
    Ok(init_type) => {
        println!("Using: {}", init_type.name());
    }
    Err(e) => {
        eprintln!("Could not detect init system: {}", e);
        eprintln!("Please specify manually");
    }
}
```

## Security Considerations

### User and Group Management

**Best Practices**:
```bash
# Create dedicated user for service
sudo useradd -r -s /bin/false myapp

# Create service-specific group
sudo groupadd myapp

# Assign user to group
sudo usermod -g myapp myapp
```

**In Configuration**:
```rust
let service = ServiceConfig::new("app")
    .with_user("myapp")      // Non-root user
    .with_group("myapp")     // Service-specific group
    .with_working_directory("/var/lib/myapp");  // Restricted directory
```

### File Permissions

```bash
# Set proper ownership
sudo chown -R myapp:myapp /var/lib/myapp
sudo chown -R myapp:myapp /etc/myapp

# Set restrictive permissions
sudo chmod 700 /var/lib/myapp
sudo chmod 750 /etc/myapp
sudo chmod 640 /etc/myapp/*.conf
```

### Environment Variable Security

```rust
let service = ServiceConfig::new("app")
    .with_user("myapp")  // Run as non-root
    // Don't include secrets in environment
    // Use credential files with restricted permissions instead
    .with_environment("CONFIG_PATH", "/etc/myapp/secrets.conf");
```

**Secure file permissions**:
```bash
sudo chmod 600 /etc/myapp/secrets.conf
sudo chown myapp:myapp /etc/myapp/secrets.conf
```

## Troubleshooting

### Service Won't Start

**Diagnosis**:
```bash
# systemd
sudo systemctl status tracker
sudo journalctl -u tracker -n 50

# launchd
log show --predicate 'process == "tracker"' --last 1h

# OpenRC
sudo rc-service tracker status
sudo tail -f /var/log/tracker.log

# runit
sudo sv status tracker
sudo tail -f /var/log/tracker/current
```

**Common Issues**:
1. Executable not found: Check `with_exec_start()` path
2. User doesn't exist: Create user before installing service
3. Working directory doesn't exist: Create directory first
4. Permission denied: Check file ownership and permissions

### Service Crashes Immediately

**Causes**:
1. Missing dependencies (check `with_requires()`)
2. Configuration file not found
3. Port already in use
4. Insufficient resources

**Solution**:
```bash
# Run service manually to see error
/usr/bin/my-app

# Check logs
journalctl -u app -n 100 --no-pager
```

### Service Not Restarting

**Check restart policy**:
```rust
let service = ServiceConfig::new("app")
    .with_restart_policy(init_servs::RestartPolicy::OnFailure);
```

**Verify in generated file**:
```bash
# systemd
cat /etc/systemd/system/app.service | grep Restart

# launchd
defaults read ~/Library/LaunchAgents/com.example.myapp.plist | grep -i keepalive
```

### Managing Service Lifecycle

```bash
# systemd
sudo systemctl start app
sudo systemctl stop app
sudo systemctl restart app
sudo systemctl status app
sudo systemctl enable app        # Start on boot
sudo systemctl disable app       # Don't start on boot

# launchd
launchctl load ~/Library/LaunchAgents/com.example.myapp.plist
launchctl unload ~/Library/LaunchAgents/com.example.myapp.plist
launchctl start com.example.myapp
launchctl stop com.example.myapp

# OpenRC
sudo rc-service tracker start
sudo rc-service tracker stop
sudo rc-service tracker restart
sudo rc-update add tracker          # Start on boot
sudo rc-update del tracker          # Don't start on boot

# runit
sudo sv start tracker
sudo sv stop tracker
sudo sv restart tracker
```

## Integration Patterns

### Deployment Automation

```rust
use init_servs::{InitSystem, InitConfig, ServiceConfig};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Auto-detect init system
    let config = InitConfig::default();
    let init = InitSystem::new(config);

    // Create and install services
    let services = vec![
        create_database_service(),
        create_tracker_service(),
        create_api_service(),
    ];

    for service in services {
        init.install_service(&service).await?;
    }

    println!("All services installed successfully");
    Ok(())
}
```

### Infrastructure as Code

```rust
use init_servs::ServiceConfig;
use serde::Deserialize;
use std::fs;

#[derive(Deserialize)]
struct ServiceManifest {
    services: Vec<ServiceDefinition>,
}

#[derive(Deserialize)]
struct ServiceDefinition {
    name: String,
    description: String,
    exec_start: String,
    user: String,
    group: String,
    environment: std::collections::HashMap<String, String>,
}

fn load_manifest(path: &str) -> Result<Vec<ServiceConfig>, Box<dyn std::error::Error>> {
    let content = fs::read_to_string(path)?;
    let manifest: ServiceManifest = serde_yaml::from_str(&content)?;

    Ok(manifest
        .services
        .into_iter()
        .map(|def| {
            let mut service = ServiceConfig::new(&def.name)
                .with_description(&def.description)
                .with_exec_start(&def.exec_start)
                .with_user(&def.user)
                .with_group(&def.group);

            for (key, value) in def.environment {
                service = service.with_environment(&key, &value);
            }

            service
        })
        .collect())
}
```

### Container-to-Systemd Migration

For migrating containerized services to native systemd:

```rust
use init_servs::ServiceConfig;

fn migrate_from_docker(
    image: &str,
    binary_path: &str,
) -> ServiceConfig {
    ServiceConfig::new("migrated-service")
        .with_description(&format!("Migrated from {}", image))
        .with_exec_start(binary_path)
        .with_user("appuser")
        .with_restart_policy(init_servs::RestartPolicy::OnFailure)
        // Additional configuration
}
```

## Conclusion

The Init-Servs crate provides a robust, cross-platform abstraction for service management. Proper configuration and understanding of platform-specific behaviors ensure reliable service operation across diverse environments.

For more information, see:
- [Init-Servs README.md](README.md) - API documentation
- [Examples](../../../examples/) - Complete examples
- [Integration Architecture](../../../docs/integration-architecture.md) - System integration patterns