Update Infrastructure Guide

Guide for safely updating existing infrastructure deployments.

Overview

This guide covers strategies and procedures for updating provisioned infrastructure, including servers, task services, and cluster configurations.

Prerequisites

Before updating infrastructure:

• ✅ Backup current configuration
• ✅ Test updates in development environment
• ✅ Review changelog and breaking changes
• ✅ Schedule maintenance window

Update Strategies

1. In-Place Update

Update existing resources without replacement:

# Check for available updates
provisioning version check

# Update specific taskserv
provisioning taskserv update kubernetes --version 1.29.0 --check

# Update all taskservs
provisioning taskserv update --all --check

Pros: Fast, no downtime Cons: Risk of service interruption

2. Rolling Update

Update resources one at a time:

# Enable rolling update strategy
provisioning config set update.strategy rolling

# Update cluster with rolling strategy
provisioning cluster update my-cluster --rolling --max-unavailable 1

Pros: No downtime, gradual rollout Cons: Slower, requires multiple nodes

3. Blue-Green Deployment

Create new infrastructure alongside old:

# Create new "green" environment
provisioning workspace create my-cluster-green

# Deploy updated infrastructure
provisioning cluster create my-cluster --workspace my-cluster-green

# Test green environment
provisioning test env cluster my-cluster-green

# Switch traffic to green
provisioning cluster switch my-cluster-green --production

# Cleanup old "blue" environment
provisioning workspace delete my-cluster-blue --confirm

Pros: Zero downtime, easy rollback Cons: Requires 2x resources temporarily

Update Procedures

Updating Task Services

# List installed taskservs with versions
provisioning taskserv list --with-versions

# Check for updates
provisioning taskserv check-updates

# Update specific service
provisioning taskserv update kubernetes \
    --version 1.29.0 \
    --backup \
    --check

# Verify update
provisioning taskserv status kubernetes

Updating Server Configuration

# Update server plan (resize)
provisioning server update web-01 \
    --plan 4xCPU-8GB \
    --check

# Update server zone (migrate)
provisioning server migrate web-01 \
    --to-zone us-west-2 \
    --check

Updating Cluster Configuration

# Update cluster configuration
provisioning cluster update my-cluster \
    --config updated-config.k \
    --backup \
    --check

# Apply configuration changes
provisioning cluster apply my-cluster

Rollback Procedures

If update fails, rollback to previous state:

# List available backups
provisioning backup list

# Rollback to specific backup
provisioning backup restore my-cluster-20251010-1200 --confirm

# Verify rollback
provisioning cluster status my-cluster

Post-Update Verification

After updating, verify system health:

# Check system status
provisioning status

# Verify all services
provisioning taskserv list --health

# Run smoke tests
provisioning test quick kubernetes
provisioning test quick postgres

# Check orchestrator
provisioning workflow orchestrator

Update Best Practices

Before Update

1. Backup everything: provisioning backup create --all
2. Review docs: Check taskserv update notes
3. Test first: Use test environment
4. Schedule window: Plan for maintenance time

During Update

1. Monitor logs: provisioning logs follow
2. Check health: provisioning health continuously
3. Verify phases: Ensure each phase completes
4. Document changes: Keep update log

After Update

1. Verify functionality: Run test suite
2. Check performance: Monitor metrics
3. Review logs: Check for errors
4. Update documentation: Record changes
5. Cleanup: Remove old backups after verification

Automated Updates

Enable automatic updates for non-critical updates:

# Configure auto-update policy
provisioning config set auto-update.enabled true
provisioning config set auto-update.strategy minor
provisioning config set auto-update.schedule "0 2 * * 0"  # Weekly Sunday 2AM

# Check auto-update status
provisioning config show auto-update

Update Notifications

Configure notifications for update events:

# Enable update notifications
provisioning config set notifications.updates.enabled true
provisioning config set notifications.updates.email "admin@example.com"

# Test notifications
provisioning test notification update-available

Troubleshooting Updates

Common Issues

Update Fails Mid-Process:

# Check update status
provisioning update status

# Resume failed update
provisioning update resume --from-checkpoint

# Or rollback
provisioning update rollback

Service Incompatibility:

# Check compatibility
provisioning taskserv compatibility kubernetes 1.29.0

# See dependency tree
provisioning taskserv dependencies kubernetes

Configuration Conflicts:

# Validate configuration
provisioning validate config

# Show configuration diff
provisioning config diff --before --after

Related Documentation

• Quick Start Guide - Initial setup
• Service Management - Service operations
• Backup & Restore - Backup procedures
• Troubleshooting - Common issues

Need Help? Run provisioning help update or see Troubleshooting Guide.