prvng_extensions/taskservs/infrastructure/polkadot/node/README.md

# Polkadot Node Task Service

## Overview

The Polkadot Node task service provides a complete installation and configuration of a [Polkadot](https://polkadot.network/) network node. This service supports multiple node types including full nodes, light clients, and archive nodes for the Polkadot, Kusama, and Westend networks. It's designed for both development and production environments with comprehensive configuration options.

## Features

### Node Types
- **Full Node** - Complete blockchain synchronization with state pruning
- **Light Client** - Minimal resource usage with header-only synchronization
- **Archive Node** - Complete historical data retention without pruning
- **Validator Node** - Production-ready validator configuration (see polkadot-validator service)

### Network Support
- **Polkadot** - Main Polkadot relay chain
- **Kusama** - Polkadot's canary network
- **Westend** - Test network for development
- **Custom Chains** - Support for custom chain specifications

### Consensus Support
- **GRANDPA Finality** - Byzantine fault-tolerant finality gadget
- **Aura Consensus** - Authority-based block authoring
- **ELVES Consensus** - Ethereum-Like Validation Execution System for enhanced compatibility
- **Hybrid Consensus** - Support for multiple consensus mechanisms simultaneously

### Synchronization Modes
- **Full Sync** - Downloads and validates all blocks
- **Fast Sync** - Downloads recent state and validates headers
- **Warp Sync** - Ultra-fast sync using finality proofs

### Storage & Performance
- **State Pruning** - Configurable historical state retention
- **Block Pruning** - Optional old block data removal
- **Database Cache** - Configurable memory caching for performance
- **Execution Strategies** - Optimized runtime execution modes

### Network & RPC Services
- **P2P Networking** - Peer-to-peer network participation
- **WebSocket RPC** - Real-time blockchain data access
- **HTTP RPC** - Standard REST-like API access
- **Rate Limiting** - Configurable request rate limiting
- **WSS Support** - Secure WebSocket with SSL/TLS

## Configuration

### Basic Full Node
```kcl
polkadot_node: PolkadotNode = {
    name: "polkadot-full-node"
    version: "1.5.0"
    run_user: {
        name: "polkadot"
        home: "/home/polkadot"
    }
    chain: "polkadot"
    node_type: "full"
    base_path: "/var/lib/polkadot"
    ports: {
        p2p_port: 30333
        ws_port: 9944
        http_port: 9933
    }
    sync_mode: "full"
    pruning: {
        mode: "state"
        blocks_to_keep: 256
    }
}
```

### Archive Node Configuration
```kcl
polkadot_node: PolkadotNode = {
    name: "polkadot-archive"
    version: "1.5.0"
    run_user: {
        name: "polkadot"
        group: "polkadot"
        home: "/opt/polkadot"
    }
    chain: "polkadot"
    node_type: "full"
    base_path: "/data/polkadot"
    archive_mode: true
    ports: {
        p2p_port: 30333
        ws_port: 9944
        http_port: 9933
    }
    sync_mode: "warp"
    max_peers: 50
    database_cache: 2048
    state_cache_size: 1073741824
    bootnodes: [
        "/dns/bootnode.polkadot.io/tcp/30333/p2p/12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp"
    ]
    telemetry_enabled: true
    telemetry_url: "wss://telemetry.polkadot.io/submit/ 0"
}
```

### Production Node with WSS
```kcl
polkadot_node: PolkadotNode = {
    name: "polkadot-prod-node"
    version: "1.5.0"
    run_user: {
        name: "polkadot"
        group: "polkadot"
        home: "/opt/polkadot"
    }
    chain: "polkadot"
    node_type: "full"
    base_path: "/var/lib/polkadot"
    ports: {
        p2p_port: 30333
        ws_port: 9944
        http_port: 9933
        prometheus_port: 9615
    }
    wss: {
        enabled: true
        domain: "node.company.com"
        rate_limit: 1000
    }
    ssl: {
        enabled: true
        cert_file: "/etc/ssl/certs/polkadot-node.crt"
        key_file: "/etc/ssl/private/polkadot-node.key"
    }
    rpc: {
        enabled: true
        cors: ["https://polkadot.js.org"]
        rate_limit: 2000
        max_connections: 100
        batch_config: {
            max_size: 100
            max_len: 1000000
        }
    }
    sync_mode: "warp"
    pruning: {
        mode: "state"
        blocks_to_keep: 256
    }
    max_peers: 75
    database_cache: 4096
    state_cache_size: 2147483648
    execution_strategies: {
        syncing: "NativeElseWasm"
        importing: "NativeElseWasm"
        block_construction: "Native"
        offchain_worker: "Native"
        other: "Native"
    }
    log_level: "info"
    telemetry_enabled: true
}
```

### Light Client Configuration
```kcl
polkadot_node: PolkadotNode = {
    name: "polkadot-light"
    version: "1.5.0"
    run_user: {
        name: "polkadot"
        home: "/home/polkadot"
    }
    chain: "polkadot"
    node_type: "light"
    base_path: "/var/lib/polkadot-light"
    sync_mode: "fast"
    max_peers: 25
    database_cache: 256
    state_cache_size: 268435456
    log_level: "warn"
}
```

### ELVES Consensus Node Configuration
```kcl
polkadot_node: PolkadotNode = {
    name: "polkadot-elves-node"
    version: "1.5.0"
    run_user: {
        name: "polkadot"
        group: "polkadot"
        home: "/opt/polkadot"
    }
    chain: "polkadot"
    node_type: "full"
    base_path: "/var/lib/polkadot"
    ports: {
        p2p_port: 30333
        ws_port: 9944
        http_port: 9933
        prometheus_port: 9615
    }
    consensus: {
        type: "elves"
        elves_config: {
            epoch_duration: 2400  # blocks per epoch
            validators_per_epoch: 21
            proposal_timeout: 3000
            prevote_timeout: 3000
            precommit_timeout: 3000
            commit_timeout: 1000
            ethereum_compatibility: true
        }
        finality: {
            type: "grandpa"
            grandpa_interval: 8
        }
    }
    ethereum_compatibility: {
        enabled: true
        chain_id: 1
        evm_runtime: true
    }
    sync_mode: "warp"
    max_peers: 50
    database_cache: 2048
    state_cache_size: 2147483648
    log_level: "info"
    telemetry_enabled: true
}
```

## Usage

### Deploy Polkadot Node
```bash
./core/nulib/provisioning taskserv create polkadot-node --infra <infrastructure-name>
```

### List Available Task Services
```bash
./core/nulib/provisioning taskserv list
```

### SSH to Node Server
```bash
./core/nulib/provisioning server ssh <polkadot-node-server>
```

### Service Management
```bash
# Check node status
systemctl status polkadot-node

# Start/stop node
systemctl start polkadot-node
systemctl stop polkadot-node
systemctl restart polkadot-node

# View node logs
journalctl -u polkadot-node -f

# Check sync status
curl -H "Content-Type: application/json" \
  -d '{"id":1, "jsonrpc":"2.0", "method": "system_syncState", "params":[]}' \
  http://localhost:9933/
```

### Node Monitoring
```bash
# Check node health
curl http://localhost:9933/health

# Get node information
curl -H "Content-Type: application/json" \
  -d '{"id":1, "jsonrpc":"2.0", "method": "system_name", "params":[]}' \
  http://localhost:9933/

# Check connected peers
curl -H "Content-Type: application/json" \
  -d '{"id":1, "jsonrpc":"2.0", "method": "system_peers", "params":[]}' \
  http://localhost:9933/

# Monitor prometheus metrics
curl http://localhost:9615/metrics
```

### RPC API Usage
```bash
# WebSocket connection
wscat -c ws://localhost:9944

# Get latest block
curl -H "Content-Type: application/json" \
  -d '{"id":1, "jsonrpc":"2.0", "method": "chain_getBlock", "params":[]}' \
  http://localhost:9933/

# Get chain head
curl -H "Content-Type: application/json" \
  -d '{"id":1, "jsonrpc":"2.0", "method": "chain_getHead", "params":[]}' \
  http://localhost:9933/

# ELVES Consensus Operations
# Check ELVES consensus state
curl -H "Content-Type: application/json" \
  -d '{"id":1, "jsonrpc":"2.0", "method": "elves_getConsensusState", "params":[]}' \
  http://localhost:9933/

# Get ELVES epoch information
curl -H "Content-Type: application/json" \
  -d '{"id":1, "jsonrpc":"2.0", "method": "elves_getCurrentEpoch", "params":[]}' \
  http://localhost:9933/

# Check validator set for ELVES consensus
curl -H "Content-Type: application/json" \
  -d '{"id":1, "jsonrpc":"2.0", "method": "elves_getValidators", "params":[]}' \
  http://localhost:9933/
```

## Architecture

### System Architecture
```
┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│   Applications  │────│  Polkadot Node   │────│   Polkadot      │
│                 │    │                  │    │   Network       │
│ • Polkadot.js   │    │ • P2P (30333)    │    │                 │
│ • dApps         │────│ • WS (9944)      │────│ • Relay Chain   │
│ • Wallets       │    │ • HTTP (9933)    │    │ • Parachains    │
│ • Analytics     │    │ • Prometheus     │    │ • Validators    │
└─────────────────┘    └──────────────────┘    └─────────────────┘
```

### Network Ports
- **P2P Port (30333)** - Peer-to-peer network communication
- **WebSocket Port (9944)** - Real-time RPC and subscriptions
- **HTTP Port (9933)** - Standard RPC API calls
- **Prometheus Port (9615)** - Metrics and monitoring data

### File Structure
```
/var/lib/polkadot/          # Main data directory
├── chains/                 # Chain-specific data
│   ├── polkadot/          # Polkadot chain data
│   ├── ksmcc3/            # Kusama chain data
│   └── westend2/          # Westend chain data
├── node.key               # Node identity key
└── network/               # Network state

/etc/polkadot/             # Configuration directory
├── polkadot.conf          # Main configuration
└── ssl/                   # SSL certificates

/home/polkadot/            # Service user home
├── .local/                # Local binaries
└── logs/                  # Application logs
```

## Supported Operating Systems

- Ubuntu 20.04+ / Debian 11+
- CentOS 8+ / RHEL 8+ / Fedora 35+

## System Requirements

### Light Client Requirements
- **RAM**: 2GB (4GB recommended)
- **Storage**: 10GB (minimal chain data)
- **CPU**: 1 core (2 cores recommended)
- **Network**: Standard internet connection

### Full Node Requirements
- **RAM**: 8GB (16GB recommended)
- **Storage**: 200GB SSD (500GB+ recommended)
- **CPU**: 4 cores (8 cores recommended)
- **Network**: Good bandwidth and low latency

### Archive Node Requirements
- **RAM**: 32GB+ (for optimal performance)
- **Storage**: 2TB+ NVMe SSD (grows over time)
- **CPU**: 8+ cores with high clock speed
- **Network**: High bandwidth dedicated connection

### Network Requirements
- **Internet Access** - Stable connection to Polkadot network
- **Port Access** - P2P port (30333) must be accessible
- **Firewall** - Proper firewall configuration for services
- **DNS** - Reliable DNS resolution

## Troubleshooting

### Synchronization Issues
```bash
# Check sync status
curl -H "Content-Type: application/json" \
  -d '{"id":1, "jsonrpc":"2.0", "method": "system_syncState", "params":[]}' \
  http://localhost:9933/

# Check connected peers
curl -H "Content-Type: application/json" \
  -d '{"id":1, "jsonrpc":"2.0", "method": "system_peers", "params":[]}' \
  http://localhost:9933/

# Restart with fresh sync
systemctl stop polkadot-node
rm -rf /var/lib/polkadot/chains/polkadot/db/
systemctl start polkadot-node
```

### Performance Issues
```bash
# Check system resources
htop
df -h /var/lib/polkadot
iostat -x 1

# Monitor Polkadot metrics
curl -s http://localhost:9615/metrics | grep polkadot_

# Check database size
du -sh /var/lib/polkadot/chains/polkadot/db/

# Analyze slow queries
journalctl -u polkadot-node | grep -i "slow"
```

### Network Connectivity
```bash
# Check if ports are listening
netstat -tlnp | grep -E ':(30333|9944|9933)'

# Test P2P connectivity
telnet your-server-ip 30333

# Test WebSocket
wscat -c ws://localhost:9944

# Check external connectivity
curl -4 ifconfig.co  # Get public IP
```

### RPC Issues
```bash
# Test RPC health
curl http://localhost:9933/health

# Check RPC configuration
curl -H "Content-Type: application/json" \
  -d '{"id":1, "jsonrpc":"2.0", "method": "rpc_methods", "params":[]}' \
  http://localhost:9933/

# Test WebSocket RPC
wscat -c ws://localhost:9944
# Send: {"id":1, "jsonrpc":"2.0", "method": "chain_getHead", "params":[]}
```

### Storage Issues
```bash
# Check disk space
df -h /var/lib/polkadot

# Check database integrity
sudo -u polkadot polkadot check-block --chain polkadot

# Purge old data (if pruning enabled)
sudo -u polkadot polkadot purge-chain --chain polkadot

# Database statistics
du -sh /var/lib/polkadot/chains/polkadot/db/*
```

## Security Considerations

### Node Security
- **User Isolation** - Run node as dedicated system user
- **File Permissions** - Secure node key and configuration files
- **System Updates** - Keep OS and Polkadot binary updated
- **Firewall Rules** - Limit access to necessary ports only

### RPC Security
- **Access Control** - Restrict RPC access to trusted networks
- **Rate Limiting** - Implement proper rate limiting
- **CORS Configuration** - Configure CORS for web applications
- **SSL/TLS** - Use HTTPS for production RPC endpoints

### Network Security
- **DDoS Protection** - Implement connection limits and rate limiting
- **Monitoring** - Monitor for unusual network activity
- **Peer Filtering** - Use reserved nodes for trusted connections
- **Regular Audits** - Regularly audit node configuration and logs

## Performance Optimization

### Hardware Optimization
- **Storage** - Use NVMe SSDs for best I/O performance
- **Memory** - Allocate sufficient RAM for database caching
- **CPU** - Use processors with good single-thread performance
- **Network** - Ensure low-latency, high-bandwidth connection

### Configuration Tuning
- **Database Cache** - Increase cache size based on available RAM
- **State Cache** - Configure appropriate state cache size
- **Pruning** - Use state pruning to manage disk usage
- **Execution Strategy** - Use native execution where possible

### System Tuning
- **File Descriptors** - Increase ulimit for network connections
- **TCP Configuration** - Tune TCP settings for high-throughput
- **Disk I/O** - Optimize filesystem and disk scheduler
- **Memory Management** - Configure swappiness and memory overcommit

## Backup and Recovery

### Backup Procedure
```bash
# Stop node service
systemctl stop polkadot-node

# Backup node key (critical!)
cp /var/lib/polkadot/node.key /backup/node-key-$(date +%Y%m%d).key

# Backup configuration
tar -czf /backup/polkadot-config-$(date +%Y%m%d).tar.gz \
    /etc/polkadot/ \
    /var/lib/polkadot/node.key

# For archive nodes, backup database (optional, can re-sync)
tar -czf /backup/polkadot-db-$(date +%Y%m%d).tar.gz \
    /var/lib/polkadot/chains/

# Restart service
systemctl start polkadot-node
```

### Recovery Procedure
1. **Stop node service**
2. **Restore node key** to maintain same peer identity
3. **Restore configuration** files
4. **Restore database** (optional, can re-sync from network)
5. **Verify file permissions** and ownership
6. **Start node service**
7. **Monitor synchronization** progress

## Integration Examples

### Polkadot.js Integration
```javascript
// Connect to local node
import { ApiPromise, WsProvider } from '@polkadot/api';

const provider = new WsProvider('ws://localhost:9944');
const api = await ApiPromise.create({ provider });

// Get latest block
const lastHeader = await api.rpc.chain.getHeader();
console.log(`Latest block: ${lastHeader.number}`);
```

### Monitoring with Prometheus
```yaml
# prometheus.yml
scrape_configs:
  - job_name: 'polkadot-node'
    static_configs:
      - targets: ['localhost:9615']
    scrape_interval: 30s
    metrics_path: '/metrics'
```

### Load Balancer Configuration
```nginx
upstream polkadot_rpc {
    server node1.company.com:9933;
    server node2.company.com:9933;
    server node3.company.com:9933;
}

server {
    listen 80;
    server_name api.company.com;
    
    location / {
        proxy_pass http://polkadot_rpc;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}
```

## Resources

- **Official Documentation**: [wiki.polkadot.network](https://wiki.polkadot.network)
- **GitHub Repository**: [paritytech/polkadot](https://github.com/paritytech/polkadot)
- **Polkadot.js Apps**: [polkadot.js.org/apps](https://polkadot.js.org/apps)
- **Telemetry**: [telemetry.polkadot.io](https://telemetry.polkadot.io)
- **Community**: [polkadot.network/community](https://polkadot.network/community)