chore init repo and codebase
This commit is contained in:
Jesús Pérez
commit
d325dfad3a
GPG Key ID:
9F243E355E0BC939
112
.gitignore
vendored
Normal file
112
.gitignore
vendored
Normal file
@ -0,0 +1,112 @@
|
|||||||
|
.p
|
||||||
|
.claude
|
||||||
|
.vscode
|
||||||
|
.shellcheckrc
|
||||||
|
.coder
|
||||||
|
.migration
|
||||||
|
.zed
|
||||||
|
ai_demo.nu
|
||||||
|
CLAUDE.md
|
||||||
|
.cache
|
||||||
|
.coder
|
||||||
|
wrks
|
||||||
|
ROOT
|
||||||
|
OLD
|
||||||
|
# Generated by Cargo
|
||||||
|
# will have compiled files and executables
|
||||||
|
debug/
|
||||||
|
target/
|
||||||
|
# Encryption keys and related files (CRITICAL - NEVER COMMIT)
|
||||||
|
.k
|
||||||
|
.k.backup
|
||||||
|
*.k
|
||||||
|
*.key.backup
|
||||||
|
|
||||||
|
config.*.toml
|
||||||
|
config.*back
|
||||||
|
|
||||||
|
# where book is written
|
||||||
|
_book
|
||||||
|
|
||||||
|
# Remove Cargo.lock from gitignore if creating an executable, leave it for libraries
|
||||||
|
# More information here https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html
|
||||||
|
Cargo.lock
|
||||||
|
|
||||||
|
# These are backup files generated by rustfmt
|
||||||
|
**/*.rs.bk
|
||||||
|
|
||||||
|
# MSVC Windows builds of rustc generate these, which store debugging information
|
||||||
|
*.pdb
|
||||||
|
|
||||||
|
node_modules/
|
||||||
|
|
||||||
|
**/output.css
|
||||||
|
**/input.css
|
||||||
|
|
||||||
|
# Environment files
|
||||||
|
.env
|
||||||
|
.env.local
|
||||||
|
.env.production
|
||||||
|
.env.development
|
||||||
|
.env.staging
|
||||||
|
|
||||||
|
# Keep example files
|
||||||
|
!.env.example
|
||||||
|
|
||||||
|
# Configuration files (may contain sensitive data)
|
||||||
|
config.prod.toml
|
||||||
|
config.production.toml
|
||||||
|
config.local.toml
|
||||||
|
config.*.local.toml
|
||||||
|
|
||||||
|
# Keep example configuration files
|
||||||
|
!config.toml
|
||||||
|
!config.dev.toml
|
||||||
|
!config.example.toml
|
||||||
|
|
||||||
|
# Log files
|
||||||
|
logs/
|
||||||
|
*.log
|
||||||
|
|
||||||
|
# TLS certificates and keys
|
||||||
|
certs/
|
||||||
|
*.pem
|
||||||
|
*.crt
|
||||||
|
*.key
|
||||||
|
*.p12
|
||||||
|
*.pfx
|
||||||
|
|
||||||
|
# Database files
|
||||||
|
*.db
|
||||||
|
*.sqlite
|
||||||
|
*.sqlite3
|
||||||
|
|
||||||
|
# Backup files
|
||||||
|
*.bak
|
||||||
|
*.backup
|
||||||
|
*.tmp
|
||||||
|
*~
|
||||||
|
|
||||||
|
# Encryption and security related files
|
||||||
|
*.encrypted
|
||||||
|
*.enc
|
||||||
|
secrets/
|
||||||
|
private/
|
||||||
|
security/
|
||||||
|
|
||||||
|
# Configuration backups that may contain secrets
|
||||||
|
config.*.backup
|
||||||
|
config.backup.*
|
||||||
|
|
||||||
|
# OS generated files
|
||||||
|
.DS_Store
|
||||||
|
.DS_Store?
|
||||||
|
._*
|
||||||
|
.Spotlight-V100
|
||||||
|
.Trashes
|
||||||
|
ehthumbs.db
|
||||||
|
Thumbs.db
|
||||||
|
# Documentation build output
|
||||||
|
book-output/
|
||||||
|
# Generated setup report
|
||||||
|
SETUP_COMPLETE.md
|
||||||
176
README.md
Normal file
176
README.md
Normal file
@ -0,0 +1,176 @@
|
|||||||
|
<p align="center">
|
||||||
|
<img src="https://repo.jesusperez.pro/jesus/provisioning/media/branch/main/resources/provisioning_logo.svg" alt="Provisioning Logo" width="300"/>
|
||||||
|
</p>
|
||||||
|
<p align="center">
|
||||||
|
<img src="https://repo.jesusperez.pro/jesus/provisioning/media/branch/main/resources/logo-text.svg" alt="Provisioning" width="500"/>
|
||||||
|
</p>
|
||||||
|
|
||||||
|
# Provisioning Extensions
|
||||||
|
|
||||||
|
This directory contains the extensible components of the [Provisioning project](https://repo.jesusperez.pro/jesus/provisioning). Extensions provide modular, configurable infrastructure components that can be combined to create complete deployment solutions.
|
||||||
|
|
||||||
|
## Extension Types
|
||||||
|
|
||||||
|
### [Providers](providers/)
|
||||||
|
Cloud provider implementations for infrastructure provisioning:
|
||||||
|
- **AWS**: Amazon Web Services with EC2, VPC, and EBS support
|
||||||
|
- **UpCloud**: UpCloud infrastructure with backup and server grouping
|
||||||
|
- **Local**: Local development environment simulation
|
||||||
|
|
||||||
|
### [Task Services](taskservs/)
|
||||||
|
Modular infrastructure services that can be installed on servers:
|
||||||
|
- **Container Runtimes**: containerd, crio, podman, crun, youki
|
||||||
|
- **Orchestration**: kubernetes, cilium, coredns, etcd, rook-ceph
|
||||||
|
- **Development**: coder, desktop, gitea, webhook
|
||||||
|
- **Databases**: postgres, redis, external-nfs, mayastor
|
||||||
|
- **Networking**: ip-aliases, proxy, resolv, kms
|
||||||
|
- **Security**: oras, radicle
|
||||||
|
|
||||||
|
### [Clusters](clusters/)
|
||||||
|
Complete deployment configurations combining providers and task services:
|
||||||
|
- **Web**: Basic web service cluster
|
||||||
|
- **OCI Registry**: Container registry with storage and security
|
||||||
|
- **Planned**: buildkit, CI/CD pipelines, git hosting, databases
|
||||||
|
|
||||||
|
### Workflows
|
||||||
|
Core workflow templates integrated with the orchestrator:
|
||||||
|
- Server creation and management workflows
|
||||||
|
- Task service deployment workflows
|
||||||
|
- Cluster setup and configuration workflows
|
||||||
|
- Batch operations and multi-provider deployments
|
||||||
|
- Backup and recovery workflows
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
### Configuration-Driven Design
|
||||||
|
All extensions are defined using KCL schemas providing:
|
||||||
|
- Type safety and validation
|
||||||
|
- Hierarchical configuration inheritance
|
||||||
|
- Modular composition capabilities
|
||||||
|
- Provider-agnostic interfaces
|
||||||
|
|
||||||
|
### Dependency Management
|
||||||
|
Extensions support sophisticated dependency management:
|
||||||
|
- Service dependencies and ordering
|
||||||
|
- Resource requirements validation
|
||||||
|
- Health checks and monitoring
|
||||||
|
- Rollback and recovery capabilities
|
||||||
|
|
||||||
|
### Integration Points
|
||||||
|
Extensions integrate with:
|
||||||
|
- **Core Provisioning System**: Main CLI and library functions
|
||||||
|
- **Orchestrator**: High-performance Rust coordination layer
|
||||||
|
- **Workflow System**: Batch operations and automation
|
||||||
|
- **Configuration System**: KCL schema validation and templating
|
||||||
|
|
||||||
|
## Usage Patterns
|
||||||
|
|
||||||
|
### Basic Infrastructure Setup
|
||||||
|
```bash
|
||||||
|
# 1. Generate infrastructure configuration
|
||||||
|
provisioning/core/cli/provisioning generate infra --new myproject
|
||||||
|
|
||||||
|
# 2. Create servers using provider
|
||||||
|
provisioning/core/cli/provisioning server create --infra myproject
|
||||||
|
|
||||||
|
# 3. Install task services
|
||||||
|
provisioning/core/cli/provisioning taskserv create kubernetes --infra myproject
|
||||||
|
|
||||||
|
# 4. Deploy cluster services
|
||||||
|
provisioning/core/cli/provisioning cluster create web --infra myproject
|
||||||
|
```
|
||||||
|
|
||||||
|
### Batch Operations
|
||||||
|
```bash
|
||||||
|
# Multi-provider batch deployment
|
||||||
|
nu -c "use core/nulib/workflows/batch.nu *; batch submit workflows/multi_cloud.k"
|
||||||
|
|
||||||
|
# Monitor batch progress
|
||||||
|
nu -c "use core/nulib/workflows/batch.nu *; batch monitor <workflow_id>"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Workflow Management
|
||||||
|
```bash
|
||||||
|
# List running workflows
|
||||||
|
nu -c "use core/nulib/workflows/management.nu *; workflow list"
|
||||||
|
|
||||||
|
# Monitor specific workflow
|
||||||
|
nu -c "use core/nulib/workflows/management.nu *; workflow monitor <task_id>"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Extension Development
|
||||||
|
|
||||||
|
### KCL Schema Structure
|
||||||
|
Extensions use standardized KCL schema patterns:
|
||||||
|
|
||||||
|
```kcl
|
||||||
|
# Provider schema
|
||||||
|
schema ProviderName(provisioning.Storage):
|
||||||
|
# Provider-specific fields
|
||||||
|
provider_field: str
|
||||||
|
check:
|
||||||
|
len(provider_field) > 0
|
||||||
|
|
||||||
|
# Task service schema
|
||||||
|
schema TaskServiceName:
|
||||||
|
name: str = "service-name"
|
||||||
|
version: str
|
||||||
|
enabled: bool = True
|
||||||
|
# Service-specific configuration
|
||||||
|
check:
|
||||||
|
len(name) > 0
|
||||||
|
|
||||||
|
# Cluster schema
|
||||||
|
schema ClusterName:
|
||||||
|
name: str = "cluster-name"
|
||||||
|
components: [str]
|
||||||
|
# Cluster composition
|
||||||
|
check:
|
||||||
|
len(components) > 0
|
||||||
|
```
|
||||||
|
|
||||||
|
### Module Configuration
|
||||||
|
Each extension includes a `kcl.mod` file:
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[package]
|
||||||
|
name = "extension-name"
|
||||||
|
edition = "v0.11.2"
|
||||||
|
version = "0.0.1"
|
||||||
|
|
||||||
|
[dependencies]
|
||||||
|
provisioning = { path = "../../../kcl", version = "0.0.1" }
|
||||||
|
# Additional dependencies as needed
|
||||||
|
```
|
||||||
|
|
||||||
|
### Directory Structure
|
||||||
|
```
|
||||||
|
extension-name/
|
||||||
|
├── kcl/ # KCL configuration schemas
|
||||||
|
│ ├── extension-name.k # Main schema definition
|
||||||
|
│ ├── version.k # Version management (optional)
|
||||||
|
│ ├── dependencies.k # Dependencies (optional)
|
||||||
|
│ └── kcl.mod # Module configuration
|
||||||
|
├── default/ # Default configurations
|
||||||
|
├── templates/ # Jinja2 templates (optional)
|
||||||
|
└── README.md # Extension documentation
|
||||||
|
```
|
||||||
|
|
||||||
|
## Quality Assurance
|
||||||
|
|
||||||
|
### Validation Results
|
||||||
|
- **43 KCL directories** with comprehensive schema validation
|
||||||
|
- **44 kcl.mod files** with proper import structure
|
||||||
|
- **Syntax validation**: All major components pass KCL validation
|
||||||
|
- **Schema compliance**: Follows project architecture principles (PAP)
|
||||||
|
|
||||||
|
### Best Practices
|
||||||
|
- Follow project architecture principles (PAP)
|
||||||
|
- Use configuration-driven approaches
|
||||||
|
- Implement comprehensive validation rules
|
||||||
|
- Provide detailed documentation
|
||||||
|
- Include usage examples
|
||||||
|
- Support batch operations
|
||||||
|
- Enable workflow orchestration
|
||||||
|
|
||||||
|
For detailed information about specific extension types, see the documentation in each subdirectory and the main [provisioning documentation](../../docs/).
|
||||||
100
clusters/README.md
Normal file
100
clusters/README.md
Normal file
@ -0,0 +1,100 @@
|
|||||||
|
# Cluster Configurations
|
||||||
|
|
||||||
|
This directory contains cluster configuration definitions for the provisioning system. Clusters represent complete deployments that combine multiple task services and infrastructure components.
|
||||||
|
|
||||||
|
## Available Clusters
|
||||||
|
|
||||||
|
### Web Cluster (`web/`)
|
||||||
|
**Purpose**: Basic web service cluster deployment
|
||||||
|
- **Configuration**: Simple web service with configurable name validation
|
||||||
|
- **Dependencies**: None (self-contained)
|
||||||
|
- **Use Case**: Basic web applications and static sites
|
||||||
|
|
||||||
|
### OCI Registry (`oci-reg/`)
|
||||||
|
**Purpose**: Full OCI-compliant container registry deployment
|
||||||
|
- **Configuration**: Comprehensive container registry with storage, security, and networking
|
||||||
|
- **Features**:
|
||||||
|
- Multiple storage drivers with deduplication and garbage collection
|
||||||
|
- Authentication via htpasswd and TLS support
|
||||||
|
- UI interface and search capabilities
|
||||||
|
- Registry synchronization and CVE scanning
|
||||||
|
- Structured logging and monitoring
|
||||||
|
- **Dependencies**: Requires provisioning module base schemas
|
||||||
|
- **Use Case**: Private container registries for Kubernetes clusters
|
||||||
|
|
||||||
|
### Planned Clusters
|
||||||
|
- **buildkit**: Container build infrastructure
|
||||||
|
- **cdci-argocd**: GitOps CD pipeline with ArgoCD
|
||||||
|
- **cdci-tekton**: CI/CD with Tekton pipelines
|
||||||
|
- **git**: Git repository hosting
|
||||||
|
- **pod_repo**: Pod repository management
|
||||||
|
- **postgresql**: Database cluster
|
||||||
|
- **repository**: General repository services
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
### Creating a Cluster
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Using the main CLI
|
||||||
|
provisioning/core/cli/provisioning cluster create <cluster-name> <infra-name>
|
||||||
|
|
||||||
|
# Using workflow system
|
||||||
|
nu -c "use core/nulib/workflows/cluster.nu *; cluster create '<cluster-name>' '<infra-name>' --check"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Listing Clusters
|
||||||
|
|
||||||
|
```bash
|
||||||
|
provisioning/core/cli/provisioning cluster list
|
||||||
|
```
|
||||||
|
|
||||||
|
### Deleting a Cluster
|
||||||
|
|
||||||
|
```bash
|
||||||
|
nu -c "use core/nulib/workflows/cluster.nu *; cluster delete '<cluster-name>' '<infra-name>' --check"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Configuration Structure
|
||||||
|
|
||||||
|
Cluster configurations are defined using KCL schemas:
|
||||||
|
|
||||||
|
```kcl
|
||||||
|
schema ClusterName:
|
||||||
|
name: str = "cluster-name"
|
||||||
|
# Cluster-specific configuration
|
||||||
|
check:
|
||||||
|
len(name) > 0, "Cluster name cannot be empty"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Integration
|
||||||
|
|
||||||
|
Clusters integrate with:
|
||||||
|
- **Providers**: AWS, UpCloud, Local for infrastructure provisioning
|
||||||
|
- **Task Services**: Container runtimes, networking, storage, monitoring
|
||||||
|
- **Orchestrator**: Workflow management and coordination
|
||||||
|
- **Configuration System**: KCL schema validation and type safety
|
||||||
|
|
||||||
|
## Development
|
||||||
|
|
||||||
|
### Adding a New Cluster
|
||||||
|
|
||||||
|
1. Create cluster directory: `mkdir <cluster-name>`
|
||||||
|
2. Create KCL directory: `mkdir <cluster-name>/kcl`
|
||||||
|
3. Define cluster schema: `<cluster-name>/kcl/<cluster-name>.k`
|
||||||
|
4. Add module definition: `<cluster-name>/kcl/kcl.mod`
|
||||||
|
5. Update cluster registry in main provisioning module
|
||||||
|
|
||||||
|
### KCL Module Structure
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[package]
|
||||||
|
name = "cluster-name"
|
||||||
|
edition = "v0.11.2"
|
||||||
|
version = "0.0.1"
|
||||||
|
|
||||||
|
[dependencies]
|
||||||
|
provisioning = { path = "../../../kcl", version = "0.0.1" }
|
||||||
|
```
|
||||||
|
|
||||||
|
For more information, see the main [provisioning documentation](../../../docs/).
|
||||||
63
clusters/REFERENCE.md
Normal file
63
clusters/REFERENCE.md
Normal file
@ -0,0 +1,63 @@
|
|||||||
|
# Clusters Reference
|
||||||
|
|
||||||
|
This directory will reference the existing cluster implementations.
|
||||||
|
|
||||||
|
## Current Implementation Location
|
||||||
|
`/Users/Akasha/repo-cnz/src/provisioning/cluster/`
|
||||||
|
|
||||||
|
## Available Clusters
|
||||||
|
|
||||||
|
### Buildkit (`buildkit/`)
|
||||||
|
Container build system cluster configuration
|
||||||
|
|
||||||
|
### CI/CD Systems
|
||||||
|
- **ArgoCD**: GitOps continuous delivery
|
||||||
|
- **Tekton**: Cloud-native CI/CD pipelines
|
||||||
|
|
||||||
|
### Git Server (`git/`)
|
||||||
|
Git server cluster setup and configuration
|
||||||
|
|
||||||
|
### Web Services (`web/`)
|
||||||
|
Web service deployments and configurations
|
||||||
|
|
||||||
|
### OCI Registry (`oci-registry/`)
|
||||||
|
Container registry cluster setup
|
||||||
|
|
||||||
|
## Cluster Structure
|
||||||
|
Each cluster includes:
|
||||||
|
- **Configuration Files**: Cluster-specific settings
|
||||||
|
- **Service Definitions**: Required services and dependencies
|
||||||
|
- **Templates**: Kubernetes manifests and configurations
|
||||||
|
- **Scripts**: Deployment and management automation
|
||||||
|
|
||||||
|
## Integration Status
|
||||||
|
- **Current**: Fully functional in original location
|
||||||
|
- **New Structure**: Reference established
|
||||||
|
- **Migration**: Planned for future phase
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
Clusters remain fully functional via the main provisioning CLI:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# List available clusters
|
||||||
|
./core/nulib/provisioning cluster list
|
||||||
|
|
||||||
|
# Create cluster
|
||||||
|
./core/nulib/provisioning cluster create buildkit
|
||||||
|
|
||||||
|
# Delete cluster
|
||||||
|
./core/nulib/provisioning cluster delete buildkit
|
||||||
|
|
||||||
|
# Generate cluster configuration
|
||||||
|
./core/nulib/provisioning generate cluster
|
||||||
|
```
|
||||||
|
|
||||||
|
## Development
|
||||||
|
Cluster development continues in the original location with full functionality.
|
||||||
|
|
||||||
|
## Workflow Integration
|
||||||
|
Clusters integrate with the workflow system for:
|
||||||
|
- Automated deployment
|
||||||
|
- Dependency management
|
||||||
|
- State tracking
|
||||||
|
- Rollback capabilities
|
||||||
BIN
clusters/git/default/data.tar.gz
Normal file
BIN
clusters/git/default/data.tar.gz
Normal file
Binary file not shown.
20
clusters/git/default/gitconfig
Normal file
20
clusters/git/default/gitconfig
Normal file
@ -0,0 +1,20 @@
|
|||||||
|
[user]
|
||||||
|
name = DevAdm
|
||||||
|
email = devadm@cloudnative.zone
|
||||||
|
signingkey = /home/devadm/.ssh/id_cdci.pub
|
||||||
|
[filter "lfs"]
|
||||||
|
process = git-lfs filter-process
|
||||||
|
required = true
|
||||||
|
clean = git-lfs clean -- %f
|
||||||
|
smudge = git-lfs smudge -- %f
|
||||||
|
[core]
|
||||||
|
quotepath = false
|
||||||
|
[commit]
|
||||||
|
template = /home/devadm/.stCommitMsg
|
||||||
|
gpgsign = true
|
||||||
|
[branch]
|
||||||
|
autosetuprebase = always
|
||||||
|
[init]
|
||||||
|
defaultBranch = main
|
||||||
|
[gpg]
|
||||||
|
format = ssh
|
||||||
154
clusters/git/default/gitea/full_app.ini
Normal file
154
clusters/git/default/gitea/full_app.ini
Normal file
@ -0,0 +1,154 @@
|
|||||||
|
APP_NAME = Local Repo CloudNative zone
|
||||||
|
RUN_MODE = prod
|
||||||
|
RUN_USER = git
|
||||||
|
WORK_PATH = /data/gitea
|
||||||
|
|
||||||
|
[repository]
|
||||||
|
ROOT = /data/git/repositories
|
||||||
|
|
||||||
|
[repository.local]
|
||||||
|
LOCAL_COPY_PATH = /data/gitea/tmp/local-repo
|
||||||
|
|
||||||
|
[repository.upload]
|
||||||
|
TEMP_PATH = /data/gitea/uploads
|
||||||
|
|
||||||
|
[server]
|
||||||
|
PROTOCOL = http
|
||||||
|
APP_DATA_PATH = /data/gitea
|
||||||
|
SSH_DOMAIN = localrepo.cloudnative.zone
|
||||||
|
DOMAIN = localrepo.cloudnative.zone
|
||||||
|
HTTP_ADDR = 0.0.0.0
|
||||||
|
HTTP_PORT = 3000
|
||||||
|
ROOT_URL = https://localrepo.cloudnative.zone/
|
||||||
|
DISABLE_SSH = false
|
||||||
|
LFS_START_SERVER = true
|
||||||
|
shFS_MAX_FILE_SIZE = 0
|
||||||
|
LFS_LOCK_PAGING_NUM = 50
|
||||||
|
; Permission for unix socket
|
||||||
|
UNIX_SOCKET_PERMISSION = 666
|
||||||
|
START_SSH_SERVER = true
|
||||||
|
BUILTIN_SSH_SERVER_USER = git
|
||||||
|
; The network interface the builtin SSH server should listen on
|
||||||
|
; SSH_LISTEN_HOST =
|
||||||
|
; Port number to be exposed in clone URL
|
||||||
|
SSH_PORT = 2022
|
||||||
|
; The port number the builtin SSH server should listen on
|
||||||
|
SSH_LISTEN_PORT = %(SSH_PORT)s
|
||||||
|
; Root path of SSH directory, default is '~/.ssh', but you have to use '/home/git/.ssh'.
|
||||||
|
; SSH_ROOT_PATH =
|
||||||
|
SSH_ROOT_PATH = /data/git/repositories
|
||||||
|
; Gitea will create a authorized_keys file by default when it is not using the internal ssh server
|
||||||
|
; If you intend to use the AuthorizedKeysCommand functionality then you should turn this off.
|
||||||
|
SSH_CREATE_AUTHORIZED_KEYS_FILE = false
|
||||||
|
; For the built-in SSH server, choose the ciphers to support for SSH connections,
|
||||||
|
; for system SSH this setting has no effect
|
||||||
|
SSH_SERVER_CIPHERS = aes128-ctr, aes192-ctr, aes256-ctr, aes128-gcm@openssh.com, arcfour256, arcfour128
|
||||||
|
; For the built-in SSH server, choose the key exchange algorithms to support for SSH connections
|
||||||
|
; for system SSH this setting has no effect
|
||||||
|
SSH_SERVER_KEY_EXCHANGES = diffie-hellman-group1-sha1, diffie-hellman-group14-sha1, ecdh-sha2-nistp256, ecdh-sha2-nistp384, ecdh-sha2-nistp521, curve25519-sha256@libssh.org
|
||||||
|
; for system SSH this setting has no effect
|
||||||
|
SSH_SERVER_MACS = hmac-sha2-256-etm@openssh.com, hmac-sha2-256, hmac-sha1, hmac-sha1-96
|
||||||
|
; Directory to create temporary files in when testing public keys using ssh-keygen,
|
||||||
|
; default is the system temporary directory.
|
||||||
|
; SSH_KEY_TEST_PATH =
|
||||||
|
; Path to ssh-keygen, default is 'ssh-keygen' which means the shell is responsible for finding out which one to call.
|
||||||
|
SSH_KEYGEN_PATH = ssh-keygen
|
||||||
|
; Enable SSH Authorized Key Backup when rewriting all keys, default is true
|
||||||
|
SSH_BACKUP_AUTHORIZED_KEYS = true
|
||||||
|
; Enable exposure of SSH clone URL to anonymous visitors, default is false
|
||||||
|
SSH_EXPOSE_ANONYMOUS = false
|
||||||
|
; Indicate whether to check minimum key size with corresponding type
|
||||||
|
MINIMUM_KEY_SIZE_CHECK = false
|
||||||
|
; Disable CDN even in "prod" mode
|
||||||
|
DISABLE_ROUTER_LOG = false
|
||||||
|
OFFLINE_MODE = true
|
||||||
|
|
||||||
|
; Generate steps:
|
||||||
|
; $ ./gitea cert -ca=true -duration=8760h0m0s -host=myhost.example.com
|
||||||
|
;
|
||||||
|
; Or from a .pfx file exported from the Windows certificate store (do
|
||||||
|
; not forget to export the private key):
|
||||||
|
; $ openssl pkcs12 -in cert.pfx -out cert.pem -nokeys
|
||||||
|
; $ openssl pkcs12 -in cert.pfx -out key.pem -nocerts -nodes
|
||||||
|
# CERT_FILE = /data/gitea/conf/ssl/fullchain.pem
|
||||||
|
# KEY_FILE = /data/gitea/conf/ssl/privkey.pem
|
||||||
|
[database]
|
||||||
|
PATH = /data/gitea/gitea.db
|
||||||
|
DB_TYPE = postgres
|
||||||
|
HOST = db:5432
|
||||||
|
NAME = gitea
|
||||||
|
USER = gitea
|
||||||
|
PASSWD = gitea
|
||||||
|
LOG_SQL = false
|
||||||
|
SCHEMA =
|
||||||
|
SSL_MODE = disable
|
||||||
|
|
||||||
|
[indexer]
|
||||||
|
ISSUE_INDEXER_PATH = /data/gitea/indexers/issues.bleve
|
||||||
|
|
||||||
|
[session]
|
||||||
|
PROVIDER_CONFIG = /data/gitea/sessions
|
||||||
|
PROVIDER = file
|
||||||
|
|
||||||
|
[picture]
|
||||||
|
AVATAR_UPLOAD_PATH = /data/gitea/avatars
|
||||||
|
REPOSITORY_AVATAR_UPLOAD_PATH = /data/gitea/repo-avatars
|
||||||
|
|
||||||
|
[attachment]
|
||||||
|
PATH = /data/gitea/attachments
|
||||||
|
|
||||||
|
[log]
|
||||||
|
MODE = console
|
||||||
|
LEVEL = info
|
||||||
|
ROOT_PATH = /data/gitea/log
|
||||||
|
|
||||||
|
[security]
|
||||||
|
INSTALL_LOCK = false
|
||||||
|
SECRET_KEY =
|
||||||
|
REVERSE_PROXY_LIMIT = 1
|
||||||
|
REVERSE_PROXY_TRUSTED_PROXIES = *
|
||||||
|
PASSWORD_HASH_ALGO = pbkdf2
|
||||||
|
|
||||||
|
[service]
|
||||||
|
DISABLE_REGISTRATION = false
|
||||||
|
REQUIRE_SIGNIN_VIEW = false
|
||||||
|
REGISTER_EMAIL_CONFIRM = false
|
||||||
|
ENABLE_NOTIFY_MAIL = false
|
||||||
|
ALLOW_ONLY_EXTERNAL_REGISTRATION = false
|
||||||
|
ENABLE_CAPTCHA = false
|
||||||
|
DEFAULT_KEEP_EMAIL_PRIVATE = false
|
||||||
|
DEFAULT_ALLOW_CREATE_ORGANIZATION = true
|
||||||
|
DEFAULT_ENABLE_TIMETRACKING = true
|
||||||
|
NO_REPLY_ADDRESS = noreply.localrepo.cloudnative.zone
|
||||||
|
|
||||||
|
[lfs]
|
||||||
|
PATH = /data/git/lfs
|
||||||
|
|
||||||
|
[mailer]
|
||||||
|
ENABLED = false
|
||||||
|
|
||||||
|
[openid]
|
||||||
|
ENABLE_OPENID_SIGNIN = true
|
||||||
|
ENABLE_OPENID_SIGNUP = true
|
||||||
|
|
||||||
|
[cron.update_checker]
|
||||||
|
ENABLED = false
|
||||||
|
|
||||||
|
[repository.pull-request]
|
||||||
|
DEFAULT_MERGE_STYLE = merge
|
||||||
|
|
||||||
|
[repository.signing]
|
||||||
|
DEFAULT_TRUST_MODEL = committer
|
||||||
|
|
||||||
|
[oauth2]
|
||||||
|
|
||||||
|
[webhook]
|
||||||
|
; Hook task queue length, increase if webhook shooting starts hanging
|
||||||
|
QUEUE_LENGTH = 1000
|
||||||
|
; Deliver timeout in seconds
|
||||||
|
DELIVER_TIMEOUT =
|
||||||
|
; Allow insecure certification
|
||||||
|
SKIP_TLS_VERIFY = false
|
||||||
|
; Number of history information in each page
|
||||||
|
PAGING_NUM = 10
|
||||||
|
ALLOWED_HOST_LIST = 10.11.1.0/24
|
||||||
18
clusters/git/default/gitea/patch-app-ini.sh
Executable file
18
clusters/git/default/gitea/patch-app-ini.sh
Executable file
@ -0,0 +1,18 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Info: Script to patch Gita app.ini after init
|
||||||
|
# Author: JesusPerezLorenzo
|
||||||
|
# Release: 1.0
|
||||||
|
# Date: 19-11-2023
|
||||||
|
|
||||||
|
ROOT_DATA=${ROOT_DATA:-/data}
|
||||||
|
DATA_REPO=${DATA_REPO:-$ROOT_DATA/repo}
|
||||||
|
|
||||||
|
[ ! -r "$DATA_REPO/gitea/conf/app.ini" ] && echo "Error: app.ini not found " && exit 1
|
||||||
|
|
||||||
|
[ ! -r "gitea/webhook_app.ini" ] && echo "Error: no gitea/webhook_api.ini" && exit 1
|
||||||
|
|
||||||
|
if ! grep -q "\[webhook\]" "$DATA_REPO/gitea/conf/app.ini" ; then
|
||||||
|
cat gitea/webhook_app.ini >> "$DATA_REPO/gitea/conf/app.ini"
|
||||||
|
sudo systemctl restart pod-repo.service
|
||||||
|
fi
|
||||||
|
|
||||||
11
clusters/git/default/gitea/webhook_app.ini
Normal file
11
clusters/git/default/gitea/webhook_app.ini
Normal file
@ -0,0 +1,11 @@
|
|||||||
|
|
||||||
|
[webhook]
|
||||||
|
; Hook task queue length, increase if webhook shooting starts hanging
|
||||||
|
QUEUE_LENGTH = 1000
|
||||||
|
; Deliver timeout in seconds
|
||||||
|
DELIVER_TIMEOUT =
|
||||||
|
; Allow insecure certification
|
||||||
|
SKIP_TLS_VERIFY = false
|
||||||
|
; Number of history information in each page
|
||||||
|
PAGING_NUM = 10
|
||||||
|
ALLOWED_HOST_LIST = 10.11.1.0/24
|
||||||
95
clusters/git/default/install-git.sh
Normal file
95
clusters/git/default/install-git.sh
Normal file
@ -0,0 +1,95 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Info: Script to install/create service pod_repo
|
||||||
|
# Author: JesusPerezLorenzo
|
||||||
|
# Release: 1.0
|
||||||
|
# Date: 19-11-2023
|
||||||
|
|
||||||
|
ROOT_DATA=${ROOT_DATA:-/data}
|
||||||
|
DATA_REPO=${DATA_REPO:-$ROOT_DATA/repo}
|
||||||
|
DATA_DOC=${DATA_DOC:-$ROOT_DATA/doc}
|
||||||
|
DATA_DBS=${DATA_DBS:-$ROOT_DATA/dbs}
|
||||||
|
DATA_WEBHOOKS=${DATA_WEBHOOKS:-$ROOT_DATA/webhooks}
|
||||||
|
|
||||||
|
ROOT_SOURCE=$(dirname "$0")
|
||||||
|
|
||||||
|
exit 1
|
||||||
|
sudo mkdir -p $ROOT_DATA
|
||||||
|
sudo chown -R $(id -u):$(id -g) $ROOT_DATA
|
||||||
|
|
||||||
|
if [ ! -r "env" ] ; then
|
||||||
|
echo "# Env settings " >env
|
||||||
|
echo "DATA_REPO=$DATA_REPO" >>env
|
||||||
|
echo "DATA_DOC=$DATA_DOC" >>env
|
||||||
|
echo "DATA_DBS=$DATA_DBS" >>env
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ ! -d "$DATA_REPO" ] && [ -r "$ROOT_SOURCE/data.tar.gz" ] ; then
|
||||||
|
sudo tar -C / -xzf "$ROOT_SOURCE/data.tar.gz" && echo "Data Services installed !"
|
||||||
|
else
|
||||||
|
sudo mkdir -p $DATA_REPO/gitea/conf
|
||||||
|
sudo mkdir -p $DATA_DOC
|
||||||
|
sudo mkdir -p $DATA_DBS
|
||||||
|
fi
|
||||||
|
|
||||||
|
hostname=$(hostname -s)
|
||||||
|
id=$(id -u)
|
||||||
|
|
||||||
|
if [ -r "gitconfig" ] ; then
|
||||||
|
[ ! -r "$HOME/.gitconfig" ] && cp gitconfig "$HOME/.gitconfig"
|
||||||
|
[ -d "/home/devadm" ] && [ ! -r "/home/devadm/.gitconfig" ] && sudo cp gitconfig "/home/devadm/.gitconfig" && sudo chown devadm "/home/devadm/.gitconfig"
|
||||||
|
fi
|
||||||
|
|
||||||
|
[ ! -d "/dao/$hostname/services/pod_repo" ] && sudo mkdir -p "/dao/$hostname/services/pod_repo"
|
||||||
|
|
||||||
|
sudo chown -R $id /dao
|
||||||
|
|
||||||
|
cp -pr * "/dao/$hostname/services/pod_repo"
|
||||||
|
|
||||||
|
cd "/dao/$hostname/services/pod_repo" || exit 1
|
||||||
|
|
||||||
|
if [ -r "gitea/full_app.ini" ] && [ ! -r "$DATA_REPO/gitea/conf/app.ini" ] ; then
|
||||||
|
cp gitea/full_app.ini "$DATA_REPO/gitea/conf/app.ini"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ ! -r "app.ini" ] ; then
|
||||||
|
ln -s $DATA_REPO/gitea/conf/app.ini .
|
||||||
|
fi
|
||||||
|
|
||||||
|
# [ -r "bin/apply.sh" ] && ./bin/apply.sh
|
||||||
|
|
||||||
|
# Add systemd service
|
||||||
|
sudo cp pod-repo.service /lib/systemd/system
|
||||||
|
sudo systemctl daemon-reload
|
||||||
|
sudo systemctl enable pod-repo.service
|
||||||
|
sudo systemctl restart pod-repo.service
|
||||||
|
|
||||||
|
if [ -r 'ddeploy_docker-compose.yml' ] ; then
|
||||||
|
mv deploy_docker-compose.yml docker-compose.yml
|
||||||
|
val_timeout=10
|
||||||
|
wait=10
|
||||||
|
echo -n "Waiting services to come up ... "
|
||||||
|
while [ -z "$nc_port" ]
|
||||||
|
do
|
||||||
|
if nc -zv -w 1 "10.11.1.10" 3000 >/dev/null 2>/dev/null ; then
|
||||||
|
nc_port=1
|
||||||
|
fi
|
||||||
|
if [ -z "$nc_port" ] ; then
|
||||||
|
sleep "$wait"
|
||||||
|
num=$((num + wait))
|
||||||
|
[ "$val_timeout" -gt 0 ] && [ "$num" -gt "$val_timeout" ] && break
|
||||||
|
echo -n "$num "
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
echo ""
|
||||||
|
[ -r "gitea/full_app.ini" ] && cp gitea/full_app.ini "$DATA_REPO/gitea/conf/app.ini"
|
||||||
|
sudo systemctl restart pod-repo.service
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Fix /etc/hosts for repo operations
|
||||||
|
sudo sed -i /^10.11.1.10/d /etc/hosts
|
||||||
|
sudo sed -i "s/$hostname/$hostname.pub/g" /etc/hosts
|
||||||
|
echo "10.11.1.10 $hostname localrepo.cloudnative.zone" | sudo tee -a /etc/hosts
|
||||||
|
|
||||||
|
|
||||||
|
exit 0
|
||||||
|
|
||||||
56
clusters/git/default/nginx.conf
Normal file
56
clusters/git/default/nginx.conf
Normal file
@ -0,0 +1,56 @@
|
|||||||
|
worker_processes 1;
|
||||||
|
user root root;
|
||||||
|
|
||||||
|
events { worker_connections 1024; }
|
||||||
|
http {
|
||||||
|
|
||||||
|
sendfile on;
|
||||||
|
|
||||||
|
upstream gitea {
|
||||||
|
server basecamp-0:3000;
|
||||||
|
}
|
||||||
|
|
||||||
|
server {
|
||||||
|
#listen 80;
|
||||||
|
#server_name basecamp-0;
|
||||||
|
listen 443 ssl;
|
||||||
|
listen [::]:443 ssl;
|
||||||
|
http2 on;
|
||||||
|
server_name localrepo.cloudnative.zone
|
||||||
|
charset utf-8;
|
||||||
|
client_max_body_size 300m;
|
||||||
|
# Paths to certificate files.
|
||||||
|
ssl_certificate /etc/ssl-dom/fullchain.pem;
|
||||||
|
ssl_certificate_key /etc/ssl-dom/privkey.pem;
|
||||||
|
# File to be used as index
|
||||||
|
index index.html;
|
||||||
|
|
||||||
|
# Overrides logs defined in nginx.conf, allows per site logs.
|
||||||
|
# error_log /dev/stdout warn;
|
||||||
|
#access_log /dev/stdout main;
|
||||||
|
|
||||||
|
location / {
|
||||||
|
proxy_pass http://gitea/;
|
||||||
|
|
||||||
|
proxy_redirect off;
|
||||||
|
proxy_set_header Host $host:$server_port;
|
||||||
|
proxy_set_header X-Forwarded-Proto $scheme;
|
||||||
|
proxy_set_header X-Real-IP $remote_addr;
|
||||||
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||||
|
proxy_set_header X-NginX-Proxy true;
|
||||||
|
proxy_set_header Referer $http_referer;
|
||||||
|
proxy_http_version 1.1;
|
||||||
|
proxy_hide_header X-Powered-By;
|
||||||
|
}
|
||||||
|
|
||||||
|
location /doc/ {
|
||||||
|
autoindex on;
|
||||||
|
alias /doc/;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
server {
|
||||||
|
listen 80;
|
||||||
|
listen [::]:80;
|
||||||
|
return 301 https://$host$request_uri;
|
||||||
|
}
|
||||||
|
}
|
||||||
4
clusters/kcl.mod
Normal file
4
clusters/kcl.mod
Normal file
@ -0,0 +1,4 @@
|
|||||||
|
[package]
|
||||||
|
name = "clusters"
|
||||||
|
edition = "v0.11.3"
|
||||||
|
version = "0.0.1"
|
||||||
0
clusters/kcl.mod.lock
Normal file
0
clusters/kcl.mod.lock
Normal file
12
clusters/oci-reg/default/env-oci-reg.j2
Normal file
12
clusters/oci-reg/default/env-oci-reg.j2
Normal file
@ -0,0 +1,12 @@
|
|||||||
|
{%- if service.name == "oci-reg" %}
|
||||||
|
VERSION="{{service.version}}"
|
||||||
|
OCI_DATA="{{service.oci_data}}"
|
||||||
|
OCI_ETC="{{service.oci_etc}}"
|
||||||
|
OCI_LOG="{{service.oci_log}}"
|
||||||
|
OCI_USER="{{service.oci_user}}"
|
||||||
|
OCI_USER_GROUP="{{service.oci_user_group}}"
|
||||||
|
OCI_CMDS="{{service.oci_cmds}}"
|
||||||
|
OCI_BIN_PATH="{{service.oci_bin_path}}"
|
||||||
|
PROVISIONING_MAIN_NAME="{{main_name}}"
|
||||||
|
SERVICES_SAVE_PATH="{{services_save_path}}"
|
||||||
|
{%- endif %}
|
||||||
29
clusters/oci-reg/default/install-oci-reg.sh
Normal file
29
clusters/oci-reg/default/install-oci-reg.sh
Normal file
@ -0,0 +1,29 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
[ -r "env-oci-reg" ] && . ./env-oci-reg
|
||||||
|
|
||||||
|
[ -f "bin/apply.sh" ] && chmod +x bin/apply.sh
|
||||||
|
[ -f "make_istio-system_secret.sh" ] && chmod +x make_istio-system_secret.sh
|
||||||
|
|
||||||
|
if [ -f "install-reg.sh" ] ; then
|
||||||
|
chmod +x install-reg.sh
|
||||||
|
./install-reg.sh
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ -n "$SERVICES_SAVE_PATH" ] ; then
|
||||||
|
sudo mkdir -p "$SERVICES_SAVE_PATH/oci-reg"
|
||||||
|
for it in ./*
|
||||||
|
do
|
||||||
|
if [ -d "$it" ] ; then
|
||||||
|
sudo cp -pr "$it" "$SERVICES_SAVE_PATH/oci-reg" && rm -rf "$it"
|
||||||
|
elif [ -f "$it" ] ; then
|
||||||
|
sudo mv "$it" "$SERVICES_SAVE_PATH/oci-reg"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
sudo rm -f "$SERVICES_SAVE_PATH/oci-reg/$(basename "$0")"
|
||||||
|
sudo rm -f "$SERVICES_SAVE_PATH/oci-reg/env-oci-reg"
|
||||||
|
sudo chown -R devadm "$SERVICES_SAVE_PATH/oci-reg"
|
||||||
|
echo "service saved in $SERVICES_SAVE_PATH/oci-reg"
|
||||||
|
fi
|
||||||
|
|
||||||
|
#exit 0
|
||||||
9
clusters/oci-reg/default/install-reg.sh
Normal file
9
clusters/oci-reg/default/install-reg.sh
Normal file
@ -0,0 +1,9 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
kubectl apply -f ns
|
||||||
|
kubectl apply -f volumes
|
||||||
|
|
||||||
|
[ -r "bin/apply.sh" ] && ./bin/apply.sh
|
||||||
|
|
||||||
|
exit 0
|
||||||
|
|
||||||
74
clusters/oci-reg/default/prepare
Executable file
74
clusters/oci-reg/default/prepare
Executable file
@ -0,0 +1,74 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Info: Prepare for oci-reg installation
|
||||||
|
# Author: JesusPerezLorenzo
|
||||||
|
# Release: 1.0.2
|
||||||
|
# Date: 15-01-2024
|
||||||
|
|
||||||
|
set +o errexit
|
||||||
|
set +o pipefail
|
||||||
|
|
||||||
|
SETTINGS_FILE=$1
|
||||||
|
SERVICE_NAME=$2
|
||||||
|
SERVICE_POS=$3
|
||||||
|
#SETTINGS_ROOT=$4
|
||||||
|
RUN_ROOT=$(dirname "$0")
|
||||||
|
#ORG=$(pwd)
|
||||||
|
|
||||||
|
[ -z "$SETTINGS_FILE" ] && [ -z "$SERVICE_NAME" ] && [ -z "$SERVICE_POS" ] && exit 0
|
||||||
|
|
||||||
|
YQ=$(type -P yq)
|
||||||
|
JQ=$(type -P jq)
|
||||||
|
[ -z "$YQ" ] && echo "yq not installed " && exit 1
|
||||||
|
[ -z "$JQ" ] && echo "jq not installed " && exit 1
|
||||||
|
|
||||||
|
_fix_name_in_files() {
|
||||||
|
local source=$1
|
||||||
|
local name_in_file=$2
|
||||||
|
local new_name
|
||||||
|
for item in "$source"/*
|
||||||
|
do
|
||||||
|
if [ -d "$item" ] ; then
|
||||||
|
_fix_name_in_files "$item" "$name_in_file"
|
||||||
|
elif [ -r "$item" ] ; then
|
||||||
|
new_name=$(basename "$item" | sed "s,deploy,$name_in_file,g")
|
||||||
|
#[ -r "$(dirname "$item")/$new_name" ] && rm -f "$item"
|
||||||
|
[ -r "$item" ] && [ "$(basename "$item")" != "$new_name" ] && mv "$item" "$(dirname "$item")/$new_name"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
[ -r "$RUN_ROOT/env-oci-reg" ] && . "$RUN_ROOT"/env-oci-reg
|
||||||
|
|
||||||
|
[ -z "$PROVISIONING" ] && echo "PROVISIONING not found in environment" && exit 1
|
||||||
|
|
||||||
|
. "$PROVISIONING"/core/lib/sops
|
||||||
|
|
||||||
|
if $YQ e -o=json '.service.config' < "$SETTINGS_FILE" | tee "$RUN_ROOT/config.json" >/dev/null; then
|
||||||
|
echo "zot config.json generated !"
|
||||||
|
else
|
||||||
|
echo "Error: zot config.json generation !"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
prxy=$($YQ -er '.k8s_deploy.prxy' < "$SETTINGS_FILE" 2>/dev/null | sed 's/ //g' | sed 's/null//g')
|
||||||
|
case "$prxy" in
|
||||||
|
istio) ;;
|
||||||
|
*) [ -f "$RUN_ROOT/make_istio-system_secret.sh.j2" ] && rm -f "$RUN_ROOT/make_istio-system_secret.sh.j2"
|
||||||
|
esac
|
||||||
|
name_in_files=$($YQ -er '.k8s_deploy.name_in_files' < "$SETTINGS_FILE" 2>/dev/null | sed 's/ //g' | sed 's/null//g')
|
||||||
|
[ -n "$name_in_files" ] && _fix_name_in_files "$RUN_ROOT" "$name_in_files"
|
||||||
|
|
||||||
|
if [ -r "$RUN_ROOT/configMap-etc.yaml.j2" ] ; then
|
||||||
|
if [ -r "$RUN_ROOT/htpasswd" ] ; then
|
||||||
|
echo " htpasswd: | " >> "$RUN_ROOT/configMap-etc.yaml.j2"
|
||||||
|
sed 's,^, ,g' <"$RUN_ROOT/htpasswd" >> "$RUN_ROOT/configMap-etc.yaml.j2"
|
||||||
|
rm -f "$RUN_ROOT/htpasswd"
|
||||||
|
echo "htpasswd added to configMap-etc.yaml"
|
||||||
|
fi
|
||||||
|
if [ -r "$RUN_ROOT/config.json" ] ; then
|
||||||
|
echo " config.json: | " >> "$RUN_ROOT/configMap-etc.yaml.j2"
|
||||||
|
sed 's,^, ,g' <"$RUN_ROOT/config.json" >> "$RUN_ROOT/configMap-etc.yaml.j2"
|
||||||
|
rm -f "$RUN_ROOT/config.json"
|
||||||
|
echo "zot config.json added to configMap-etc.yaml"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
echo "Prepare $SERVICE_NAME $SERVICE_POS Done !"
|
||||||
8
clusters/oci-reg/kcl/kcl.mod
Normal file
8
clusters/oci-reg/kcl/kcl.mod
Normal file
@ -0,0 +1,8 @@
|
|||||||
|
[package]
|
||||||
|
name = "oci-reg"
|
||||||
|
edition = "v0.11.3"
|
||||||
|
version = "0.0.1"
|
||||||
|
|
||||||
|
[dependencies]
|
||||||
|
clusters = { path = "../..", version = "0.0.1" }
|
||||||
|
provisioning = { path = "../../../kcl", version = "0.0.1" }
|
||||||
9
clusters/oci-reg/kcl/kcl.mod.lock
Normal file
9
clusters/oci-reg/kcl/kcl.mod.lock
Normal file
@ -0,0 +1,9 @@
|
|||||||
|
[dependencies]
|
||||||
|
[dependencies.clusters]
|
||||||
|
name = "clusters"
|
||||||
|
full_name = "clusters_0.0.1"
|
||||||
|
version = "0.0.1"
|
||||||
|
[dependencies.provisioning]
|
||||||
|
name = "provisioning"
|
||||||
|
full_name = "vPkg_4841e690-6459-4a46-a3b3-7c77eebfb804_0.0.1"
|
||||||
|
version = "0.0.1"
|
||||||
12
clusters/pod_repo/default/bin/apply.sh
Executable file
12
clusters/pod_repo/default/bin/apply.sh
Executable file
@ -0,0 +1,12 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
#
|
||||||
|
TASK=${1:-up}
|
||||||
|
|
||||||
|
[ -r "docker-compose.yml" ] && [ "$TASK" == "up" ] && ARGS="-d"
|
||||||
|
|
||||||
|
ROOT_PATH=$(dirname "$0")
|
||||||
|
|
||||||
|
[ -r "$ROOT_PATH/../env" ] && . "$ROOT_PATH"/../env
|
||||||
|
|
||||||
|
sudo podman-compose $TASK $ARGS
|
||||||
|
|
||||||
29
clusters/pod_repo/default/install-pod_repo.sh
Normal file
29
clusters/pod_repo/default/install-pod_repo.sh
Normal file
@ -0,0 +1,29 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
ROOT_DATA=${ROOT_DATA:-/data}
|
||||||
|
DATA_REPO=${DATA_REPO:-$ROOT_DATA/repo}
|
||||||
|
DATA_DOC=${DATA_DOC:-$ROOT_DATA/doc}
|
||||||
|
DATA_DBS=${DATA_DBS:-$ROOT_DATA/dbs}
|
||||||
|
DATA_WEBHOOKS=${DATA_WEBHOOKS:-$ROOT_DATA/webhooks}
|
||||||
|
|
||||||
|
sudo mkdir -p $ROOT_DATA
|
||||||
|
sudo chown -R $(id -u):$(id -g) $ROOT_DATA
|
||||||
|
|
||||||
|
if [ ! -r ".env" ] ; then
|
||||||
|
echo "# Env settings " >.env
|
||||||
|
# Set your data directory, this is where gitea save files
|
||||||
|
echo "GITEA_DATA_DIR=$DATA_REPO" >>.env
|
||||||
|
|
||||||
|
echo "DOC_DIR=$DATA_DOC" >>.env
|
||||||
|
echo "DBS_DIR=$DATA_DBS" >>.env
|
||||||
|
echo "WEBHOOKS_DIR=$DATA_WEBHOOKS" >>.env
|
||||||
|
fi
|
||||||
|
|
||||||
|
sudo mkdir -p $GITEA_DATA_DIR/gitea/conf
|
||||||
|
sudo mkdir -p $DATA_DOC
|
||||||
|
sudo mkdir -p $DATA_DBS
|
||||||
|
|
||||||
|
[ -r "bin/apply.sh" ] && ./bin/apply.sh
|
||||||
|
|
||||||
|
exit 0
|
||||||
|
|
||||||
30
clusters/postrun
Executable file
30
clusters/postrun
Executable file
@ -0,0 +1,30 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Info: postrun for oci-reg installation
|
||||||
|
# Author: JesusPerezLorenzo
|
||||||
|
# Release: 1.0.2
|
||||||
|
# Date: 15-01-2024
|
||||||
|
|
||||||
|
set +o errexit
|
||||||
|
set +o pipefail
|
||||||
|
|
||||||
|
SETTINGS_FILE=$1
|
||||||
|
SERVER_POS=$2
|
||||||
|
TASK_POS=$3
|
||||||
|
#SETTINGS_ROOT=$4
|
||||||
|
RUN_ROOT=$(dirname "$0")
|
||||||
|
#ORG=$(pwd)
|
||||||
|
|
||||||
|
[ -z "$SETTINGS_FILE" ] && [ -z "$SERVER_POS" ] && [ -z "$TASK_POS" ] && exit 0
|
||||||
|
|
||||||
|
YQ=$(type -P yq)
|
||||||
|
JQ=$(type -P jq)
|
||||||
|
[ -z "$YQ" ] && echo "yq not installed " && exit 1
|
||||||
|
[ -z "$JQ" ] && echo "jq not installed " && exit 1
|
||||||
|
|
||||||
|
[ -r "$RUN_ROOT/env-oci-reg" ] && . "$RUN_ROOT"/env-oci-reg
|
||||||
|
|
||||||
|
[ -z "$PROVISIONING" ] && echo "PROVISIONING not found in environment" && exit 1
|
||||||
|
|
||||||
|
. "$PROVISIONING"/core/lib/sops
|
||||||
|
|
||||||
|
#rm -f /tmp/oci-reg_config.json
|
||||||
31
clusters/web/default/bin/apply.sh
Executable file
31
clusters/web/default/bin/apply.sh
Executable file
@ -0,0 +1,31 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
ROOT=${ROOT:-.}
|
||||||
|
if [ -r "$ROOT/ssl/fullchain.pem" ] ; then
|
||||||
|
if [ -x "$ROOT/make_istio-system_secret.sh" ] ; then
|
||||||
|
$ROOT/make_istio-system_secret.sh $ROOT/ssl
|
||||||
|
else
|
||||||
|
kubectl delete secret web-certs -n cloudnative-zone 2>/dev/null
|
||||||
|
kubectl create secret tls web-certs --cert=$ROOT/ssl/fullchain.pem --key=$ROOT/ssl/privkey.pem -n cloudnative-zone
|
||||||
|
fi
|
||||||
|
if [ ! -r "$ROOT/ssl/fullchain.pem" ] ; then
|
||||||
|
echo "No SSL certificate"
|
||||||
|
exit
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
echo "checking configMaps ..."
|
||||||
|
kubectl delete -f $ROOT/configMap-etc.yaml 2>/dev/null
|
||||||
|
kubectl apply -f $ROOT/configMap-etc.yaml
|
||||||
|
|
||||||
|
kubectl delete -f $ROOT/web.yaml 2>/dev/null
|
||||||
|
kubectl delete -f $ROOT/srvc-web.yaml 2>/dev/null
|
||||||
|
kubectl delete -f $ROOT/prxy-virtual-srvc-web.yaml 2>/dev/null
|
||||||
|
kubectl delete -f $ROOT/prxy-gateway-web.yaml 2>/dev/null
|
||||||
|
|
||||||
|
kubectl apply -f $ROOT/srvc-web.yaml
|
||||||
|
kubectl apply -f $ROOT/prxy-virtual-srvc-web.yaml
|
||||||
|
kubectl apply -f $ROOT/prxy-gateway-web.yaml
|
||||||
|
kubectl apply -f $ROOT/web.yaml
|
||||||
|
|
||||||
|
#echo "web.cloudnative-zone reload ..."
|
||||||
|
#curl -s -o /dev/null -I -w "%{http_code}" https://web.cloudnative.zone
|
||||||
|
echo "__oOo__________oOo__________oOo__"
|
||||||
126
clusters/web/default/configMap-etc.yaml
Normal file
126
clusters/web/default/configMap-etc.yaml
Normal file
@ -0,0 +1,126 @@
|
|||||||
|
apiVersion: v1
|
||||||
|
kind: ConfigMap
|
||||||
|
metadata:
|
||||||
|
name: web-etc
|
||||||
|
namespace: cloudnative-zone
|
||||||
|
data:
|
||||||
|
htpasswd: |
|
||||||
|
daka:saTqF5QXUuD26
|
||||||
|
nginx.conf: |
|
||||||
|
user nginx;
|
||||||
|
|
||||||
|
# Set to number of CPU cores, auto will try to autodetect.
|
||||||
|
worker_processes auto;
|
||||||
|
|
||||||
|
# Maximum open file descriptors per process. Should be greater than worker_connections.
|
||||||
|
worker_rlimit_nofile 8192;
|
||||||
|
|
||||||
|
events {
|
||||||
|
# Set the maximum number of connection each worker process can open. Anything higher than this
|
||||||
|
# will require Unix optimisations.
|
||||||
|
worker_connections 8000;
|
||||||
|
|
||||||
|
# Accept all new connections as they're opened.
|
||||||
|
multi_accept on;
|
||||||
|
}
|
||||||
|
|
||||||
|
http {
|
||||||
|
# HTTP
|
||||||
|
#include global/http.conf;
|
||||||
|
|
||||||
|
# MIME Types
|
||||||
|
include mime.types;
|
||||||
|
default_type application/octet-stream;
|
||||||
|
|
||||||
|
# Limits & Timeouts
|
||||||
|
#include global/limits.conf;
|
||||||
|
|
||||||
|
# Specifies the main log format.
|
||||||
|
#log_format main '$http_x_real_ip - $real_ip_header - $http_x_forwarder_for - $http_x_real_ip - $remote_addr - $remote_user [$time_local] "$request" '
|
||||||
|
log_format main '$http_x_real_ip - $http_x_forwarder_for - $http_x_real_ip - $remote_addr - $remote_user [$time_local] "$request" '
|
||||||
|
'$status $body_bytes_sent "$http_referer" '
|
||||||
|
'"$http_user_agent" ';
|
||||||
|
# Default Logs
|
||||||
|
error_log /var/log/nginx/error.log warn;
|
||||||
|
access_log /var/log/nginx/access.log main;
|
||||||
|
|
||||||
|
# Gzip
|
||||||
|
#include global/gzip.conf;
|
||||||
|
|
||||||
|
# Modules
|
||||||
|
include /etc/nginx/conf.d/*.conf;
|
||||||
|
#upstream web {
|
||||||
|
# server auth:8080;
|
||||||
|
#}
|
||||||
|
# Sites
|
||||||
|
#include /etc/nginx/sites-enabled/*;
|
||||||
|
}
|
||||||
|
default: |
|
||||||
|
# Define path to cache and memory zone. The memory zone should be unique.
|
||||||
|
# keys_zone=fatstcgi-cache:100m creates the memory zone and sets the maximum size in MBs.
|
||||||
|
# inactive=60m will remove cached items that haven't been accessed for 60 minutes or more.
|
||||||
|
fastcgi_cache_path /cache levels=1:2 keys_zone=fatstcgi-cache:100m inactive=60m;
|
||||||
|
|
||||||
|
server {
|
||||||
|
# Ports to listen on, uncomment one.
|
||||||
|
listen 443 ssl http2;
|
||||||
|
listen [::]:443 ssl http2;
|
||||||
|
|
||||||
|
# Server name to listen for
|
||||||
|
server_name web.cloudnative.zone;
|
||||||
|
|
||||||
|
# Path to document root
|
||||||
|
root /var/www/static;
|
||||||
|
|
||||||
|
# Paths to certificate files.
|
||||||
|
ssl_certificate /etc/ssl-dom/fullchain.pem;
|
||||||
|
ssl_certificate_key /etc/ssl-dom/privkey.pem;
|
||||||
|
|
||||||
|
# File to be used as index
|
||||||
|
index index.php;
|
||||||
|
|
||||||
|
# Overrides logs defined in nginx.conf, allows per site logs.
|
||||||
|
error_log /dev/stdout warn;
|
||||||
|
access_log /dev/stdout main;
|
||||||
|
# Default server block rules
|
||||||
|
include server/defaults.conf;
|
||||||
|
# Fastcgi cache rules
|
||||||
|
include server/fastcgi-cache.conf;
|
||||||
|
|
||||||
|
# SSL rules
|
||||||
|
include server/ssl.conf;
|
||||||
|
# disable_symlinks off;
|
||||||
|
|
||||||
|
#Used when a load balancer wants to determine if this server is up or not
|
||||||
|
location /health_check {
|
||||||
|
return 200;
|
||||||
|
}
|
||||||
|
location / {
|
||||||
|
root /usr/share/nginx/html;
|
||||||
|
index index.html index.htm;
|
||||||
|
}
|
||||||
|
#location / {
|
||||||
|
# #auth_basic "Login";
|
||||||
|
# #auth_basic_user_file /etc/nginx/htpasswd;
|
||||||
|
# proxy_set_header Host $http_host;
|
||||||
|
# proxy_set_header X-Real-IP $remote_addr;
|
||||||
|
# proxy_set_header X-Forwarded-For
|
||||||
|
# $proxy_add_x_forwarded_for;
|
||||||
|
# proxy_redirect off;
|
||||||
|
# proxy_pass web;
|
||||||
|
#}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Redirect http to https
|
||||||
|
server {
|
||||||
|
listen 80;
|
||||||
|
listen [::]:80;
|
||||||
|
server_name web.cloudnative.zone;
|
||||||
|
#server_name localhost;
|
||||||
|
#return 301 https://web.cloudnative.zone$request_uri;
|
||||||
|
#return 301 https://fatstcgi-cache$request_uri;
|
||||||
|
location / {
|
||||||
|
root /usr/share/nginx/html;
|
||||||
|
index index.html index.htm;
|
||||||
|
}
|
||||||
|
}
|
||||||
9
clusters/web/default/install-web.sh
Normal file
9
clusters/web/default/install-web.sh
Normal file
@ -0,0 +1,9 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
kubectl apply -f ns
|
||||||
|
kubectl apply -f volumes
|
||||||
|
|
||||||
|
[ -r "bin/apply.sh" ] && ./bin/apply.sh
|
||||||
|
|
||||||
|
exit 0
|
||||||
|
|
||||||
13
clusters/web/default/make_istio-system_secret.sh
Executable file
13
clusters/web/default/make_istio-system_secret.sh
Executable file
@ -0,0 +1,13 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
SECRET_NAME=cloudnative-web-credentials
|
||||||
|
SSL_PATH=${1:-ssl}
|
||||||
|
[ ! -r "$SSL_PATH" ] && echo "SSL_PATH $SSLPATH not directory" && exit 1
|
||||||
|
|
||||||
|
NAMESPACE=istio-system
|
||||||
|
|
||||||
|
echo "create $NAMESPACE secret $SECRET_NAME for tls ... "
|
||||||
|
kubectl delete -n $NAMESPACE secret $SECRET_NAME 2>/dev/null
|
||||||
|
kubectl create -n $NAMESPACE secret tls $SECRET_NAME \
|
||||||
|
--key=$SSL_PATH/privkey.pem \
|
||||||
|
--cert=$SSL_PATH/fullchain.pem
|
||||||
|
|
||||||
5
clusters/web/default/ns/namespace.yaml
Normal file
5
clusters/web/default/ns/namespace.yaml
Normal file
@ -0,0 +1,5 @@
|
|||||||
|
---
|
||||||
|
apiVersion: v1
|
||||||
|
kind: Namespace
|
||||||
|
metadata:
|
||||||
|
name: cloudnative-zone
|
||||||
29
clusters/web/default/prxy-gateway-web.yaml
Normal file
29
clusters/web/default/prxy-gateway-web.yaml
Normal file
@ -0,0 +1,29 @@
|
|||||||
|
---
|
||||||
|
apiVersion: networking.istio.io/v1alpha3
|
||||||
|
kind: Gateway
|
||||||
|
metadata:
|
||||||
|
name: web-cloudnative-zone-gwy
|
||||||
|
namespace: istio-system
|
||||||
|
spec:
|
||||||
|
selector:
|
||||||
|
istio: ingressgateway # use istio default ingress gateway
|
||||||
|
servers:
|
||||||
|
- port:
|
||||||
|
number: 80
|
||||||
|
name: http-cnr
|
||||||
|
protocol: HTTP
|
||||||
|
tls:
|
||||||
|
httpsRedirect: true
|
||||||
|
hosts:
|
||||||
|
- "web.cloudnative.zone"
|
||||||
|
- port:
|
||||||
|
number: 443
|
||||||
|
name: https-cnr
|
||||||
|
protocol: HTTPS
|
||||||
|
tls:
|
||||||
|
#mode: PASSTHROUGH
|
||||||
|
mode: SIMPLE
|
||||||
|
credentialName: cloudnative-web-credentials
|
||||||
|
hosts:
|
||||||
|
- "web.cloudnative.zone"
|
||||||
|
|
||||||
46
clusters/web/default/prxy-virtual-srvc-web.yaml
Normal file
46
clusters/web/default/prxy-virtual-srvc-web.yaml
Normal file
@ -0,0 +1,46 @@
|
|||||||
|
---
|
||||||
|
apiVersion: networking.istio.io/v1alpha3
|
||||||
|
kind: VirtualService
|
||||||
|
metadata:
|
||||||
|
name: web-cloudnative-zone
|
||||||
|
namespace: istio-system
|
||||||
|
spec:
|
||||||
|
hosts:
|
||||||
|
- "web.cloudnative.zone"
|
||||||
|
gateways:
|
||||||
|
- web-cloudnative-zone-gwy
|
||||||
|
# tcp:
|
||||||
|
# - match:
|
||||||
|
# - port:
|
||||||
|
# route:
|
||||||
|
# - destination:
|
||||||
|
# port:
|
||||||
|
# number:
|
||||||
|
# host: web.cloudnative-zone.svc.cluster.local
|
||||||
|
http:
|
||||||
|
- match:
|
||||||
|
- port: 443
|
||||||
|
route:
|
||||||
|
- destination:
|
||||||
|
port:
|
||||||
|
number: 80
|
||||||
|
host: web.cloudnative-zone.svc.cluster.local
|
||||||
|
# tls:
|
||||||
|
# - match:
|
||||||
|
# - port:
|
||||||
|
# sniHosts:
|
||||||
|
# - "web.cloudnative.zone"
|
||||||
|
# route:
|
||||||
|
# - destination:
|
||||||
|
# port:
|
||||||
|
# number:
|
||||||
|
# host: crates.cloudnative-zone.svc.cluster.local
|
||||||
|
# - match:
|
||||||
|
# - port: 443
|
||||||
|
# sniHosts:
|
||||||
|
# - "web.cloudnative.zone"
|
||||||
|
# route:
|
||||||
|
# - destination:
|
||||||
|
# port:
|
||||||
|
# number: 3000
|
||||||
|
# host: web.cloudnative-zone.svc.cluster.local
|
||||||
15
clusters/web/default/srvc-web.yaml
Normal file
15
clusters/web/default/srvc-web.yaml
Normal file
@ -0,0 +1,15 @@
|
|||||||
|
apiVersion: v1
|
||||||
|
kind: Service
|
||||||
|
metadata:
|
||||||
|
name: web
|
||||||
|
namespace: cloudnative-zone
|
||||||
|
labels:
|
||||||
|
app: web-cloudnative
|
||||||
|
spec:
|
||||||
|
ports:
|
||||||
|
- port: 443
|
||||||
|
name: cn-https
|
||||||
|
- port: 80
|
||||||
|
name: cn-http
|
||||||
|
selector:
|
||||||
|
app: web-cloudnative
|
||||||
14
clusters/web/default/volumes/PersistentVolumeData.yaml
Normal file
14
clusters/web/default/volumes/PersistentVolumeData.yaml
Normal file
@ -0,0 +1,14 @@
|
|||||||
|
apiVersion: v1
|
||||||
|
kind: PersistentVolumeClaim
|
||||||
|
metadata:
|
||||||
|
name: web-data-vol
|
||||||
|
namespace: cloudnative-zone
|
||||||
|
labels:
|
||||||
|
app: cloudnative-zone-repo
|
||||||
|
spec:
|
||||||
|
storageClassName: nfs-client
|
||||||
|
accessModes:
|
||||||
|
- ReadWriteMany
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
storage: 5Gi
|
||||||
56
clusters/web/default/web.yaml
Normal file
56
clusters/web/default/web.yaml
Normal file
@ -0,0 +1,56 @@
|
|||||||
|
apiVersion: apps/v1
|
||||||
|
kind: Deployment
|
||||||
|
metadata:
|
||||||
|
namespace: cloudnative-zone
|
||||||
|
name: web-deployment
|
||||||
|
labels:
|
||||||
|
app: web-cloudnative
|
||||||
|
spec:
|
||||||
|
replicas: 1
|
||||||
|
selector:
|
||||||
|
matchLabels:
|
||||||
|
app: web-cloudnative
|
||||||
|
template:
|
||||||
|
metadata:
|
||||||
|
labels:
|
||||||
|
app: web-cloudnative
|
||||||
|
spec:
|
||||||
|
containers:
|
||||||
|
- name: web-container
|
||||||
|
image: docker.io/nginx:alpine
|
||||||
|
imagePullPolicy: IfNotPresent
|
||||||
|
ports:
|
||||||
|
- containerPort: 80
|
||||||
|
name: cn-http
|
||||||
|
- containerPort: 443
|
||||||
|
name: cn-https
|
||||||
|
env:
|
||||||
|
volumeMounts:
|
||||||
|
- name: web-data-storage
|
||||||
|
mountPath: /usr/share/nginx/html
|
||||||
|
#- mountPath: /etc/ssl-dom
|
||||||
|
# readOnly: true
|
||||||
|
# name: web-certs
|
||||||
|
- mountPath: /etc/nginx/nginx.conf
|
||||||
|
readOnly: true
|
||||||
|
name: web-etc
|
||||||
|
subPath: nginx.conf
|
||||||
|
volumes:
|
||||||
|
- name: web-data-storage
|
||||||
|
persistentVolumeClaim:
|
||||||
|
claimName: web-data-vol
|
||||||
|
#claimName: web-data-claim
|
||||||
|
- name: web-etc
|
||||||
|
configMap:
|
||||||
|
name: web-etc
|
||||||
|
items:
|
||||||
|
- key: nginx.conf
|
||||||
|
path: nginx.conf
|
||||||
|
#- name: web-certs
|
||||||
|
# secret:
|
||||||
|
# secretName: repo-certs
|
||||||
|
# items:
|
||||||
|
# - key: tls.crt
|
||||||
|
# path: fullchain.pem
|
||||||
|
# - key: tls.key
|
||||||
|
# path: privkey.pem
|
||||||
8
clusters/web/kcl/kcl.mod
Normal file
8
clusters/web/kcl/kcl.mod
Normal file
@ -0,0 +1,8 @@
|
|||||||
|
[package]
|
||||||
|
name = "web"
|
||||||
|
edition = "v0.11.3"
|
||||||
|
version = "0.0.1"
|
||||||
|
|
||||||
|
[dependencies]
|
||||||
|
clusters = { path = "../..", version = "0.0.1" }
|
||||||
|
provisioning = { path = "../../../kcl", version = "0.0.1" }
|
||||||
9
clusters/web/kcl/kcl.mod.lock
Normal file
9
clusters/web/kcl/kcl.mod.lock
Normal file
@ -0,0 +1,9 @@
|
|||||||
|
[dependencies]
|
||||||
|
[dependencies.clusters]
|
||||||
|
name = "clusters"
|
||||||
|
full_name = "clusters_0.0.1"
|
||||||
|
version = "0.0.1"
|
||||||
|
[dependencies.provisioning]
|
||||||
|
name = "provisioning"
|
||||||
|
full_name = "vPkg_21492148-301b-4f98-8a03-8d9380e04f76_0.0.1"
|
||||||
|
version = "0.0.1"
|
||||||
195
providers/README.md
Normal file
195
providers/README.md
Normal file
@ -0,0 +1,195 @@
|
|||||||
|
# Cloud Providers
|
||||||
|
|
||||||
|
This directory contains provider implementations for different cloud platforms and local environments. Providers handle the infrastructure provisioning and management for servers, networking, and storage resources.
|
||||||
|
|
||||||
|
## Available Providers
|
||||||
|
|
||||||
|
### AWS Provider (`aws/`)
|
||||||
|
**Platform**: Amazon Web Services
|
||||||
|
**Features**:
|
||||||
|
- EC2 instance management with multiple instance types
|
||||||
|
- VPC networking with Security Groups and multi-AZ support
|
||||||
|
- EBS volume types (gp2, gp3, io1, io2, st1, sc1)
|
||||||
|
- Advanced networking with subnet and route table management
|
||||||
|
- IAM integration for security and access control
|
||||||
|
|
||||||
|
**Configuration**: `aws/kcl/defaults_aws.k`, `server_aws.k`, `provision_aws.k`
|
||||||
|
|
||||||
|
### UpCloud Provider (`upcloud/`)
|
||||||
|
**Platform**: UpCloud infrastructure
|
||||||
|
**Features**:
|
||||||
|
- Server provisioning with flexible plans and zones
|
||||||
|
- Advanced backup scheduling with automated snapshots
|
||||||
|
- Server grouping and organization capabilities
|
||||||
|
- Multiple storage types (maxiops, hdd, custom) with backup support
|
||||||
|
- Network configuration with firewall rules
|
||||||
|
|
||||||
|
**Configuration**: `upcloud/kcl/defaults_upcloud.k`, `server_upcloud.k`, `provision_upcloud.k`
|
||||||
|
|
||||||
|
### Local Provider (`local/`)
|
||||||
|
**Platform**: Local development environment
|
||||||
|
**Features**:
|
||||||
|
- Local server simulation for development and testing
|
||||||
|
- Simplified storage and networking configuration
|
||||||
|
- Container-based local infrastructure
|
||||||
|
- Development workflow optimization
|
||||||
|
- Testing and validation support
|
||||||
|
|
||||||
|
**Configuration**: `local/kcl/defaults_local.k`, `server_local.k`, `provision_local.k`
|
||||||
|
|
||||||
|
## Provider Architecture
|
||||||
|
|
||||||
|
### Schema Structure
|
||||||
|
Each provider implements a consistent interface while providing platform-specific capabilities:
|
||||||
|
|
||||||
|
```kcl
|
||||||
|
# Common base schemas extended by each provider
|
||||||
|
Storage_provider(provisioning.Storage)
|
||||||
|
ServerDefaults_provider(provisioning.ServerDefaults)
|
||||||
|
Provision_provider(provisioning.Provision)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Configuration Patterns
|
||||||
|
1. **defaults_*.k**: Provider-specific default configurations and schemas
|
||||||
|
2. **server_*.k**: Server provisioning and management schemas
|
||||||
|
3. **provision_*.k**: Environment and provisioning settings
|
||||||
|
4. **dependencies.k**: Dependency management for batch workflows
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
### Server Creation
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Using specific provider
|
||||||
|
PROVISIONING_PROVIDER=aws provisioning/core/cli/provisioning server create
|
||||||
|
|
||||||
|
# Provider auto-detection based on infrastructure configuration
|
||||||
|
provisioning/core/cli/provisioning server create --infra <infra-name>
|
||||||
|
```
|
||||||
|
|
||||||
|
### Provider Configuration
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Show provider-specific settings
|
||||||
|
provisioning/core/cli/provisioning show settings --provider aws
|
||||||
|
|
||||||
|
# List provider capabilities
|
||||||
|
provisioning/core/cli/provisioning show providers
|
||||||
|
```
|
||||||
|
|
||||||
|
### Batch Operations
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Multi-provider batch workflows
|
||||||
|
nu -c "use core/nulib/workflows/batch.nu *; batch submit workflows/multi_cloud.k"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Configuration Examples
|
||||||
|
|
||||||
|
### AWS Configuration
|
||||||
|
```kcl
|
||||||
|
aws_config: Storage_aws = {
|
||||||
|
volumes = [
|
||||||
|
{
|
||||||
|
device = "/dev/sda1"
|
||||||
|
size_gb = 20
|
||||||
|
volume_type = "gp3"
|
||||||
|
encrypted = True
|
||||||
|
}
|
||||||
|
]
|
||||||
|
vpc = {
|
||||||
|
cidr_block = "10.0.0.0/16"
|
||||||
|
enable_dns_hostnames = True
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### UpCloud Configuration
|
||||||
|
```kcl
|
||||||
|
upcloud_config: Storage_upcloud = {
|
||||||
|
volumes = [
|
||||||
|
{
|
||||||
|
size_gb = 25
|
||||||
|
volume_type = "maxiops"
|
||||||
|
backup = {
|
||||||
|
enabled = True
|
||||||
|
schedule = "daily"
|
||||||
|
retention_days = 7
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Local Configuration
|
||||||
|
```kcl
|
||||||
|
local_config: Storage_local = {
|
||||||
|
volumes = [
|
||||||
|
{
|
||||||
|
size_gb = 10
|
||||||
|
mount_path = "/data"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
network = {
|
||||||
|
bridge_name = "provisioning-br0"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Integration Features
|
||||||
|
|
||||||
|
### Batch Workflow Support
|
||||||
|
All providers support batch operations for:
|
||||||
|
- Multiple server provisioning
|
||||||
|
- Cross-provider deployments
|
||||||
|
- Dependency management and ordering
|
||||||
|
- Rollback and recovery capabilities
|
||||||
|
|
||||||
|
### Task Service Integration
|
||||||
|
Providers integrate seamlessly with task services for:
|
||||||
|
- Container runtime installation
|
||||||
|
- Kubernetes cluster setup
|
||||||
|
- Networking configuration
|
||||||
|
- Storage provisioning
|
||||||
|
|
||||||
|
### Orchestrator Support
|
||||||
|
Full integration with the Rust orchestrator for:
|
||||||
|
- High-performance coordination
|
||||||
|
- REST API access
|
||||||
|
- Real-time monitoring
|
||||||
|
- State management
|
||||||
|
|
||||||
|
## Development
|
||||||
|
|
||||||
|
### Adding a New Provider
|
||||||
|
|
||||||
|
1. Create provider directory: `mkdir <provider-name>`
|
||||||
|
2. Create KCL directory: `mkdir <provider-name>/kcl`
|
||||||
|
3. Implement required schemas:
|
||||||
|
- `defaults_<provider>.k` - Provider defaults and schemas
|
||||||
|
- `server_<provider>.k` - Server management
|
||||||
|
- `provision_<provider>.k` - Environment settings
|
||||||
|
- `dependencies.k` - Dependency management
|
||||||
|
4. Add module definition: `kcl.mod`
|
||||||
|
5. Register provider in main provisioning module
|
||||||
|
|
||||||
|
### Provider Interface Requirements
|
||||||
|
|
||||||
|
Providers must implement:
|
||||||
|
- Server lifecycle management (create, delete, list, status)
|
||||||
|
- Storage and volume management
|
||||||
|
- Network configuration
|
||||||
|
- Security and access control
|
||||||
|
- Monitoring and health checks
|
||||||
|
|
||||||
|
### Validation and Testing
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Validate provider KCL definitions
|
||||||
|
kcl run <provider>/kcl/
|
||||||
|
|
||||||
|
# Test provider functionality
|
||||||
|
provisioning/core/cli/provisioning --debug --check server create --provider <provider>
|
||||||
|
```
|
||||||
|
|
||||||
|
For detailed provider-specific documentation, see the individual provider directories and the main [provisioning documentation](../../../docs/).
|
||||||
41
providers/REFERENCE.md
Normal file
41
providers/REFERENCE.md
Normal file
@ -0,0 +1,41 @@
|
|||||||
|
# Providers Reference
|
||||||
|
|
||||||
|
This directory will reference the existing provider implementations.
|
||||||
|
|
||||||
|
## Current Implementation Location
|
||||||
|
`/Users/Akasha/repo-cnz/src/provisioning/providers/`
|
||||||
|
|
||||||
|
## Available Providers
|
||||||
|
- **AWS**: Amazon Web Services provider
|
||||||
|
- **UpCloud**: UpCloud provider
|
||||||
|
- **Local**: Local development provider
|
||||||
|
|
||||||
|
## Provider Structure
|
||||||
|
Each provider includes:
|
||||||
|
- KCL configuration files (`kcl/`)
|
||||||
|
- Nushell libraries (`nulib/`)
|
||||||
|
- Jinja2 templates (`templates/`)
|
||||||
|
- Generation scripts (`generate/`)
|
||||||
|
|
||||||
|
## Integration Status
|
||||||
|
- **Current**: Fully functional in original location
|
||||||
|
- **New Structure**: Reference established
|
||||||
|
- **Migration**: Planned for future phase
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
Providers remain fully functional via the main provisioning CLI:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# List available providers
|
||||||
|
./core/nulib/provisioning provider list
|
||||||
|
|
||||||
|
# Use specific provider
|
||||||
|
./core/nulib/provisioning --provider aws server create
|
||||||
|
```
|
||||||
|
|
||||||
|
## Adding New Providers
|
||||||
|
New providers should follow the existing patterns in:
|
||||||
|
`/Users/Akasha/repo-cnz/src/provisioning/providers/`
|
||||||
|
|
||||||
|
## Development
|
||||||
|
Provider development continues in the original location with full functionality.
|
||||||
65
providers/aws/README.md
Normal file
65
providers/aws/README.md
Normal file
@ -0,0 +1,65 @@
|
|||||||
|
# AWS Declarative Provision via scripts & templates
|
||||||
|
|
||||||
|
Part of [Cloud Native zone Provision](/CloudNativeZone/cnz-provision)
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
Install [Python](https://es.wikipedia.org/wiki/Python)
|
||||||
|
|
||||||
|
For [Ubuntu](https://ubuntu.com/)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo apt install wget build-essential libncursesw5-dev libssl-dev \
|
||||||
|
libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev libffi-dev zlib1g-dev
|
||||||
|
|
||||||
|
sudo add-apt-repository ppa:deadsnakes/ppa
|
||||||
|
|
||||||
|
sudo apt install python3.13
|
||||||
|
sudo apt-get -y install python3-pip
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
Install [Jinja2 engine](https://jinja.palletsprojects.com/en/3.1.x/)
|
||||||
|
|
||||||
|
```python
|
||||||
|
pip3 install Jinja2
|
||||||
|
```
|
||||||
|
|
||||||
|
Install [Python YAML](https://pypi.org/project/PyYAML/)
|
||||||
|
|
||||||
|
```python
|
||||||
|
pip3 install PyYAML
|
||||||
|
```
|
||||||
|
|
||||||
|
[Install YQ](https://github.com/mikefarah/yq/#install)
|
||||||
|
|
||||||
|
[Install JQ](https://jqlang.github.io/jq/download/)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
apt install jq
|
||||||
|
```
|
||||||
|
|
||||||
|
## How To
|
||||||
|
|
||||||
|
- For private network ID in same **zone/region**, for example:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
```
|
||||||
|
|
||||||
|
- Complete a **settings.yaml** using [setttings.yaml](settings.yaml) file as model (vars and values are commented as help)
|
||||||
|
|
||||||
|
- Use command [on_aws_server](on_aws_server)
|
||||||
|
|
||||||
|
> **on_aws_server** will use [parsertemplate.py]([parsertemplate.py) to load **settings.yaml** as data
|
||||||
|
> It will generate [aws CLI](https://aws.amazon.com/cli/) commands to create resources
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
[YAML org](https://yaml.org/)
|
||||||
|
|
||||||
|
[YQ](https://github.com/mikefarah/yq)
|
||||||
|
|
||||||
|
[YQ Documentation](https://mikefarah.gitbook.io/yq/)
|
||||||
|
|
||||||
|
[Jinja2 Tempalte engine](https://jinja.palletsprojects.com/en/3.1.x/)
|
||||||
|
|
||||||
9
providers/aws/bin/create-default-subnet.sh
Executable file
9
providers/aws/bin/create-default-subnet.sh
Executable file
@ -0,0 +1,9 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
[ -z "$AWS_PROFILE" ] || [ ! -r "$HOME/.aws/credentials" ] && echo "AWS credentials not found" && exit 1
|
||||||
|
|
||||||
|
[ -z "$1" ] && echo "No zone provided (example eu-west-1)" && exit 1
|
||||||
|
|
||||||
|
aws ec2 create-default-subnet --availability-zone ${1}a
|
||||||
|
aws ec2 create-default-subnet --availability-zone ${1}b
|
||||||
|
aws ec2 create-default-subnet --availability-zone ${1}c
|
||||||
4
providers/aws/bin/get-image.sh
Executable file
4
providers/aws/bin/get-image.sh
Executable file
@ -0,0 +1,4 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
match="debian-12-amd64"
|
||||||
|
aws ec2 describe-images --owners --out json | jq '.Images[] | select( .Name | contains("'$match'")) '
|
||||||
125
providers/aws/bin/install.sh
Executable file
125
providers/aws/bin/install.sh
Executable file
@ -0,0 +1,125 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Info: Script to install aws tools
|
||||||
|
# Author: JesusPerezLorenzo
|
||||||
|
# Release: 1.0
|
||||||
|
# Date: 15-04-2024
|
||||||
|
|
||||||
|
[ "$DEBUG" == "-x" ] && set -x
|
||||||
|
|
||||||
|
USAGE="install-tools [ tool-name: tera k9s, etc | all] [--update]
|
||||||
|
As alternative use environment var TOOL_TO_INSTALL with a list-of-tools (separeted with spaces)
|
||||||
|
Versions are set in ./versions file
|
||||||
|
|
||||||
|
This can be called by directly with an argumet or from an other srcipt
|
||||||
|
"
|
||||||
|
|
||||||
|
ORG=$(pwd)
|
||||||
|
function _info_tools {
|
||||||
|
local match=$1
|
||||||
|
local info_keys
|
||||||
|
info_keys="info version site"
|
||||||
|
|
||||||
|
if [ -z "$match" ] || [ "$match" == "all" ] || [ "$match" == "-" ]; then
|
||||||
|
match="all"
|
||||||
|
fi
|
||||||
|
echo "$PROVIDER_TITLE"
|
||||||
|
[ ! -r "$PROVIDERS_PATH/$PROVIDER_NAME/provisioning.yaml" ] && return
|
||||||
|
echo "-------------------------------------------------------"
|
||||||
|
case "$match" in
|
||||||
|
"i" | "?" | "info")
|
||||||
|
for key in $info_keys
|
||||||
|
do
|
||||||
|
echo -n "$key:"
|
||||||
|
[ "$key" != "version" ] && echo -ne "\t"
|
||||||
|
echo " $(grep "^$key:" "$PROVIDERS_PATH/$PROVIDER_NAME/provisioning.yaml" | sed "s/$key: //g")"
|
||||||
|
done
|
||||||
|
;;
|
||||||
|
"all")
|
||||||
|
cat "$PROVIDERS_PATH/$PROVIDER_NAME/provisioning.yaml"
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
echo -e "$match:\t $(grep "^$match:" "$PROVIDERS_PATH/$PROVIDER_NAME/provisioning.yaml" | sed "s/$match: //g")"
|
||||||
|
esac
|
||||||
|
echo "________________________________________________________"
|
||||||
|
}
|
||||||
|
function _install_tools {
|
||||||
|
local match=$1
|
||||||
|
shift
|
||||||
|
local options
|
||||||
|
options="$*"
|
||||||
|
local has_aws
|
||||||
|
local aws_version
|
||||||
|
|
||||||
|
OS="$(uname | tr '[:upper:]' '[:lower:]')"
|
||||||
|
ORG_OS=$(uname)
|
||||||
|
ARCH="$(uname -m | sed -e 's/x86_64/amd64/' -e 's/\(arm\)\(64\)\?.*/\1\2/' -e 's/aarch64$/arm64/')"
|
||||||
|
ORG_ARCH="$(uname -m)"
|
||||||
|
|
||||||
|
AWS_VERSION=${AWS_AWS_VERSION:-}
|
||||||
|
if [ -n "$AWS_VERSION" ] && [ "$match" == "all" ] || [ "$match" == "aws" ] ; then
|
||||||
|
[ -r "/usr/bin/aws" ] && mv /usr/bin/aws /usr/bin/_aws
|
||||||
|
has_aws=$(type -P aws)
|
||||||
|
num_version=0
|
||||||
|
[ -n "$has_aws" ] && aws_version=$(aws --version | cut -f1 -d" " | sed 's,aws-cli/,,g') && num_version=${aws_version//\./}
|
||||||
|
[ -z "$num_version" ] && num_version=0
|
||||||
|
expected_version_num=${AWS_VERSION//\./}
|
||||||
|
if [ -z "$CHECK_ONLY" ] && [ "$num_version" -ne "$expected_version_num" ] ; then
|
||||||
|
cd "$ORG" || exit 1
|
||||||
|
curl "https://awscli.amazonaws.com/awscli-exe-${OS}-${ORG_ARCH}.zip" -o "/tmp/awscliv2.zip"
|
||||||
|
cd /tmp
|
||||||
|
unzip awscliv2.zip >/dev/null
|
||||||
|
[ "$1" != "-update" ] && [ -d "/usr/local/aws-cli" ] && sudo rm -rf "/usr/local/aws-cli"
|
||||||
|
sudo ./aws/install && printf "%s\t%s\n" "aws" "installed $AWS_VERSION"
|
||||||
|
#sudo ./aws/install $options && echo "aws cli installed"
|
||||||
|
cd "$ORG" && rm -rf /tmp/awscliv2.zip /tmp/aws
|
||||||
|
elif [ -n "$CHECK_ONLY" ] ; then
|
||||||
|
printf "%s\t%s\t%s\n" "aws" "$aws_version" "expected $AWS_VERSION"
|
||||||
|
else
|
||||||
|
printf "%s\t%s\n" "aws" "already $AWS_VERSION"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
function _on_tools {
|
||||||
|
local tools_list=$1
|
||||||
|
[ -z "$tools_list" ] || [[ "$tools_list" == -* ]] && tools_list=${TOOL_TO_INSTALL:-all}
|
||||||
|
case $tools_list in
|
||||||
|
"all")
|
||||||
|
_install_tools "all" "$@"
|
||||||
|
;;
|
||||||
|
"info" | "i" | "?")
|
||||||
|
shift
|
||||||
|
_info_tools "$@"
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
for tool in $tools_list
|
||||||
|
do
|
||||||
|
[[ "$tool" == -* ]] && continue
|
||||||
|
_install_tools "$tool" "${*//$tool/}"
|
||||||
|
done
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
|
||||||
|
set -o allexport
|
||||||
|
## shellcheck disable=SC1090
|
||||||
|
[ -n "$PROVISIONING_ENV" ] && [ -r "$PROVISIONING_ENV" ] && source "$PROVISIONING_ENV"
|
||||||
|
[ -r "../env-provisioning" ] && source ../env-provisioning
|
||||||
|
[ -r "env-provisioning" ] && source ./env-provisioning
|
||||||
|
set +o allexport
|
||||||
|
|
||||||
|
export PROVISIONING=${PROVISIONING:-/usr/local/provisioning}
|
||||||
|
|
||||||
|
PROVIDERS_PATH=${PROVIDERS_PATH:-"$PROVISIONING/providers"}
|
||||||
|
|
||||||
|
PROVIDER_NAME="aws"
|
||||||
|
PROVIDER_TITLE="AWS"
|
||||||
|
|
||||||
|
if [ -r "$(dirname "$0")/../versions" ] ; then
|
||||||
|
. "$(dirname "$0")"/../versions
|
||||||
|
elif [ -r "$(dirname "$0")/versions" ] ; then
|
||||||
|
. "$(dirname "$0")"/versions
|
||||||
|
fi
|
||||||
|
|
||||||
|
[ "$1" == "-h" ] && echo "$USAGE" && shift
|
||||||
|
[ "$1" == "check" ] && CHECK_ONLY="yes" && shift
|
||||||
|
[ -n "$1" ] && cd /tmp && _on_tools "$@"
|
||||||
|
[ -z "$1" ] && _on_tools
|
||||||
31
providers/aws/bin/on-ssh.sh
Executable file
31
providers/aws/bin/on-ssh.sh
Executable file
@ -0,0 +1,31 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
USAGE="on-ssh.sh show|describe | create (key-name) | import (pub-key-path) | delete (key-name)
|
||||||
|
reference: https://docs.aws.amazon.com/cli/latest/reference/ec2/import-key-pair.html
|
||||||
|
"
|
||||||
|
[ -z "$AWS_PROFILE" ] || [ ! -r "$HOME/.aws/credentials" ] && echo "AWS credentials not found" && exit 1
|
||||||
|
|
||||||
|
case "$1" in
|
||||||
|
show|describe)
|
||||||
|
aws ec2 describe-key-pairs
|
||||||
|
;;
|
||||||
|
create)
|
||||||
|
[ -z "$2" ] && echo "No name to create ssh found" && exit 1
|
||||||
|
aws ec2 create-key-pair \
|
||||||
|
--key-name "$2" \
|
||||||
|
--key-type ed25519 \
|
||||||
|
--query 'KeyMaterial' --output text
|
||||||
|
;;
|
||||||
|
import)
|
||||||
|
[ -z "$2" ] && echo "No name to reate ssh found" && exit 1
|
||||||
|
[ ! -r "$HOME/.ssh/$2" ] && echo "No public key found in $HOME/.ssh/$2" && exit 1
|
||||||
|
--out json | jq -r '.InstanceStatuses[] | select(.InstanceState.Name == "running")aws ec2 import-key-pair --key-name "$2" --public-key-material fileb://~/.ssh/$2
|
||||||
|
;;
|
||||||
|
delete)
|
||||||
|
[ -z "$2" ] && echo "No name for create ssh found" && exit 1
|
||||||
|
aws ec2 delete-key-pair --key-name "$2"
|
||||||
|
;;
|
||||||
|
-h|help) echo "$USAGE"
|
||||||
|
;;
|
||||||
|
*) echo "Option $1 not defined"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
11
providers/aws/bin/public_ip_ec2.sh
Executable file
11
providers/aws/bin/public_ip_ec2.sh
Executable file
@ -0,0 +1,11 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
[ -z "$AWS_PROFILE" ] || [ ! -r "$HOME/.aws/credentials" ] && echo "AWS credentials not found" && exit 1
|
||||||
|
|
||||||
|
[ -z "$1" ] && echo "No instance id found" && exit 1
|
||||||
|
|
||||||
|
instace_id=$1
|
||||||
|
|
||||||
|
aws ec2 describe-instances --instance-ids $instance_id \
|
||||||
|
--query 'Reservations[*].Instances[*].PublicIpAddress' \
|
||||||
|
--output text
|
||||||
66
providers/aws/generate/aws_defaults.k.j2
Normal file
66
providers/aws/generate/aws_defaults.k.j2
Normal file
@ -0,0 +1,66 @@
|
|||||||
|
import aws_prov
|
||||||
|
# Settings from servers has priority over defaults ones, if a value is not set in server item, defaults one will be used instead
|
||||||
|
aws_prov.ServerDefaults_aws {
|
||||||
|
# AWS provision data settings
|
||||||
|
#prov_settings = "defs/aws_data.k"
|
||||||
|
time_zone = "UTC"
|
||||||
|
# UpCloud Zone like = "es-mad1"
|
||||||
|
#zone = "es-mad1"
|
||||||
|
#zone = "eu-west-1"
|
||||||
|
zone = "eu-south-2"
|
||||||
|
# Second to wait before check in for running state
|
||||||
|
running_wait = 10
|
||||||
|
# Total seconds to wait for running state before timeout
|
||||||
|
running_timeout = 200
|
||||||
|
# If not Storage size, Plan Storage size will be used
|
||||||
|
storages = [
|
||||||
|
{ name = "root", size = 15, total = 15, type = "ext4" , mount = True, mount_path = "/", parts = [
|
||||||
|
# { name = "root", size = 25, total = 80, type = "ext4" , mount = True, mount_path = "/", parts = [
|
||||||
|
# { name = "kluster", size = 55, type = "xfs" , mount = False }
|
||||||
|
]}
|
||||||
|
]
|
||||||
|
# Server OS to use (will be the first storage device). The value should be title or UUID of an either
|
||||||
|
# public or private template. Set to empty to fully customise the storages.
|
||||||
|
# Default = "Ubuntu Server 20.04 LTS (Focal Fossa) "
|
||||||
|
#storage_os = "Debian GNU/Linux 12 (Bookworm)"
|
||||||
|
storage_os_find = "name: debian-12 | arch: x86_64"
|
||||||
|
#storage_os = "find"
|
||||||
|
# eu-west-1
|
||||||
|
#storage_os = "ami-0eb11ab33f229b26c"
|
||||||
|
# eu-south-2 ami-0e733f933140cf5cd (64 bits (x86)) / ami-0696f50508962ab62 (64 bits (Arm))
|
||||||
|
storage_os = "ami-0e733f933140cf5cd"
|
||||||
|
# Add one or more SSH keys to the admin account. Accepted values are SSH public keys or filenames from
|
||||||
|
# where to read the keys.
|
||||||
|
# ssh public key to be included in /root/.ssh/authorized_keys
|
||||||
|
ssh_key_path = "~/.ssh/id_cdci.pub"
|
||||||
|
ssh_key_name = "cdci"
|
||||||
|
# utility network, if no value it will not be set and utility IP will not be set
|
||||||
|
network_utility_ipv4 = True
|
||||||
|
network_utility_ipv6 = False
|
||||||
|
# public network, if no value it will not be set and public IP will not be set
|
||||||
|
network_public_ipv4 = True
|
||||||
|
network_public_ipv6 = False
|
||||||
|
# To use private network needs to be created previously to get ID and IP
|
||||||
|
# If network_private_id contains "CREATE" it will be created with 'name' in 'cidr_block' and updated here
|
||||||
|
# network_private_id = "CREATE"
|
||||||
|
# Otherwise created manually and update id
|
||||||
|
# Example = upctl network create --name "Custom Net" --zone nl-ams1 --ip-network address = 10.11.2.0/24
|
||||||
|
# IF content is 'CREATE' a network_private_id will be created and create here
|
||||||
|
# IF ID does not already exist a new network_private_id will be created and replaced here
|
||||||
|
network_private_id = "03d64e84-50ab-46a3-bf28-b4d93783aa04"
|
||||||
|
network_private_name = "Private_Net"
|
||||||
|
# To use private network, IPs will be set in servers items
|
||||||
|
priv_cidr_block = "10.11.2.0/24"
|
||||||
|
primary_dns = ""
|
||||||
|
secondary_dns = ""
|
||||||
|
main_domain = "librecloud.local"
|
||||||
|
domains_search = "librecloud.local"
|
||||||
|
# Main user (default Debian user is admin)
|
||||||
|
user = "devadm"
|
||||||
|
user_home = "/home/devadm"
|
||||||
|
user_ssh_port = 22
|
||||||
|
fix_local_hosts = True
|
||||||
|
#installer_user = "root"
|
||||||
|
installer_user = "admin"
|
||||||
|
}
|
||||||
|
|
||||||
57
providers/aws/generate/servers.k.j2
Normal file
57
providers/aws/generate/servers.k.j2
Normal file
@ -0,0 +1,57 @@
|
|||||||
|
aws_prov.Server_aws {
|
||||||
|
# Hostname as reference for resource if is changed later inside server, change will not be updated in resource inventory
|
||||||
|
hostname = "lab-cp-0"
|
||||||
|
title = "Infra CP 0"
|
||||||
|
plan = "t3.micro"
|
||||||
|
reqplan = {
|
||||||
|
scale = True
|
||||||
|
arch = "x86_64"
|
||||||
|
cores = 2
|
||||||
|
memory = 1024
|
||||||
|
infaces = 2
|
||||||
|
ena = "supported,required"
|
||||||
|
# virtyp = "hvm"
|
||||||
|
gen = "current"
|
||||||
|
}
|
||||||
|
# If not Storage size, Plan Storage size will be used
|
||||||
|
storages = [
|
||||||
|
aws_prov.Storage_aws {
|
||||||
|
name = "root",
|
||||||
|
total = 30,
|
||||||
|
# size = 50, total = 50,
|
||||||
|
# size = 15, total = 25,
|
||||||
|
# size = 25, total = 50,
|
||||||
|
labels = "{Key=storager,Value=vol0}",
|
||||||
|
parts = [
|
||||||
|
{ name = "root", size = 30, type = "ext4" , mount = True, mount_path = "/" },
|
||||||
|
#{ name = "kluster", size = 10, type = "xfs" , mount = False }
|
||||||
|
]
|
||||||
|
},
|
||||||
|
aws_prov.Storage_aws {
|
||||||
|
name = "vol",
|
||||||
|
total = 30,
|
||||||
|
voldevice = "sdg",
|
||||||
|
labels = "{Key=storage,Value=vol1}",
|
||||||
|
parts = [
|
||||||
|
{ name = "home2", size = 15, type = "xfs" , mount = True, mount_path = "/home2" }
|
||||||
|
{ name = "other", size = 15, type = "ext4" , mount = True, mount_path = "/others" }
|
||||||
|
]
|
||||||
|
},
|
||||||
|
]
|
||||||
|
# Labels to describe the server in `key = "value` format, multiple can be declared.
|
||||||
|
# Usage = "env = "dev
|
||||||
|
labels = "{Key=Use,Value=lab-cp-0}"
|
||||||
|
# To use private network it a VPC + Subnet + NetworkInfterface has to be created
|
||||||
|
# IP will be assign here
|
||||||
|
network_private_ip = "10.11.2.11"
|
||||||
|
liveness_ip = "$network_public_ip"
|
||||||
|
liveness_port = 22
|
||||||
|
extra_hostnames = [ "lab-cp-0" ]
|
||||||
|
taskservs = [
|
||||||
|
{ name = "os", profile = "controlpanel"},
|
||||||
|
{ name = "kubernetes" },
|
||||||
|
{ name = "rook-ceph" },
|
||||||
|
#{ name = "kubernetes/kubeconfig", profile = "kubeconfig", install_mode = "getfile" },
|
||||||
|
{ name = "external-nfs" },
|
||||||
|
]
|
||||||
|
},
|
||||||
202
providers/aws/kcl/docs/aws_prov.md
Normal file
202
providers/aws/kcl/docs/aws_prov.md
Normal file
@ -0,0 +1,202 @@
|
|||||||
|
# aws_prov
|
||||||
|
|
||||||
|
## Index
|
||||||
|
|
||||||
|
- [Permission](#permission)
|
||||||
|
- [Provision_aws](#provision_aws)
|
||||||
|
- [Provision_env_aws](#provision_env_aws)
|
||||||
|
- [ReqPlan](#reqplan)
|
||||||
|
- [SecurityGroup](#securitygroup)
|
||||||
|
- [ServerDefaults_aws](#serverdefaults_aws)
|
||||||
|
- [Server_aws](#server_aws)
|
||||||
|
- [Storage_aws](#storage_aws)
|
||||||
|
|
||||||
|
## Schemas
|
||||||
|
|
||||||
|
### Permission
|
||||||
|
|
||||||
|
Permisssion for Security Groups
|
||||||
|
|
||||||
|
#### Attributes
|
||||||
|
|
||||||
|
| name | type | description | default value |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
|**fromPort** `required`|int|||
|
||||||
|
|**name** `required`|str|||
|
||||||
|
|**protocol** `required`|"tcp" | "udp"||"tcp"|
|
||||||
|
|**ranges** `required`|str||"[{CidrIp=0.0.0.0/0},{CidrIp=10.0.0.0/24}]"|
|
||||||
|
|**toPort** `required`|int|||
|
||||||
|
### Provision_aws
|
||||||
|
|
||||||
|
AWS provision data settings
|
||||||
|
|
||||||
|
#### Attributes
|
||||||
|
|
||||||
|
| name | type | description | default value |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
|**main** `required`|[Provision_env_aws](#provision_env_aws)|||
|
||||||
|
|**priv**|[Provision_env_aws](#provision_env_aws)|||
|
||||||
|
### Provision_env_aws
|
||||||
|
|
||||||
|
AWS provision env data settings
|
||||||
|
|
||||||
|
#### Attributes
|
||||||
|
|
||||||
|
| name | type | description | default value |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
|**avail_zone** `required`|str|||
|
||||||
|
|**cidr_block** `required`|str|||
|
||||||
|
|**sg**|[SecurityGroup](#securitygroup)|||
|
||||||
|
|**subnet** `required`|str|||
|
||||||
|
|**vpc** `required`|str|||
|
||||||
|
### ReqPlan
|
||||||
|
|
||||||
|
RequiredPlan settings
|
||||||
|
|
||||||
|
#### Attributes
|
||||||
|
|
||||||
|
| name | type | description | default value |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
|**arch** `required`|"x86_64" | "arm64"||"x86_64"|
|
||||||
|
|**cores** `required`|int||1|
|
||||||
|
|**ena** `required`|str||"supported,required"|
|
||||||
|
|**gen**|str||"current"|
|
||||||
|
|**infaces** `required`|int||2|
|
||||||
|
|**memory** `required`|int|||
|
||||||
|
|**scale** `required`|bool||True|
|
||||||
|
### SecurityGroup
|
||||||
|
|
||||||
|
Security Groups
|
||||||
|
|
||||||
|
#### Attributes
|
||||||
|
|
||||||
|
| name | type | description | default value |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
|**id**|str|||
|
||||||
|
|**name** `required`|str|||
|
||||||
|
|**perms**|[[Permission](#permission)]|||
|
||||||
|
### ServerDefaults_aws
|
||||||
|
|
||||||
|
Server Defaults settings
|
||||||
|
|
||||||
|
#### Attributes
|
||||||
|
|
||||||
|
| name | type | description | default value |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
|**disable_stop** `required`|bool||True|
|
||||||
|
|**domains_search**|str|||
|
||||||
|
|**fix_local_hosts** `required`|bool||True|
|
||||||
|
|**installer_user**|str||"${user}"|
|
||||||
|
|**labels** `required`|str||"{Key=cluster,Value=k8s}"|
|
||||||
|
|**liveness_ip**|str|||
|
||||||
|
|**liveness_port** `required`|int||22|
|
||||||
|
|**lock** `required`|bool||False|
|
||||||
|
|**main_domain**|str|||
|
||||||
|
|**network_private_id**|str|||
|
||||||
|
|**network_private_name**|str|||
|
||||||
|
|**network_public_ip**|str|||
|
||||||
|
|**network_public_ipv4**|bool||True|
|
||||||
|
|**network_public_ipv6**|bool||False|
|
||||||
|
|**network_utility_ipv4** `required`|bool||True|
|
||||||
|
|**network_utility_ipv6** `required`|bool||False|
|
||||||
|
|**not_use** `required`|bool||False|
|
||||||
|
|**plan**|str|||
|
||||||
|
|**primary_dns**|str|||
|
||||||
|
|**priv_cidr_block**|str|||
|
||||||
|
|**prov_settings** `required`|str||"defs/aws_data.k"|
|
||||||
|
|**prov_settings_clean** `required`|bool||False|
|
||||||
|
|**provider** `required` `readOnly`|"aws"||"aws"|
|
||||||
|
|**reqplan**|[ReqPlan](#reqplan)|||
|
||||||
|
|**running_timeout** `required`|int||200|
|
||||||
|
|**running_wait** `required`|int||10|
|
||||||
|
|**scale**|[ScaleResource](#scaleresource)|||
|
||||||
|
|**secondary_dns**|str|||
|
||||||
|
|**ssh_key_name**|str|||
|
||||||
|
|**ssh_key_path**|str|||
|
||||||
|
|**storage_os**|str|||
|
||||||
|
|**storage_os_find** `required`|str||"name: debian-12 \| arch: x86_64"|
|
||||||
|
|**storages**|[[Storage_aws](#storage_aws)]|||
|
||||||
|
|**time_zone** `required`|str||"UTC"|
|
||||||
|
|**user** `required`|str||"admin"|
|
||||||
|
|**user_home**|str||"/home/${user}"|
|
||||||
|
|**user_ssh_key_path**|str|||
|
||||||
|
|**user_ssh_port**|int||22|
|
||||||
|
|**zone**|str|||
|
||||||
|
### Server_aws
|
||||||
|
|
||||||
|
AWS server settings
|
||||||
|
|
||||||
|
#### Attributes
|
||||||
|
|
||||||
|
| name | type | description | default value |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
|**clusters**|[[ClusterDef](#clusterdef)]|||
|
||||||
|
|**disable_stop** `required`|bool||True|
|
||||||
|
|**domains_search**|str|||
|
||||||
|
|**extra_hostnames**|[str]|||
|
||||||
|
|**fix_local_hosts** `required`|bool||True|
|
||||||
|
|**hostname** `required`|str|||
|
||||||
|
|**installer_user**|str||"${user}"|
|
||||||
|
|**labels** `required`|str||"{Key=cluster,Value=k8s}"|
|
||||||
|
|**liveness_ip**|str|||
|
||||||
|
|**liveness_port** `required`|int||22|
|
||||||
|
|**lock** `required`|bool||False|
|
||||||
|
|**main_domain**|str|||
|
||||||
|
|**network_private_id**|str|||
|
||||||
|
|**network_private_ip**|str|||
|
||||||
|
|**network_private_name**|str|||
|
||||||
|
|**network_public_ip**|str|||
|
||||||
|
|**network_public_ipv4**|bool||True|
|
||||||
|
|**network_public_ipv6**|bool||False|
|
||||||
|
|**network_utility_ipv4** `required`|bool||True|
|
||||||
|
|**network_utility_ipv6** `required`|bool||False|
|
||||||
|
|**not_use** `required`|bool||False|
|
||||||
|
|**plan**|str|||
|
||||||
|
|**primary_dns**|str|||
|
||||||
|
|**priv_cidr_block**|str|||
|
||||||
|
|**prov_settings** `required`|str||"defs/aws_data.k"|
|
||||||
|
|**prov_settings_clean** `required`|bool||False|
|
||||||
|
|**provider** `required` `readOnly`|"aws"||"aws"|
|
||||||
|
|**reqplan**|[ReqPlan](#reqplan)|||
|
||||||
|
|**running_timeout** `required`|int||200|
|
||||||
|
|**running_wait** `required`|int||10|
|
||||||
|
|**scale**|[ScaleResource](#scaleresource)|||
|
||||||
|
|**secondary_dns**|str|||
|
||||||
|
|**ssh_key_name**|str|||
|
||||||
|
|**ssh_key_path**|str|||
|
||||||
|
|**storage_os**|str|||
|
||||||
|
|**storage_os_find** `required`|str||"name: debian-12 \| arch: x86_64"|
|
||||||
|
|**storages**|[[Storage_aws](#storage_aws)]|||
|
||||||
|
|**taskservs**|[[TaskServDef](#taskservdef)]|||
|
||||||
|
|**time_zone** `required`|str||"UTC"|
|
||||||
|
|**title** `required`|str|||
|
||||||
|
|**user** `required`|str||"admin"|
|
||||||
|
|**user_home**|str||"/home/${user}"|
|
||||||
|
|**user_ssh_key_path**|str|||
|
||||||
|
|**user_ssh_port**|int||22|
|
||||||
|
|**zone**|str|||
|
||||||
|
### Storage_aws
|
||||||
|
|
||||||
|
AWS Storage settings
|
||||||
|
|
||||||
|
#### Attributes
|
||||||
|
|
||||||
|
| name | type | description | default value |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
|**deletetermination** `required`|bool||False|
|
||||||
|
|**encrypted** `required`|bool||False|
|
||||||
|
|**fstab** `required`|bool||True|
|
||||||
|
|**kms_id** `required`|str||""|
|
||||||
|
|**labels** `required`|str||""|
|
||||||
|
|**mount** `required`|bool||True|
|
||||||
|
|**mount_path**|str|||
|
||||||
|
|**name** `required`|str|||
|
||||||
|
|**parts**|[[StorageVol](#storagevol)]||[]|
|
||||||
|
|**size** `required`|int||0|
|
||||||
|
|**total** `required`|int||size|
|
||||||
|
|**type** `required`|"ext4" | "xfs" | "btrfs" | "raw" | "zfs"||"ext4"|
|
||||||
|
|**voldevice** `required`|str||"sdf"|
|
||||||
|
|**volname** `required`|str||""|
|
||||||
|
|**voltype** `required`|"standard" | "io1" | "io2" | "gp2" | "sc1" | "st1" | "gp3"||"gp2"|
|
||||||
|
|**zone** `required`|str||""|
|
||||||
|
<!-- Auto generated by kcl-doc tool, please do not edit. -->
|
||||||
13
providers/aws/kcl/kcl.mod
Normal file
13
providers/aws/kcl/kcl.mod
Normal file
@ -0,0 +1,13 @@
|
|||||||
|
# Info: KCL AWS provider module for provisioning (Provisioning)
|
||||||
|
# Author: JesusPerezLorenzo
|
||||||
|
# Release: 0.0.4
|
||||||
|
# Date: 15-12-2023
|
||||||
|
|
||||||
|
[package]
|
||||||
|
name = "aws_prov"
|
||||||
|
edition = "0.0.1"
|
||||||
|
version = "0.0.1"
|
||||||
|
|
||||||
|
[dependencies]
|
||||||
|
provisioning = { path = "../../../../kcl", version = "0.0.1" }
|
||||||
|
providers = { path = "../..", version = "0.0.1" }
|
||||||
10
providers/aws/kcl/kcl.mod.lock
Normal file
10
providers/aws/kcl/kcl.mod.lock
Normal file
@ -0,0 +1,10 @@
|
|||||||
|
[dependencies]
|
||||||
|
[dependencies.providers]
|
||||||
|
name = "providers"
|
||||||
|
full_name = "vPkg_605fa997-06b6-4821-a734-78c1fc3f52ec_0.0.1"
|
||||||
|
version = "0.0.1"
|
||||||
|
[dependencies.provisioning]
|
||||||
|
name = "provisioning"
|
||||||
|
full_name = "provisioning_0.0.1"
|
||||||
|
version = "0.0.1"
|
||||||
|
sum = "KuzJ0xi0LEoVci/EHDA9JY9oTuQ5ByHnZGdTXR4ww3U="
|
||||||
92
providers/aws/nulib/aws/cache.nu
Normal file
92
providers/aws/nulib/aws/cache.nu
Normal file
@ -0,0 +1,92 @@
|
|||||||
|
#!/usr/bin/env nu
|
||||||
|
# Info: AWS
|
||||||
|
|
||||||
|
use lib.nu *
|
||||||
|
use ../../../../../core/nulib/lib_provisioning/config/accessor.nu *
|
||||||
|
|
||||||
|
export def aws_start_cache_info [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
( $"# To start from scratch set 'vpc' 'subnet' 'sg.id' to '?' then new AWS settings will be collected. This will create 'sg.perms'.\n" +
|
||||||
|
$"# Removing 'provider_path' and 'defs/aws_data.k' would fallback to defaults with no settings for 'sg.name' and 'sg.perms', etc.\n"
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
export def aws_create_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
] {
|
||||||
|
if $settings == null {
|
||||||
|
if (is-debug-enabled) { print $"❗ No settings found " }
|
||||||
|
return
|
||||||
|
}
|
||||||
|
let provider_path = (get_provider_data_path $settings $server)
|
||||||
|
#use lib_provisioning/utils/settings.nu load_provider_env
|
||||||
|
let data = (load_provider_env $settings $server $provider_path)
|
||||||
|
if ($data | is-empty) or ($data | get -o main | get -o vpc) == "?" {
|
||||||
|
aws_scan_settings "create" $provider_path $settings $server false
|
||||||
|
let new_data = (load_provider_env $settings $server $provider_path)
|
||||||
|
if ($new_data | is-empty) or ($new_data | get -o main | get -o vpc) == "?" {
|
||||||
|
print $"❗AWS no valid provider settings for (_ansi red)($server.hostname)(_ansi reset)"
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
if (is-debug-enabled) {
|
||||||
|
print $"aws main data already exists in ($provider_path | path basename)"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
aws_scan_servers $provider_path $settings $server
|
||||||
|
if (is-debug-enabled) { print $"Cache for ($server.provider) on ($server.hostname) saved in: ($provider_path | path basename)" }
|
||||||
|
# load_provider_env $settings $server $provider_path
|
||||||
|
}
|
||||||
|
export def aws_read_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
] {
|
||||||
|
if $settings == null {
|
||||||
|
print $"❗ No settings found "
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_clean_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
] {
|
||||||
|
if $settings == null {
|
||||||
|
print $"❗ No settings found "
|
||||||
|
return
|
||||||
|
}
|
||||||
|
let provider_path = (get_provider_data_path $settings $server)
|
||||||
|
let data = if ($provider_path | path exists) {
|
||||||
|
open $provider_path
|
||||||
|
} else {
|
||||||
|
{ servers: null }
|
||||||
|
}
|
||||||
|
if ($data.servers? != null) and ($data.servers | where {|it| ($it.hostname? | default "") == $server.hostname} | length) == 0 {
|
||||||
|
if (is-debug-enabled) {
|
||||||
|
print $"❗server ($server.hostname) already deleted from ($provider_path | path basename)"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let all_servers = ( $data.servers? | default [] | where {|it| $it.hostname != $server.hostname})
|
||||||
|
if (is-debug-enabled) { print $"Cache for ($server.provider) delete ($server.hostname) in: ($provider_path | path basename)" }
|
||||||
|
let new_data = if ($all_servers | length) == 0 {
|
||||||
|
aws_delete_settings "all" $provider_path $settings $server
|
||||||
|
{}
|
||||||
|
} else {
|
||||||
|
( $data | merge { servers: $all_servers})
|
||||||
|
}
|
||||||
|
save_provider_env $new_data $settings $provider_path
|
||||||
|
}
|
||||||
|
export def aws_ip_from_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
] {
|
||||||
|
let prov_settings = ($settings.providers | find $server.provider ) #| get -o settings)
|
||||||
|
if ($prov_settings | is-empty) == null { return "" }
|
||||||
|
($prov_settings | flatten | find $server.hostname | select -o ip_addresses | find "public"| get -o address | get -o 0 | default "")
|
||||||
|
}
|
||||||
92
providers/aws/nulib/aws/cache.nu-e
Normal file
92
providers/aws/nulib/aws/cache.nu-e
Normal file
@ -0,0 +1,92 @@
|
|||||||
|
#!/usr/bin/env nu
|
||||||
|
# Info: AWS
|
||||||
|
|
||||||
|
use lib.nu *
|
||||||
|
use ../../../../core/nulib/lib_provisioning/config/accessor.nu *
|
||||||
|
|
||||||
|
export def aws_start_cache_info [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
( $"# To start from scratch set 'vpc' 'subnet' 'sg.id' to '?' then new AWS settings will be collected. This will create 'sg.perms'.\n" +
|
||||||
|
$"# Removing 'provider_path' and 'defs/aws_data.k' would fallback to defaults with no settings for 'sg.name' and 'sg.perms', etc.\n"
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
export def aws_create_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
] {
|
||||||
|
if $settings == null {
|
||||||
|
if (is-debug-enabled) { print $"❗ No settings found " }
|
||||||
|
return
|
||||||
|
}
|
||||||
|
let provider_path = (get_provider_data_path $settings $server)
|
||||||
|
#use lib_provisioning/utils/settings.nu load_provider_env
|
||||||
|
let data = (load_provider_env $settings $server $provider_path)
|
||||||
|
if ($data | is-empty) or ($data | get -o main | get -o vpc) == "?" {
|
||||||
|
aws_scan_settings "create" $provider_path $settings $server false
|
||||||
|
let new_data = (load_provider_env $settings $server $provider_path)
|
||||||
|
if ($new_data | is-empty) or ($new_data | get -o main | get -o vpc) == "?" {
|
||||||
|
print $"❗AWS no valid provider settings for (_ansi red)($server.hostname)(_ansi reset)"
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
if (is-debug-enabled) {
|
||||||
|
print $"aws main data already exists in ($provider_path | path basename)"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
aws_scan_servers $provider_path $settings $server
|
||||||
|
if (is-debug-enabled) { print $"Cache for ($server.provider) on ($server.hostname) saved in: ($provider_path | path basename)" }
|
||||||
|
# load_provider_env $settings $server $provider_path
|
||||||
|
}
|
||||||
|
export def aws_read_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
] {
|
||||||
|
if $settings == null {
|
||||||
|
print $"❗ No settings found "
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_clean_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
] {
|
||||||
|
if $settings == null {
|
||||||
|
print $"❗ No settings found "
|
||||||
|
return
|
||||||
|
}
|
||||||
|
let provider_path = (get_provider_data_path $settings $server)
|
||||||
|
let data = if ($provider_path | path exists) {
|
||||||
|
open $provider_path
|
||||||
|
} else {
|
||||||
|
{ servers: null }
|
||||||
|
}
|
||||||
|
if ($data.servers? != null) and ($data.servers | where {|it| ($it.hostname? | default "") == $server.hostname} | length) == 0 {
|
||||||
|
if (is-debug-enabled) {
|
||||||
|
print $"❗server ($server.hostname) already deleted from ($provider_path | path basename)"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let all_servers = ( $data.servers? | default [] | where {|it| $it.hostname != $server.hostname})
|
||||||
|
if (is-debug-enabled) { print $"Cache for ($server.provider) delete ($server.hostname) in: ($provider_path | path basename)" }
|
||||||
|
let new_data = if ($all_servers | length) == 0 {
|
||||||
|
aws_delete_settings "all" $provider_path $settings $server
|
||||||
|
{}
|
||||||
|
} else {
|
||||||
|
( $data | merge { servers: $all_servers})
|
||||||
|
}
|
||||||
|
save_provider_env $new_data $settings $provider_path
|
||||||
|
}
|
||||||
|
export def aws_ip_from_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
] {
|
||||||
|
let prov_settings = ($settings.providers | find $server.provider ) #| get -o settings)
|
||||||
|
if ($prov_settings | is-empty) == null { return "" }
|
||||||
|
($prov_settings | flatten | find $server.hostname | select -o ip_addresses | find "public"| get -o address | get -o 0 | default "")
|
||||||
|
}
|
||||||
8
providers/aws/nulib/aws/env.nu
Normal file
8
providers/aws/nulib/aws/env.nu
Normal file
@ -0,0 +1,8 @@
|
|||||||
|
export-env {
|
||||||
|
use ../../../../../core/nulib/lib_provisioning/config/accessor.nu [get-provider-api-url get-provider-auth get-provider-interface]
|
||||||
|
|
||||||
|
# Load AWS configuration from config system
|
||||||
|
$env.AWS_API_URL = (get-provider-api-url "aws")
|
||||||
|
$env.AWS_AUTH = (get-provider-auth "aws")
|
||||||
|
$env.AWS_INTERFACE = (get-provider-interface "aws")
|
||||||
|
}
|
||||||
8
providers/aws/nulib/aws/env.nu-e
Normal file
8
providers/aws/nulib/aws/env.nu-e
Normal file
@ -0,0 +1,8 @@
|
|||||||
|
export-env {
|
||||||
|
use ../../../../core/nulib/lib_provisioning/config/accessor.nu [get-provider-api-url get-provider-auth get-provider-interface]
|
||||||
|
|
||||||
|
# Load AWS configuration from config system
|
||||||
|
$env.AWS_API_URL = (get-provider-api-url "aws")
|
||||||
|
$env.AWS_AUTH = (get-provider-auth "aws")
|
||||||
|
$env.AWS_INTERFACE = (get-provider-interface "aws")
|
||||||
|
}
|
||||||
716
providers/aws/nulib/aws/lib.nu
Normal file
716
providers/aws/nulib/aws/lib.nu
Normal file
@ -0,0 +1,716 @@
|
|||||||
|
#!/usr/bin/env nu
|
||||||
|
# Info: Script to create/delete AWS resources from file settings in bash with template/vars
|
||||||
|
# Author: JesusPerez
|
||||||
|
# Release: 1.0
|
||||||
|
# Date: 26-03-2024
|
||||||
|
|
||||||
|
use ../../../../../core/nulib/lib_provisioning/utils/templates.nu run_from_template
|
||||||
|
use ../../../../../core/nulib/lib_provisioning/config/accessor.nu *
|
||||||
|
|
||||||
|
export def aws_review_credentials [
|
||||||
|
] {
|
||||||
|
print $"❗AWS credentials not found for '$PROVIDER_CLI_CMD' command."
|
||||||
|
print $" Use default profile or env AWS_PROFILE from $HOME/.aws/credentials path or environment variables for settings"
|
||||||
|
print $" More info: "
|
||||||
|
print $" Profile mode: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html"
|
||||||
|
print $" Evironment mode: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html"
|
||||||
|
}
|
||||||
|
export def aws_check_region [
|
||||||
|
zone: string
|
||||||
|
] {
|
||||||
|
if ($zone |is-empty) {
|
||||||
|
print $"❗AWS region zone ($env.AWS_DEFAULT_REGION) not found for '$PROVIDER_CLI_CMD' command."
|
||||||
|
print $"Use set default profile or use env AWS_PROFILE with $HOME/.aws/credentials path or environment variables for settings"
|
||||||
|
}
|
||||||
|
(^aws ec2 describe-availability-zones --region $zone | complete).exit_code
|
||||||
|
}
|
||||||
|
export def aws_get_plan_info [
|
||||||
|
var: string
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
let plan = ($server | get -o $var | default "")
|
||||||
|
if ($plan | is-mepty) { return }
|
||||||
|
let res = (^aws ec2 describe-instance-types --instance-types $plan
|
||||||
|
--query 'InstanceTypes[].{ type: InstanceType, cores: VCpuInfo.DefaultCores, memory: MemoryInfo.SizeInMiB, arch: ProcessorInfo.SupportedArchitectures, gen: CurrentGeneration, infaces: NetworkInfo.MaximumNetworkInterfaces, ena: NetworkInfo.EnaSupport }'
|
||||||
|
--out=json )
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
($res.stdout | from json | get -o 0 | default "")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_find_plan [
|
||||||
|
var: string
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
let reqplan = ($server | get -o $var | default "")
|
||||||
|
if ($reqplan | is-mepty) {
|
||||||
|
print $"❗No reqplan found in settings for ($var)"
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
let res = (^ aws ec2 describe-instance-types --filters $"Name=processor-info.supported-architecture,Values=($reqplan.arch | default '')"
|
||||||
|
$"Name=memory-info.size-in-mib,Values=($reqplan.memory | default '')"
|
||||||
|
$"Name=vcpu-info.default-cores,Values=($reqplan.cores | default '')"
|
||||||
|
$"Name=network-info.maximum-network-interfaces,Values=($reqplan.infaces | default '')"
|
||||||
|
$"Name=network-info.ena-support,Values=($reqplan.ena | default '')"
|
||||||
|
--query 'InstanceTypes[].{ type: InstanceType, cores: VCpuInfo.DefaultCores, memory: MemoryInfo.SizeInMiB, arch: ProcessorInfo.SupportedArchitectures, gen: CurrentGeneration, infaces: NetworkInfo.MaximumNetworkInterfaces, ena: NetworkInfo.EnaSupport }'
|
||||||
|
--output json
|
||||||
|
)
|
||||||
|
if ($res.exit_code == 0) {
|
||||||
|
($res.stdout | from json)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_compare_plan_reqplan [
|
||||||
|
var_plan: string
|
||||||
|
var_reqplan: string
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
let plan = ($server | get -o $var_plan)
|
||||||
|
let check_plan = (aws_get_plan_info $var_plan $server)
|
||||||
|
let reqplan = ($server | get -o $var_reqplan)
|
||||||
|
|
||||||
|
if ($plan | is-empty) or ( $check_plan | is-empty) {
|
||||||
|
print $"❗No valid $plan found for $var_plan in $AWS_DEFAULT_REGION"
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
if ($reqplan | is-empty) { return }
|
||||||
|
|
||||||
|
let plan_memory = ($check_plan | get -o memory | default "")
|
||||||
|
let reqplan_memory = ($reqplan| get -o memory | default "")
|
||||||
|
if $plan_memory != $reqplan_memory {
|
||||||
|
print $"❗$plan memory does not match plan: $plan_memory expected $reqplan_memory"
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
let plan_cores = ($check_plan | get -o cores | default "")
|
||||||
|
let reqplan_cores = ($reqplan | get -o cores | default "")
|
||||||
|
if $plan_cores != $reqplan_cores {
|
||||||
|
print $"❗($plan) cores does not match plan: ($plan_cores) expected ($reqplan_cores)"
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
let plan_archs = ($check_plan | get -o arch | default "")
|
||||||
|
let reqplan_archs = ($reqplan | get -o arch | default "")
|
||||||
|
if not ($plan_archs | str contains $reqplan_archs ) {
|
||||||
|
print $"❗($plan) architectures does not match plan: ($plan_archs) expected ($reqplan_archs)"
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
let plan_infaces = ($check_plan | get -o infaces | default "")
|
||||||
|
let reqplan_infaces = ($reqplan | get -o infaces | default "")
|
||||||
|
if $plan_infaces < $reqplan_infaces {
|
||||||
|
print $"❗($plan) interfaces number does not match plan: ($plan_infaces) expected ($reqplan_infaces)"
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
0
|
||||||
|
}
|
||||||
|
export def aws_get_os_image [
|
||||||
|
name: string
|
||||||
|
arch: string
|
||||||
|
] {
|
||||||
|
let res = (^aws ec2 describe-images --owners amazon --filters $"'Name=name,Values=*'($name)'*'" $"'Name=architecture,Values=*'($arch)'*'"
|
||||||
|
--query 'reverse(sort_by(Images,&CreationDate))[:5].{id:ImageId, name: Name, date:CreationDate}[0]' --output json
|
||||||
|
)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
($res.stdout | from json)
|
||||||
|
} else { "" }
|
||||||
|
}
|
||||||
|
export def aws_delete_private_vpcs [
|
||||||
|
aws_priv_cidr_block: string
|
||||||
|
] {
|
||||||
|
let res = (^aws ec2 describe-vpcs --query Vpcs --out json | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
for it in ($res.stdout | from json | where {|it| $it.CidrBlock == $aws_priv_cidr_block } | get -o VpcId | default []) {
|
||||||
|
print $"delete vpc id ($it)"
|
||||||
|
^aws ec2 delete-vpc --vpc-id "$it"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_create_private_vpc [
|
||||||
|
aws_priv_cidr_block: string
|
||||||
|
task: string
|
||||||
|
] {
|
||||||
|
let res = (^aws ec2 describe-vpcs --query Vpcs --out json | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-empty) {
|
||||||
|
print $"❗Error ($task) vpcs ($aws_priv_cidr_block) "
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
let aws_priv_vpc = ($res.stdout | from json | where {|it| $it.CidrBlock == $aws_priv_cidr_block } | get -o 0 | get -o VpcId | default "")
|
||||||
|
match $task {
|
||||||
|
"create" => {
|
||||||
|
if ($aws_priv_vpc | is-not-empty) {
|
||||||
|
print $"Clean up VPC ($aws_priv_vpc)..."
|
||||||
|
let res = (^aws ec2 delete-vpc --vpc-id $aws_priv_vpc err> /dev/null | complete )
|
||||||
|
if $res.exit_code != 0 {
|
||||||
|
print $"vpc ($aws_priv_vpc) delete error ($res.exit_code) ($res.stdout)"
|
||||||
|
return $aws_priv_vpc
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let res = (^aws ec2 create-vpc --cidr-block $aws_priv_cidr_block --query Vpc.VpcId --output text | complete)
|
||||||
|
if $res.exit_code == 0 {
|
||||||
|
($res.stdout | str replace '"' '')
|
||||||
|
} else {
|
||||||
|
print $"❗ Error ($task) priv_vpc for ($aws_priv_cidr_block)"
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
$aws_priv_vpc
|
||||||
|
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_delete_sgs_by_name [
|
||||||
|
aws_sg_name: string
|
||||||
|
] {
|
||||||
|
let res = (^aws ec2 describe-security-groups --query SecurityGroups --out json | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
let aws_sg_id = ($res.stdout | from json | where {|it| $it.GroupName == $aws_sg_name } | get -o GroupId | default "")
|
||||||
|
if ($aws_sg_id | is-not-empty) {
|
||||||
|
print $"Clean up SGs ($aws_sg_name)"
|
||||||
|
^aws ec2 delete-security-group --group-id $aws_sg_id
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_delete_sgs [
|
||||||
|
aws_vpc: string
|
||||||
|
] {
|
||||||
|
let res = (^aws ec2 describe-security-groups --query SecurityGroups --out json)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
for it in ($res.stdout | from json | where {|it| $it.VpcId == $aws_vpc } | where {|it| $it.GroupName != "default" } | get -o GroupId | default "") {
|
||||||
|
print $"delete security group id ($it)"
|
||||||
|
^aws ec2 delete-security-group --group-id $it
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_create_sg_id [
|
||||||
|
aws_vpc: string
|
||||||
|
aws_sg_name: string
|
||||||
|
task: string
|
||||||
|
] {
|
||||||
|
let res = (^aws ec2 describe-security-groups --query SecurityGroups --out json | complete)
|
||||||
|
if $res.exit_code != 0 or ($res.stdout | is-empty) {
|
||||||
|
print $"❗Error ($task) sg_id for ($aws_sg_name) in ($aws_vpc)"
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
let aws_sg_id = ($res.stdout | from json | where {|it| $it.VpcId == $aws_vpc and $it.GroupName == $aws_sg_name } |
|
||||||
|
get -o 0 | get -o GroupId | default "")
|
||||||
|
match $task {
|
||||||
|
"create" => {
|
||||||
|
if ($aws_sg_id | is-not-empty) {
|
||||||
|
print $"Clean up sg ($aws_sg_id) ..."
|
||||||
|
let res = (^aws ec2 delete-security-group --group-id $aws_sg_id | complete)
|
||||||
|
if $res.exit_code != 0 {
|
||||||
|
print $"❗Error delete ($aws_sg_id) for ($aws_sg_name) in ($aws_vpc)"
|
||||||
|
return $aws_sg_id
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let res = (^aws ec2 create-security-group --group-name $aws_sg_name --description $"Group ($aws_sg_name)"
|
||||||
|
--tag-specifications $"ResourceType=security-group,Tags=[{Key=Name,Value=($aws_sg_name)}]"
|
||||||
|
--vpc-id ($aws_vpc | str trim) --out json | complete )
|
||||||
|
if $res.exit_code == 0 {
|
||||||
|
($res.stdout | from json | get -o GroupId | default "")
|
||||||
|
} else {
|
||||||
|
print $"❗Error ($task) sg_id for ($aws_sg_name) in ($aws_vpc)"
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
$aws_sg_id
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_add_sg_perms [
|
||||||
|
sg_data: record
|
||||||
|
server: record
|
||||||
|
check_mode: bool
|
||||||
|
] {
|
||||||
|
let perms = ($sg_data | get -o perms | default [])
|
||||||
|
if ($perms | is-empty) { return }
|
||||||
|
let res = (^aws ec2 describe-security-groups --group-id $sg_data.id --query SecurityGroups[].IpPermissions --out json | complete)
|
||||||
|
let curr_sg_perms = if $res.exit_code == 0 {
|
||||||
|
($res.stdout | from json | get -o 0 | default [])
|
||||||
|
} else { [] }
|
||||||
|
mut curr_perms = []
|
||||||
|
for p in $curr_sg_perms {
|
||||||
|
mut ranges = ""
|
||||||
|
for rng in ($p | get -o IpRanges) {
|
||||||
|
if ($ranges | is-not-empty) { $ranges = $"($ranges),"}
|
||||||
|
$ranges = $"($ranges){CidrIp=($rng.CidrIp)}"
|
||||||
|
}
|
||||||
|
let protocol = ($p | get -o IpProtocol | default "")
|
||||||
|
let from_port = ($p | get -o FromPort | default "")
|
||||||
|
let to_port = ($p | get -o ToPort | default "")
|
||||||
|
for it in $perms {
|
||||||
|
if ($protocol == ($it | get -o protocol ) and $from_port == ($it | get -o fromPort ) and
|
||||||
|
$to_port == ($it | get -o toPort ) and
|
||||||
|
$ranges == ($it | get -o ranges | str replace "[" "" | str replace "]" "" )) {
|
||||||
|
} else {
|
||||||
|
$curr_perms = ($curr_perms | append $p)
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if ($curr_perms == $curr_sg_perms) and ($curr_perms| length) == ($perms | length) { return }
|
||||||
|
if ($perms == $curr_perms) { return }
|
||||||
|
if (is-debug-enabled) {
|
||||||
|
print $"(_ansi green)current sg perms(_ansi reset) ($curr_perms | table -e)"
|
||||||
|
}
|
||||||
|
let wk_format = if (get-provisioning-wk-format) == "json" { "json" } else { "yaml" }
|
||||||
|
let wk_vars = ( "/tmp/" | path join (mktemp --tmpdir-path "/tmp" --suffix $".($wk_format)" | path basename))
|
||||||
|
let data = { sg_name: $sg_data.name, sg_id: $sg_data.id, perms: $perms, curr_perms: $curr_perms }
|
||||||
|
if $wk_format == "json" {
|
||||||
|
$data | to json | save --force $wk_vars
|
||||||
|
} else {
|
||||||
|
$data | to yaml | save --force $wk_vars
|
||||||
|
}
|
||||||
|
let run_file = ("/tmp" | path join $"onaws_run_sg_(mktemp --tmpdir-path "/tmp" --suffix ".sh" | path basename | str replace 'tmp.' '' )")
|
||||||
|
let sg_template = ((get-base-path) | path join "providers" | path join $server.provider | path join templates | path join "aws_sg.j2" )
|
||||||
|
if not ($sg_template | path exists) {
|
||||||
|
print $"❗($sg_template) not found for Security Groups ($sg_data.name)"
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
#use ../../../../../core/nulib/lib_provisioning/utils/templates.nu run_from_template
|
||||||
|
let res = if $check_mode {
|
||||||
|
run_from_template $sg_template $wk_vars $run_file --check_mode
|
||||||
|
} else {
|
||||||
|
run_from_template $sg_template $wk_vars $run_file
|
||||||
|
}
|
||||||
|
if $res {
|
||||||
|
if (is-debug-enabled) {
|
||||||
|
print $"(_ansi green)OK(_ansi reset) (_ansi green_bold)($sg_data.name)(_ansi reset)"
|
||||||
|
} else {
|
||||||
|
rm --force $wk_vars $run_file
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
print $"(_ansi red)Failed(_ansi reset) (_ansi green_bold)($sg_data.name)(_ansi reset)"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_delete_private_subnets [
|
||||||
|
aws_priv_vpc: string
|
||||||
|
aws_priv_cidr_block: string
|
||||||
|
] {
|
||||||
|
let res = (^aws ec2 describe-subnets --query Subnets --out json | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
for it in ($res.stdout | from json | where { |it| $it.VpcId == $aws_priv_vpc and $it.CidrBlock == $aws_priv_cidr_block } | get -o SubnetId | default []) {
|
||||||
|
print $"Clean up subnet ($it) in ($aws_priv_vpc)..."
|
||||||
|
let res = (^aws ec2 delete-subnet --subnet-id $it | complete)
|
||||||
|
if $res.exit_code != 0 { return false }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
true
|
||||||
|
}
|
||||||
|
export def aws_create_private_subnet [
|
||||||
|
aws_priv_cidr_block: string
|
||||||
|
aws_priv_vpc: string
|
||||||
|
aws_avail_zone: string
|
||||||
|
task: string
|
||||||
|
] {
|
||||||
|
match $task {
|
||||||
|
"create" => {
|
||||||
|
if not (aws_delete_private_subnets $aws_priv_vpc $aws_priv_cidr_block) {
|
||||||
|
let res = (^aws ec2 describe-subnets --query Subnets --out json | complete)
|
||||||
|
return ($res.stdout | from json | where { |it| $it.VpcId == $aws_priv_vpc and $it.CidrBlock == $aws_priv_cidr_block } | get -o 0)
|
||||||
|
}
|
||||||
|
let res = (^aws ec2 create-subnet --vpc-id $aws_priv_vpc --cidr-block $aws_priv_cidr_block --availability-zone $aws_avail_zone --query "Subnet" --output json | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
($res.stdout | from json)
|
||||||
|
} else {
|
||||||
|
print $"❗aws_priv_subnet not found for ($aws_priv_vpc) - ($aws_priv_cidr_block)"
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
let res = (^aws ec2 describe-subnets --query Subnets --out json | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
($res.stdout | from json | where { |it| $it.VpcId == $aws_priv_vpc and $it.CidrBlock == $aws_priv_cidr_block } | get -o 0)
|
||||||
|
} else {
|
||||||
|
{}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
def aws_vpc_subnet [
|
||||||
|
aws_avail_zone: string
|
||||||
|
aws_priv_subnet: string
|
||||||
|
task: string
|
||||||
|
] {
|
||||||
|
let res = (^aws ec2 describe-subnets --query Subnets --out json | complete)
|
||||||
|
let aws_vpc_subnet_data = if $res.exit_code == 0 {
|
||||||
|
let data = ($res.stdout | from json | where {|it| $it.AvailabilityZone == $aws_avail_zone and $it.SubnetId != $aws_priv_subnet } | get -o 0 )
|
||||||
|
{"vpc": $"($data | get -o VpcId | default '')", "subnet": $"($data | get -o SubnetId | default '')"}
|
||||||
|
} else {
|
||||||
|
{}
|
||||||
|
}
|
||||||
|
if $task == "create" and ($aws_vpc_subnet_data | is-empty) {
|
||||||
|
^aws ec2 create-default-subnet --availability-zone $aws_avail_zone
|
||||||
|
(aws_vpc_subnet $aws_avail_zone $aws_priv_subnet "scan")
|
||||||
|
} else {
|
||||||
|
$aws_vpc_subnet_data
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_delete_private_interfaces [
|
||||||
|
aws_priv_subnet: string
|
||||||
|
] {
|
||||||
|
let res = (^aws ec2 describe-network-interfaces --query NetworkInterfaces --out json | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
for it in ($res.stdout | from json | where {|it| $it.SubnetId == $aws_priv_subnet } | get -o NetworkInterfaceId | default []) {
|
||||||
|
^aws ec2 delete-network-interface --network-interface-id $it
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_delete_private_interface [
|
||||||
|
network_interface_id: string
|
||||||
|
] {
|
||||||
|
^aws ec2 delete-network-interface --network-interface-id "$network_interface_id"
|
||||||
|
}
|
||||||
|
export def aws_get_interface_defs [
|
||||||
|
ip_interface: string
|
||||||
|
aws_priv_subnet: string
|
||||||
|
] {
|
||||||
|
let res = (^aws ec2 describe-network-interfaces --query NetworkInterfaces --out json | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
($res.stdout | from json | where {|it| $it.SubnetId == $aws_priv_subnet and $it.PrivateIpAddress == $ip_interface } |
|
||||||
|
get -o 0 | get -o NetworkInterfaceId | default ""
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_get_create_private_interface [
|
||||||
|
ip_interface: string
|
||||||
|
aws_priv_subnet: string
|
||||||
|
] {
|
||||||
|
(aws_get_interface_defs $ip_interface $aws_priv_subnet)
|
||||||
|
}
|
||||||
|
export def aws_get_instance_defs [
|
||||||
|
instance: string
|
||||||
|
] {
|
||||||
|
let res = (^aws ec2 describe-instances --instance-ids $instance --out "json" | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
($res.stdout | from json | get -o "Reservations" | get -o "Instances" | default "" )
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def attach_private_interface [
|
||||||
|
interface: string
|
||||||
|
instance: string
|
||||||
|
aws_priv_subnet: string
|
||||||
|
] {
|
||||||
|
if (aws_get_instance_defs $instance | is-not-empty) and (aws_get_interface_defs $interface $aws_priv_subnet | is-not-empty) {
|
||||||
|
(^aws ec2 attach-network-interface --network-interface-id $interface --instance-id $instance --device-index 1)
|
||||||
|
} else { "" }
|
||||||
|
}
|
||||||
|
export def detach_private_interface [
|
||||||
|
interface: string
|
||||||
|
instance: string
|
||||||
|
aws_priv_subnet: string
|
||||||
|
] {
|
||||||
|
if (aws_get_instance_defs $instance | is-not-empty) and (aws_get_interface_defs $interface $aws_priv_subnet | is-not-empty) {
|
||||||
|
(^aws ec2 detach-network-interface --network-interface-id $interface --instance-id $instance)
|
||||||
|
} else { "" }
|
||||||
|
}
|
||||||
|
export def aws_delete_target [
|
||||||
|
target: string
|
||||||
|
target_id: string
|
||||||
|
settings: record
|
||||||
|
] {
|
||||||
|
mut num = 0
|
||||||
|
mut res = ""
|
||||||
|
mut status = ""
|
||||||
|
let val_timeout = if $settings.running_timeout? != null { $settings.running_timeout } else { 60 }
|
||||||
|
let wait = if $settings.running_wait? != null { $settings.running_wait } else { 10 }
|
||||||
|
let wait_duration = ($"($wait)sec"| into duration)
|
||||||
|
if (is-debug-enabled) { print -n $"Delete ($target) -> ($target_id) " }
|
||||||
|
while ($status | is-empty) {
|
||||||
|
let status = match $target {
|
||||||
|
"securityGroup" => (^aws ec2 describe-security-groups --group-id $target_id err> /dev/null),
|
||||||
|
"subnet" => (^aws ec2 describe-subnets --subnet-id $target_id err> /dev/null) ,
|
||||||
|
"vpcs" => (^aws ec2 describe-vpcs --vpc-id $target_id err> /dev/null) ,
|
||||||
|
"interface" => (^aws ec2 describe-network-interfaces --network-interface-id $target_id err> /dev/null),
|
||||||
|
}
|
||||||
|
if ($status | is-empty) {
|
||||||
|
print $" "
|
||||||
|
return
|
||||||
|
}
|
||||||
|
let res = match $target {
|
||||||
|
"securityGroup" => (^aws ec2 delete-security-group --group-id $target_id | complete).exit_code,
|
||||||
|
"subnet" => (^aws ec2 delete-subnet --subnet-id $target_id | complete).exit_code,
|
||||||
|
"vpcs" => (^aws ec2 delete-vpc --vpc-id $target_id | complete).exit_code,
|
||||||
|
"interface" => (^aws ec2 delete-network-interface --network-interface-id $target_id | complete).exit_code,
|
||||||
|
}
|
||||||
|
if ($res == 0) {
|
||||||
|
print $" "
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if ($status | is-not-empty) or ($res != 0 ) {
|
||||||
|
sleep $wait_duration
|
||||||
|
$num += $wait
|
||||||
|
if $val_timeout > 0 and $num > $val_timeout { return 1 }
|
||||||
|
print -n $"($num) "
|
||||||
|
}
|
||||||
|
}
|
||||||
|
print $""
|
||||||
|
}
|
||||||
|
export def aws_delete_settings [
|
||||||
|
target: string
|
||||||
|
provider_path: string
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
if not ($provider_path |path exists) {
|
||||||
|
print $"❗aws_settings not found ($provider_path) no delete settings "
|
||||||
|
return
|
||||||
|
}
|
||||||
|
let prov_settings = (load_provider_env $settings $server $provider_path)
|
||||||
|
let env_settings = (get_provider_env $settings $server)
|
||||||
|
if ($prov_settings | is-empty) or $prov_settings.main? == null or $prov_settings.priv? == null {
|
||||||
|
if (is-debug-enabled) { print $"❗aws_settings (_ansi yellow_bold)($provider_path | path basename)(_ansi reset) no settings main and priv found" }
|
||||||
|
return
|
||||||
|
}
|
||||||
|
let aws_priv_subnet = ($prov_settings.priv.subnet | default "")
|
||||||
|
let aws_priv_cidr_block = ($server.priv_cidr_block | default "")
|
||||||
|
print $"Scanning aws resources to clean from (_ansi yellow_bold)($provider_path | path basename)(_ansi reset) ... ($aws_priv_cidr_block)"
|
||||||
|
if $target == "all" or $target == "interface" {
|
||||||
|
for server_info in ($settings.data.servers) {
|
||||||
|
let server = ($server_info | get -o hostname | default "")
|
||||||
|
let network_private_ip = ($server_info | get -o network_private_ip | default "")
|
||||||
|
let res = (^aws ec2 describe-network-interfaces --query NetworkInterfaces --out "json" | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
let interface = ($res.stdout | from json | where {|it|($it.PrivateIpAddress == $network_private_ip)} | get -o 0 | get -o NetworkInterfaceId | default "")
|
||||||
|
if ($interface | is-not-empty) { aws_delete_target "interface" $interface $settings }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if not $server.prov_settings_clean {
|
||||||
|
print $"❗aws provider settings clean ['prov_settings_clean'] set to ($server.prov_settings_clean)"
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if $target == "all" or $target == "pub_sg" {
|
||||||
|
let aws_sg_name = ($prov_settings | get -o main | get -o sg | get -o name | default "")
|
||||||
|
if ($aws_sg_name | is-not-empty) {
|
||||||
|
let res = (^aws ec2 describe-security-groups --query SecurityGroups --out json | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
let aws_sg_id = ($res.stdout | from json | where {|it| ($it.GroupName == $aws_sg_name) } | get -o 0 | get -o GroupId | default "")
|
||||||
|
if ($aws_sg_id | is-not-empty) { aws_delete_target "securityGroup" $aws_sg_id $settings }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if ($aws_priv_cidr_block | is-not-empty) {
|
||||||
|
if $target == "all" or $target == "priv_sg" {
|
||||||
|
let aws_priv_sg_name = ($prov_settings | get -o priv | get -o sg | get -o name | default "")
|
||||||
|
if ($aws_priv_sg_name | is-not-empty) {
|
||||||
|
let res = (^aws ec2 describe-security-groups --query SecurityGroups --out json | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
let aws_priv_sg_id = ($res.stdout | from json | where {|it| ($it.GroupName == $aws_priv_sg_name)} | get -o 0 | get -o GroupId | default "")
|
||||||
|
if ($aws_priv_sg_id | is-not-empty) { aws_delete_target "securityGroup" $aws_priv_sg_id $settings }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if $target == "all" or $target == "priv_subnet" {
|
||||||
|
let res = (^aws ec2 describe-subnets --query Subnets --out json | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
($res.stdout | from json | where { |it| $it.CidrBlock == $aws_priv_cidr_block } | get -o 0 | get -o SubnetId | default [] |
|
||||||
|
each {|it| aws_delete_target "subnet" $it $settings }
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if $target == "all" or $target == "priv_vpc" {
|
||||||
|
let res = (^aws ec2 describe-vpcs --query Vpcs --out json | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
($res.stdout | from json | where { |it| $it.CidrBlock == $aws_priv_cidr_block } | get -o 0 | get -o VpcId | default [] |
|
||||||
|
each {|it| aws_delete_target "vpcs" $it $settings }
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
if (is-debug-enabled) { print $"❗aws_priv_cidr_block not found in (_ansi yellow_bold)($provider_path | path basename)(_ansi reset) " }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def default_vpc [
|
||||||
|
] {
|
||||||
|
let res = (^aws ec2 describe-vpcs --query Vpcs[].VpcId --filters "Name=is-default,Values=true" --out text | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
($res.stdout | str trim)
|
||||||
|
} else {
|
||||||
|
if (is-debug-enabled) { print$"❗Error get (_ansi red)default Vpc(_ansi reset) " }
|
||||||
|
{}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def default_subnet [
|
||||||
|
vpc: string
|
||||||
|
] {
|
||||||
|
let res = (^aws ec2 describe-subnets --query Subnets[] --filters "Name=default-for-az,Values=true" "Name=vpc-id,Values=vpc-0ffea05634122f3fa" --out json | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
($res.stdout | from json | default [] | get -o 0 | default "")
|
||||||
|
} else {
|
||||||
|
if (is-debug-enabled) { print$"❗Error get (_ansi red)default subnet(_ansi reset) VPC (_ansi yellow)($vpc)(_ansi reset)" }
|
||||||
|
""
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_scan_settings [
|
||||||
|
in_task: string
|
||||||
|
provider_path: string
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
check_mode: bool
|
||||||
|
] {
|
||||||
|
let prov_settings = (load_provider_env $settings $server $provider_path)
|
||||||
|
let env_settings = (get_provider_env $settings $server)
|
||||||
|
if (($prov_settings | get -o main ) == ($env_settings | get -o main)
|
||||||
|
and ($prov_settings | get -o priv ) == ($env_settings | get -o priv)
|
||||||
|
and ($prov_settings | get -o main | get -o vpc) != "?") { return }
|
||||||
|
let task = if $prov_settings.main? == null or ($prov_settings | get -o main | get -o vpc) == "?" {
|
||||||
|
"create"
|
||||||
|
} else if $in_task == "create" {
|
||||||
|
if (is-debug-enabled) { print $"❗aws_scan_settings task ($in_task) and ($provider_path) has content "}
|
||||||
|
"scan"
|
||||||
|
} else { $in_task }
|
||||||
|
let data_settings = if $prov_settings.main? == null or ($prov_settings | get -o main | get -o vpc) != "?" {
|
||||||
|
$prov_settings
|
||||||
|
} else { $env_settings }
|
||||||
|
print $"Scanning (_ansi green_bold)AWS(_ansi reset) resources to (_ansi purple_bold)($task)(_ansi reset) settings in (_ansi yellow_bold)($provider_path | path basename)(_ansi reset) ..."
|
||||||
|
let res = (^aws ec2 describe-availability-zones --query AvailabilityZones| complete)
|
||||||
|
if $res.exit_code != 0 {
|
||||||
|
(throw-error $"🛑 Unable lo load ($server.provider) availability zones" $"($res.exit_code) ($res.stdout)" $"server info ($server.hostname)" --span (metadata $res).span)
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
let $aws_vpc = if ($data_settings | get -o main | get -o vpc | length) > 1 { $settings.main.vpc } else { default_vpc }
|
||||||
|
let $aws_subnet_data = if ($data_settings | get -o main | get -o subnet | length) > 1 {
|
||||||
|
let res = (^aws ec2 describe-subnets --query Subnets -SubnetId $settings.main.subnet --out json | complete)
|
||||||
|
if $res.exit_code != 0 {
|
||||||
|
(throw-error $"🛑 Unable lo load ($server.provider) subnet info ($settings.main.subnet)" $"($res.exit_code) ($res.stdout)" $"server info ($server.hostname)" --span (metadata $res).span)
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
($res.stdout | from json)
|
||||||
|
} else {
|
||||||
|
default_subnet $aws_vpc
|
||||||
|
}
|
||||||
|
let aws_subnet = ($aws_subnet_data | get -o SubnetId | default "")
|
||||||
|
if ($aws_subnet | is-empty) {
|
||||||
|
(throw-error $"🛑 Unable lo load ($server.provider) subnet id" $"($aws_subnet_data)" $"server info ($server.hostname)" --span (metadata $aws_subnet_data).span)
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
let aws_avail_zone = ($aws_subnet_data | get -o AvailabilityZone | default "")
|
||||||
|
if ($aws_avail_zone | is-empty) {
|
||||||
|
(throw-error $"🛑 Unable lo load ($server.provider) subnet availability zone" $"($aws_subnet_data)" $"server info ($server.hostname)" --span (metadata $aws_avail_zone).span)
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
let aws_priv_cidr_block = ($server.priv_cidr_block | default "")
|
||||||
|
let priv = if ($aws_priv_cidr_block | is-not-empty) {
|
||||||
|
let aws_priv_vpc = (aws_create_private_vpc $aws_priv_cidr_block $task)
|
||||||
|
if ($aws_priv_vpc | is-empty) {
|
||||||
|
print $"❗ aws_priv_vpc not found in (_ansi yellow_bold)($provider_path | path basename)(_ansi reset) "
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
let aws_priv_subnet_data = (aws_create_private_subnet $aws_priv_cidr_block $aws_priv_vpc $aws_avail_zone $task)
|
||||||
|
if (is-debug-enabled) { print $aws_priv_subnet_data }
|
||||||
|
let aws_priv_subnet = ($aws_priv_subnet_data | get -o SubnetId | default "")
|
||||||
|
if ($aws_priv_subnet | is-empty) {
|
||||||
|
print $"❗aws_priv_subnet not found in (_ansi yellow_bold)($provider_path | path basename)(_ansi reset) "
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
let aws_priv_avail_zone = ($aws_priv_subnet_data | get -o AvailabilityZone | default "")
|
||||||
|
let aws_priv_sg_name = ($data_settings | get -o priv | get -o sg | get -o name | default "sg_priv")
|
||||||
|
if ($aws_priv_sg_name | is-empty) {
|
||||||
|
print $"❗aws_priv_sg.name not found in (_ansi yellow_bold)($provider_path | path basename)(_ansi reset)"
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
let aws_priv_sg_id = (aws_create_sg_id $aws_priv_vpc $aws_priv_sg_name $task)
|
||||||
|
let aws_priv_sg_data = {
|
||||||
|
id: $aws_priv_sg_id,
|
||||||
|
name: $aws_priv_sg_name,
|
||||||
|
perms: ($env_settings | get -o priv | get -o sg | get -o perms | default [])
|
||||||
|
}
|
||||||
|
if $task == "create" or $task == "scan" { aws_add_sg_perms $aws_priv_sg_data $server $check_mode}
|
||||||
|
{
|
||||||
|
vpc: $aws_priv_vpc,
|
||||||
|
subnet: $aws_priv_subnet,
|
||||||
|
cidr_block: $aws_priv_cidr_block,
|
||||||
|
avail_zone: $aws_priv_avail_zone,
|
||||||
|
sg: $aws_priv_sg_data
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
if (is-debug-enabled) { print$"❗aws_priv_cidr_block not found in (_ansi yellow_bold)($provider_path | path basename)(_ansi reset) " }
|
||||||
|
}
|
||||||
|
let aws_sg_name = ($data_settings | get -o sg | get -o name | default "sg_pub")
|
||||||
|
if ($aws_sg_name | is-empty) {
|
||||||
|
print $"aws_sg_name not found in (_ansi yellow_bold)($provider_path | path basename)(_ansi reset) "
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
let aws_vpc_subnet_data = (aws_vpc_subnet $aws_avail_zone $priv.subnet $task)
|
||||||
|
if $task == "create" and ($aws_vpc_subnet_data | is-empty) {
|
||||||
|
print $"❗No availability zone ($aws_avail_zone) "
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
print $aws_vpc_subnet_data
|
||||||
|
let aws_sg_id = (aws_create_sg_id $aws_vpc $aws_sg_name $task)
|
||||||
|
let aws_sg_data = {
|
||||||
|
id: $aws_sg_id,
|
||||||
|
name: $aws_sg_name,
|
||||||
|
perms: ($env_settings | get -o main | get -o sg | get -o perms | default [])
|
||||||
|
}
|
||||||
|
if $task == "create" or $task == "scan" { aws_add_sg_perms $aws_sg_data $server $check_mode }
|
||||||
|
let main = {
|
||||||
|
vpc: $aws_vpc,
|
||||||
|
subnet: $aws_subnet,
|
||||||
|
cidr_block: ($aws_subnet_data | get -o CidrBlock | default ""),
|
||||||
|
avail_zone: $aws_avail_zone,
|
||||||
|
sg: $aws_sg_data,
|
||||||
|
}
|
||||||
|
let data_settings = if ($aws_priv_cidr_block | is-not-empty) {
|
||||||
|
{ main: $main, priv: $priv }
|
||||||
|
} else {
|
||||||
|
{ main: $main }
|
||||||
|
}
|
||||||
|
save_provider_env (load_provider_env $settings $server $provider_path | merge $data_settings) $settings $provider_path
|
||||||
|
print $"✅ (_ansi green_bold)AWS(_ansi reset) (_ansi cyan_bold)settings(_ansi reset) completed in (_ansi yellow_bold)($provider_path | path basename)(_ansi reset)"
|
||||||
|
}
|
||||||
|
export def aws_scan_servers [
|
||||||
|
provider_path: string
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
mut servers = []
|
||||||
|
for server_info in ($settings.data.servers? | default []) {
|
||||||
|
let hostname = ($server_info | get -o hostname | default "" )
|
||||||
|
let network_private_ip = ($server_info | get -o network_private_ip | default "")
|
||||||
|
let res = (^aws ec2 describe-instances --out json --filters $'"Name=tag:hostname,Values=($hostname)"' --filters "Name=instance-state-name,Values=running"
|
||||||
|
--query "Reservations[*].Instances[].{
|
||||||
|
id: InstanceId,
|
||||||
|
priv: NetworkInterfaces[],
|
||||||
|
pub: PublicIpAddress,
|
||||||
|
type: InstanceType,
|
||||||
|
status: State.Name,
|
||||||
|
keyname: KeyName,
|
||||||
|
launchtime: LaunchTime,
|
||||||
|
block_devices: BlockDeviceMappings
|
||||||
|
}"
|
||||||
|
--output json | complete)
|
||||||
|
if $res.exit_code != 0 {
|
||||||
|
print $"❗No data found for ($hostname) in ($server.provider) "
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
for instance_data in ($res.stdout | from json ) {
|
||||||
|
if ($instance_data | get -o status | str contains "erminated") { continue }
|
||||||
|
let instance_id = ($instance_data | get -o id | default "")
|
||||||
|
mut volumes = []
|
||||||
|
for device in ($instance_data | get -o block_devices | default []) {
|
||||||
|
let vol_id = ($device | get -o Ebs | get -o VolumeId | default "")
|
||||||
|
if ($vol_id | is-empty) { continue }
|
||||||
|
let res_vols = (^aws ec2 describe-volumes --volume-id $vol_id --filters $"Name=attachment.instance-id,Values=($instance_id)"
|
||||||
|
--query "Volumes[]" --output=json | complete)
|
||||||
|
if $res_vols.exit_code == 0 {
|
||||||
|
$volumes = ($volumes | append ($res_vols.stdout | from json))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
$servers = ($servers | append {
|
||||||
|
id: $instance_id
|
||||||
|
hostname: $hostname
|
||||||
|
keyname: ($instance_data | get -o keyname | default ""),
|
||||||
|
private_ips: ($instance_data | get -o priv | default []),
|
||||||
|
puplic_ips: ($instance_data | get -o pub | default []),
|
||||||
|
volumes: $volumes,
|
||||||
|
devices: ($instance_data | get -o block_devices | default []),
|
||||||
|
launchtime: ($instance_data | get -o launchtime | default ""),
|
||||||
|
info: $server_info
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
save_provider_env (load_provider_env $settings $server $provider_path | merge { servers: $servers}) $settings $provider_path
|
||||||
|
print $"✅ (_ansi green_bold)AWS(_ansi reset) (_ansi blue_bold)servers settings(_ansi reset)
|
||||||
|
completed in (_ansi yellow_bold)($provider_path | path basename)(_ansi reset)"
|
||||||
|
}
|
||||||
716
providers/aws/nulib/aws/lib.nu-e
Normal file
716
providers/aws/nulib/aws/lib.nu-e
Normal file
@ -0,0 +1,716 @@
|
|||||||
|
#!/usr/bin/env nu
|
||||||
|
# Info: Script to create/delete AWS resources from file settings in bash with template/vars
|
||||||
|
# Author: JesusPerez
|
||||||
|
# Release: 1.0
|
||||||
|
# Date: 26-03-2024
|
||||||
|
|
||||||
|
use ../../../../core/nulib/lib_provisioning/utils/templates.nu run_from_template
|
||||||
|
use ../../../../core/nulib/lib_provisioning/config/accessor.nu *
|
||||||
|
|
||||||
|
export def aws_review_credentials [
|
||||||
|
] {
|
||||||
|
print $"❗AWS credentials not found for '$PROVIDER_CLI_CMD' command."
|
||||||
|
print $" Use default profile or env AWS_PROFILE from $HOME/.aws/credentials path or environment variables for settings"
|
||||||
|
print $" More info: "
|
||||||
|
print $" Profile mode: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html"
|
||||||
|
print $" Evironment mode: https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html"
|
||||||
|
}
|
||||||
|
export def aws_check_region [
|
||||||
|
zone: string
|
||||||
|
] {
|
||||||
|
if ($zone |is-empty) {
|
||||||
|
print $"❗AWS region zone ($env.AWS_DEFAULT_REGION) not found for '$PROVIDER_CLI_CMD' command."
|
||||||
|
print $"Use set default profile or use env AWS_PROFILE with $HOME/.aws/credentials path or environment variables for settings"
|
||||||
|
}
|
||||||
|
(^aws ec2 describe-availability-zones --region $zone | complete).exit_code
|
||||||
|
}
|
||||||
|
export def aws_get_plan_info [
|
||||||
|
var: string
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
let plan = ($server | get -o $var | default "")
|
||||||
|
if ($plan | is-mepty) { return }
|
||||||
|
let res = (^aws ec2 describe-instance-types --instance-types $plan
|
||||||
|
--query 'InstanceTypes[].{ type: InstanceType, cores: VCpuInfo.DefaultCores, memory: MemoryInfo.SizeInMiB, arch: ProcessorInfo.SupportedArchitectures, gen: CurrentGeneration, infaces: NetworkInfo.MaximumNetworkInterfaces, ena: NetworkInfo.EnaSupport }'
|
||||||
|
--out=json )
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
($res.stdout | from json | get -o 0 | default "")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_find_plan [
|
||||||
|
var: string
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
let reqplan = ($server | get -o $var | default "")
|
||||||
|
if ($reqplan | is-mepty) {
|
||||||
|
print $"❗No reqplan found in settings for ($var)"
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
let res = (^ aws ec2 describe-instance-types --filters $"Name=processor-info.supported-architecture,Values=($reqplan.arch | default '')"
|
||||||
|
$"Name=memory-info.size-in-mib,Values=($reqplan.memory | default '')"
|
||||||
|
$"Name=vcpu-info.default-cores,Values=($reqplan.cores | default '')"
|
||||||
|
$"Name=network-info.maximum-network-interfaces,Values=($reqplan.infaces | default '')"
|
||||||
|
$"Name=network-info.ena-support,Values=($reqplan.ena | default '')"
|
||||||
|
--query 'InstanceTypes[].{ type: InstanceType, cores: VCpuInfo.DefaultCores, memory: MemoryInfo.SizeInMiB, arch: ProcessorInfo.SupportedArchitectures, gen: CurrentGeneration, infaces: NetworkInfo.MaximumNetworkInterfaces, ena: NetworkInfo.EnaSupport }'
|
||||||
|
--output json
|
||||||
|
)
|
||||||
|
if ($res.exit_code == 0) {
|
||||||
|
($res.stdout | from json)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_compare_plan_reqplan [
|
||||||
|
var_plan: string
|
||||||
|
var_reqplan: string
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
let plan = ($server | get -o $var_plan)
|
||||||
|
let check_plan = (aws_get_plan_info $var_plan $server)
|
||||||
|
let reqplan = ($server | get -o $var_reqplan)
|
||||||
|
|
||||||
|
if ($plan | is-empty) or ( $check_plan | is-empty) {
|
||||||
|
print $"❗No valid $plan found for $var_plan in $AWS_DEFAULT_REGION"
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
if ($reqplan | is-empty) { return }
|
||||||
|
|
||||||
|
let plan_memory = ($check_plan | get -o memory | default "")
|
||||||
|
let reqplan_memory = ($reqplan| get -o memory | default "")
|
||||||
|
if $plan_memory != $reqplan_memory {
|
||||||
|
print $"❗$plan memory does not match plan: $plan_memory expected $reqplan_memory"
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
let plan_cores = ($check_plan | get -o cores | default "")
|
||||||
|
let reqplan_cores = ($reqplan | get -o cores | default "")
|
||||||
|
if $plan_cores != $reqplan_cores {
|
||||||
|
print $"❗($plan) cores does not match plan: ($plan_cores) expected ($reqplan_cores)"
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
let plan_archs = ($check_plan | get -o arch | default "")
|
||||||
|
let reqplan_archs = ($reqplan | get -o arch | default "")
|
||||||
|
if not ($plan_archs | str contains $reqplan_archs ) {
|
||||||
|
print $"❗($plan) architectures does not match plan: ($plan_archs) expected ($reqplan_archs)"
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
let plan_infaces = ($check_plan | get -o infaces | default "")
|
||||||
|
let reqplan_infaces = ($reqplan | get -o infaces | default "")
|
||||||
|
if $plan_infaces < $reqplan_infaces {
|
||||||
|
print $"❗($plan) interfaces number does not match plan: ($plan_infaces) expected ($reqplan_infaces)"
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
0
|
||||||
|
}
|
||||||
|
export def aws_get_os_image [
|
||||||
|
name: string
|
||||||
|
arch: string
|
||||||
|
] {
|
||||||
|
let res = (^aws ec2 describe-images --owners amazon --filters $"'Name=name,Values=*'($name)'*'" $"'Name=architecture,Values=*'($arch)'*'"
|
||||||
|
--query 'reverse(sort_by(Images,&CreationDate))[:5].{id:ImageId, name: Name, date:CreationDate}[0]' --output json
|
||||||
|
)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
($res.stdout | from json)
|
||||||
|
} else { "" }
|
||||||
|
}
|
||||||
|
export def aws_delete_private_vpcs [
|
||||||
|
aws_priv_cidr_block: string
|
||||||
|
] {
|
||||||
|
let res = (^aws ec2 describe-vpcs --query Vpcs --out json | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
for it in ($res.stdout | from json | where {|it| $it.CidrBlock == $aws_priv_cidr_block } | get -o VpcId | default []) {
|
||||||
|
print $"delete vpc id ($it)"
|
||||||
|
^aws ec2 delete-vpc --vpc-id "$it"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_create_private_vpc [
|
||||||
|
aws_priv_cidr_block: string
|
||||||
|
task: string
|
||||||
|
] {
|
||||||
|
let res = (^aws ec2 describe-vpcs --query Vpcs --out json | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-empty) {
|
||||||
|
print $"❗Error ($task) vpcs ($aws_priv_cidr_block) "
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
let aws_priv_vpc = ($res.stdout | from json | where {|it| $it.CidrBlock == $aws_priv_cidr_block } | get -o 0 | get -o VpcId | default "")
|
||||||
|
match $task {
|
||||||
|
"create" => {
|
||||||
|
if ($aws_priv_vpc | is-not-empty) {
|
||||||
|
print $"Clean up VPC ($aws_priv_vpc)..."
|
||||||
|
let res = (^aws ec2 delete-vpc --vpc-id $aws_priv_vpc err> /dev/null | complete )
|
||||||
|
if $res.exit_code != 0 {
|
||||||
|
print $"vpc ($aws_priv_vpc) delete error ($res.exit_code) ($res.stdout)"
|
||||||
|
return $aws_priv_vpc
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let res = (^aws ec2 create-vpc --cidr-block $aws_priv_cidr_block --query Vpc.VpcId --output text | complete)
|
||||||
|
if $res.exit_code == 0 {
|
||||||
|
($res.stdout | str replace '"' '')
|
||||||
|
} else {
|
||||||
|
print $"❗ Error ($task) priv_vpc for ($aws_priv_cidr_block)"
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
$aws_priv_vpc
|
||||||
|
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_delete_sgs_by_name [
|
||||||
|
aws_sg_name: string
|
||||||
|
] {
|
||||||
|
let res = (^aws ec2 describe-security-groups --query SecurityGroups --out json | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
let aws_sg_id = ($res.stdout | from json | where {|it| $it.GroupName == $aws_sg_name } | get -o GroupId | default "")
|
||||||
|
if ($aws_sg_id | is-not-empty) {
|
||||||
|
print $"Clean up SGs ($aws_sg_name)"
|
||||||
|
^aws ec2 delete-security-group --group-id $aws_sg_id
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_delete_sgs [
|
||||||
|
aws_vpc: string
|
||||||
|
] {
|
||||||
|
let res = (^aws ec2 describe-security-groups --query SecurityGroups --out json)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
for it in ($res.stdout | from json | where {|it| $it.VpcId == $aws_vpc } | where {|it| $it.GroupName != "default" } | get -o GroupId | default "") {
|
||||||
|
print $"delete security group id ($it)"
|
||||||
|
^aws ec2 delete-security-group --group-id $it
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_create_sg_id [
|
||||||
|
aws_vpc: string
|
||||||
|
aws_sg_name: string
|
||||||
|
task: string
|
||||||
|
] {
|
||||||
|
let res = (^aws ec2 describe-security-groups --query SecurityGroups --out json | complete)
|
||||||
|
if $res.exit_code != 0 or ($res.stdout | is-empty) {
|
||||||
|
print $"❗Error ($task) sg_id for ($aws_sg_name) in ($aws_vpc)"
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
let aws_sg_id = ($res.stdout | from json | where {|it| $it.VpcId == $aws_vpc and $it.GroupName == $aws_sg_name } |
|
||||||
|
get -o 0 | get -o GroupId | default "")
|
||||||
|
match $task {
|
||||||
|
"create" => {
|
||||||
|
if ($aws_sg_id | is-not-empty) {
|
||||||
|
print $"Clean up sg ($aws_sg_id) ..."
|
||||||
|
let res = (^aws ec2 delete-security-group --group-id $aws_sg_id | complete)
|
||||||
|
if $res.exit_code != 0 {
|
||||||
|
print $"❗Error delete ($aws_sg_id) for ($aws_sg_name) in ($aws_vpc)"
|
||||||
|
return $aws_sg_id
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let res = (^aws ec2 create-security-group --group-name $aws_sg_name --description $"Group ($aws_sg_name)"
|
||||||
|
--tag-specifications $"ResourceType=security-group,Tags=[{Key=Name,Value=($aws_sg_name)}]"
|
||||||
|
--vpc-id ($aws_vpc | str trim) --out json | complete )
|
||||||
|
if $res.exit_code == 0 {
|
||||||
|
($res.stdout | from json | get -o GroupId | default "")
|
||||||
|
} else {
|
||||||
|
print $"❗Error ($task) sg_id for ($aws_sg_name) in ($aws_vpc)"
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
$aws_sg_id
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_add_sg_perms [
|
||||||
|
sg_data: record
|
||||||
|
server: record
|
||||||
|
check_mode: bool
|
||||||
|
] {
|
||||||
|
let perms = ($sg_data | get -o perms | default [])
|
||||||
|
if ($perms | is-empty) { return }
|
||||||
|
let res = (^aws ec2 describe-security-groups --group-id $sg_data.id --query SecurityGroups[].IpPermissions --out json | complete)
|
||||||
|
let curr_sg_perms = if $res.exit_code == 0 {
|
||||||
|
($res.stdout | from json | get -o 0 | default [])
|
||||||
|
} else { [] }
|
||||||
|
mut curr_perms = []
|
||||||
|
for p in $curr_sg_perms {
|
||||||
|
mut ranges = ""
|
||||||
|
for rng in ($p | get -o IpRanges) {
|
||||||
|
if ($ranges | is-not-empty) { $ranges = $"($ranges),"}
|
||||||
|
$ranges = $"($ranges){CidrIp=($rng.CidrIp)}"
|
||||||
|
}
|
||||||
|
let protocol = ($p | get -o IpProtocol | default "")
|
||||||
|
let from_port = ($p | get -o FromPort | default "")
|
||||||
|
let to_port = ($p | get -o ToPort | default "")
|
||||||
|
for it in $perms {
|
||||||
|
if ($protocol == ($it | get -o protocol ) and $from_port == ($it | get -o fromPort ) and
|
||||||
|
$to_port == ($it | get -o toPort ) and
|
||||||
|
$ranges == ($it | get -o ranges | str replace "[" "" | str replace "]" "" )) {
|
||||||
|
} else {
|
||||||
|
$curr_perms = ($curr_perms | append $p)
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if ($curr_perms == $curr_sg_perms) and ($curr_perms| length) == ($perms | length) { return }
|
||||||
|
if ($perms == $curr_perms) { return }
|
||||||
|
if (is-debug-enabled) {
|
||||||
|
print $"(_ansi green)current sg perms(_ansi reset) ($curr_perms | table -e)"
|
||||||
|
}
|
||||||
|
let wk_format = if (get-provisioning-wk-format) == "json" { "json" } else { "yaml" }
|
||||||
|
let wk_vars = ( "/tmp/" | path join (mktemp --tmpdir-path "/tmp" --suffix $".($wk_format)" | path basename))
|
||||||
|
let data = { sg_name: $sg_data.name, sg_id: $sg_data.id, perms: $perms, curr_perms: $curr_perms }
|
||||||
|
if $wk_format == "json" {
|
||||||
|
$data | to json | save --force $wk_vars
|
||||||
|
} else {
|
||||||
|
$data | to yaml | save --force $wk_vars
|
||||||
|
}
|
||||||
|
let run_file = ("/tmp" | path join $"onaws_run_sg_(mktemp --tmpdir-path "/tmp" --suffix ".sh" | path basename | str replace 'tmp.' '' )")
|
||||||
|
let sg_template = ((get-base-path) | path join "providers" | path join $server.provider | path join templates | path join "aws_sg.j2" )
|
||||||
|
if not ($sg_template | path exists) {
|
||||||
|
print $"❗($sg_template) not found for Security Groups ($sg_data.name)"
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
#use ../../../../core/nulib/lib_provisioning/utils/templates.nu run_from_template
|
||||||
|
let res = if $check_mode {
|
||||||
|
run_from_template $sg_template $wk_vars $run_file --check_mode
|
||||||
|
} else {
|
||||||
|
run_from_template $sg_template $wk_vars $run_file
|
||||||
|
}
|
||||||
|
if $res {
|
||||||
|
if (is-debug-enabled) {
|
||||||
|
print $"(_ansi green)OK(_ansi reset) (_ansi green_bold)($sg_data.name)(_ansi reset)"
|
||||||
|
} else {
|
||||||
|
rm --force $wk_vars $run_file
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
print $"(_ansi red)Failed(_ansi reset) (_ansi green_bold)($sg_data.name)(_ansi reset)"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_delete_private_subnets [
|
||||||
|
aws_priv_vpc: string
|
||||||
|
aws_priv_cidr_block: string
|
||||||
|
] {
|
||||||
|
let res = (^aws ec2 describe-subnets --query Subnets --out json | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
for it in ($res.stdout | from json | where { |it| $it.VpcId == $aws_priv_vpc and $it.CidrBlock == $aws_priv_cidr_block } | get -o SubnetId | default []) {
|
||||||
|
print $"Clean up subnet ($it) in ($aws_priv_vpc)..."
|
||||||
|
let res = (^aws ec2 delete-subnet --subnet-id $it | complete)
|
||||||
|
if $res.exit_code != 0 { return false }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
true
|
||||||
|
}
|
||||||
|
export def aws_create_private_subnet [
|
||||||
|
aws_priv_cidr_block: string
|
||||||
|
aws_priv_vpc: string
|
||||||
|
aws_avail_zone: string
|
||||||
|
task: string
|
||||||
|
] {
|
||||||
|
match $task {
|
||||||
|
"create" => {
|
||||||
|
if not (aws_delete_private_subnets $aws_priv_vpc $aws_priv_cidr_block) {
|
||||||
|
let res = (^aws ec2 describe-subnets --query Subnets --out json | complete)
|
||||||
|
return ($res.stdout | from json | where { |it| $it.VpcId == $aws_priv_vpc and $it.CidrBlock == $aws_priv_cidr_block } | get -o 0)
|
||||||
|
}
|
||||||
|
let res = (^aws ec2 create-subnet --vpc-id $aws_priv_vpc --cidr-block $aws_priv_cidr_block --availability-zone $aws_avail_zone --query "Subnet" --output json | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
($res.stdout | from json)
|
||||||
|
} else {
|
||||||
|
print $"❗aws_priv_subnet not found for ($aws_priv_vpc) - ($aws_priv_cidr_block)"
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
let res = (^aws ec2 describe-subnets --query Subnets --out json | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
($res.stdout | from json | where { |it| $it.VpcId == $aws_priv_vpc and $it.CidrBlock == $aws_priv_cidr_block } | get -o 0)
|
||||||
|
} else {
|
||||||
|
{}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
def aws_vpc_subnet [
|
||||||
|
aws_avail_zone: string
|
||||||
|
aws_priv_subnet: string
|
||||||
|
task: string
|
||||||
|
] {
|
||||||
|
let res = (^aws ec2 describe-subnets --query Subnets --out json | complete)
|
||||||
|
let aws_vpc_subnet_data = if $res.exit_code == 0 {
|
||||||
|
let data = ($res.stdout | from json | where {|it| $it.AvailabilityZone == $aws_avail_zone and $it.SubnetId != $aws_priv_subnet } | get -o 0 )
|
||||||
|
{"vpc": $"($data | get -o VpcId | default '')", "subnet": $"($data | get -o SubnetId | default '')"}
|
||||||
|
} else {
|
||||||
|
{}
|
||||||
|
}
|
||||||
|
if $task == "create" and ($aws_vpc_subnet_data | is-empty) {
|
||||||
|
^aws ec2 create-default-subnet --availability-zone $aws_avail_zone
|
||||||
|
(aws_vpc_subnet $aws_avail_zone $aws_priv_subnet "scan")
|
||||||
|
} else {
|
||||||
|
$aws_vpc_subnet_data
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_delete_private_interfaces [
|
||||||
|
aws_priv_subnet: string
|
||||||
|
] {
|
||||||
|
let res = (^aws ec2 describe-network-interfaces --query NetworkInterfaces --out json | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
for it in ($res.stdout | from json | where {|it| $it.SubnetId == $aws_priv_subnet } | get -o NetworkInterfaceId | default []) {
|
||||||
|
^aws ec2 delete-network-interface --network-interface-id $it
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_delete_private_interface [
|
||||||
|
network_interface_id: string
|
||||||
|
] {
|
||||||
|
^aws ec2 delete-network-interface --network-interface-id "$network_interface_id"
|
||||||
|
}
|
||||||
|
export def aws_get_interface_defs [
|
||||||
|
ip_interface: string
|
||||||
|
aws_priv_subnet: string
|
||||||
|
] {
|
||||||
|
let res = (^aws ec2 describe-network-interfaces --query NetworkInterfaces --out json | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
($res.stdout | from json | where {|it| $it.SubnetId == $aws_priv_subnet and $it.PrivateIpAddress == $ip_interface } |
|
||||||
|
get -o 0 | get -o NetworkInterfaceId | default ""
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_get_create_private_interface [
|
||||||
|
ip_interface: string
|
||||||
|
aws_priv_subnet: string
|
||||||
|
] {
|
||||||
|
(aws_get_interface_defs $ip_interface $aws_priv_subnet)
|
||||||
|
}
|
||||||
|
export def aws_get_instance_defs [
|
||||||
|
instance: string
|
||||||
|
] {
|
||||||
|
let res = (^aws ec2 describe-instances --instance-ids $instance --out "json" | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
($res.stdout | from json | get -o "Reservations" | get -o "Instances" | default "" )
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def attach_private_interface [
|
||||||
|
interface: string
|
||||||
|
instance: string
|
||||||
|
aws_priv_subnet: string
|
||||||
|
] {
|
||||||
|
if (aws_get_instance_defs $instance | is-not-empty) and (aws_get_interface_defs $interface $aws_priv_subnet | is-not-empty) {
|
||||||
|
(^aws ec2 attach-network-interface --network-interface-id $interface --instance-id $instance --device-index 1)
|
||||||
|
} else { "" }
|
||||||
|
}
|
||||||
|
export def detach_private_interface [
|
||||||
|
interface: string
|
||||||
|
instance: string
|
||||||
|
aws_priv_subnet: string
|
||||||
|
] {
|
||||||
|
if (aws_get_instance_defs $instance | is-not-empty) and (aws_get_interface_defs $interface $aws_priv_subnet | is-not-empty) {
|
||||||
|
(^aws ec2 detach-network-interface --network-interface-id $interface --instance-id $instance)
|
||||||
|
} else { "" }
|
||||||
|
}
|
||||||
|
export def aws_delete_target [
|
||||||
|
target: string
|
||||||
|
target_id: string
|
||||||
|
settings: record
|
||||||
|
] {
|
||||||
|
mut num = 0
|
||||||
|
mut res = ""
|
||||||
|
mut status = ""
|
||||||
|
let val_timeout = if $settings.running_timeout? != null { $settings.running_timeout } else { 60 }
|
||||||
|
let wait = if $settings.running_wait? != null { $settings.running_wait } else { 10 }
|
||||||
|
let wait_duration = ($"($wait)sec"| into duration)
|
||||||
|
if (is-debug-enabled) { print -n $"Delete ($target) -> ($target_id) " }
|
||||||
|
while ($status | is-empty) {
|
||||||
|
let status = match $target {
|
||||||
|
"securityGroup" => (^aws ec2 describe-security-groups --group-id $target_id err> /dev/null),
|
||||||
|
"subnet" => (^aws ec2 describe-subnets --subnet-id $target_id err> /dev/null) ,
|
||||||
|
"vpcs" => (^aws ec2 describe-vpcs --vpc-id $target_id err> /dev/null) ,
|
||||||
|
"interface" => (^aws ec2 describe-network-interfaces --network-interface-id $target_id err> /dev/null),
|
||||||
|
}
|
||||||
|
if ($status | is-empty) {
|
||||||
|
print $" "
|
||||||
|
return
|
||||||
|
}
|
||||||
|
let res = match $target {
|
||||||
|
"securityGroup" => (^aws ec2 delete-security-group --group-id $target_id | complete).exit_code,
|
||||||
|
"subnet" => (^aws ec2 delete-subnet --subnet-id $target_id | complete).exit_code,
|
||||||
|
"vpcs" => (^aws ec2 delete-vpc --vpc-id $target_id | complete).exit_code,
|
||||||
|
"interface" => (^aws ec2 delete-network-interface --network-interface-id $target_id | complete).exit_code,
|
||||||
|
}
|
||||||
|
if ($res == 0) {
|
||||||
|
print $" "
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if ($status | is-not-empty) or ($res != 0 ) {
|
||||||
|
sleep $wait_duration
|
||||||
|
$num += $wait
|
||||||
|
if $val_timeout > 0 and $num > $val_timeout { return 1 }
|
||||||
|
print -n $"($num) "
|
||||||
|
}
|
||||||
|
}
|
||||||
|
print $""
|
||||||
|
}
|
||||||
|
export def aws_delete_settings [
|
||||||
|
target: string
|
||||||
|
provider_path: string
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
if not ($provider_path |path exists) {
|
||||||
|
print $"❗aws_settings not found ($provider_path) no delete settings "
|
||||||
|
return
|
||||||
|
}
|
||||||
|
let prov_settings = (load_provider_env $settings $server $provider_path)
|
||||||
|
let env_settings = (get_provider_env $settings $server)
|
||||||
|
if ($prov_settings | is-empty) or $prov_settings.main? == null or $prov_settings.priv? == null {
|
||||||
|
if (is-debug-enabled) { print $"❗aws_settings (_ansi yellow_bold)($provider_path | path basename)(_ansi reset) no settings main and priv found" }
|
||||||
|
return
|
||||||
|
}
|
||||||
|
let aws_priv_subnet = ($prov_settings.priv.subnet | default "")
|
||||||
|
let aws_priv_cidr_block = ($server.priv_cidr_block | default "")
|
||||||
|
print $"Scanning aws resources to clean from (_ansi yellow_bold)($provider_path | path basename)(_ansi reset) ... ($aws_priv_cidr_block)"
|
||||||
|
if $target == "all" or $target == "interface" {
|
||||||
|
for server_info in ($settings.data.servers) {
|
||||||
|
let server = ($server_info | get -o hostname | default "")
|
||||||
|
let network_private_ip = ($server_info | get -o network_private_ip | default "")
|
||||||
|
let res = (^aws ec2 describe-network-interfaces --query NetworkInterfaces --out "json" | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
let interface = ($res.stdout | from json | where {|it|($it.PrivateIpAddress == $network_private_ip)} | get -o 0 | get -o NetworkInterfaceId | default "")
|
||||||
|
if ($interface | is-not-empty) { aws_delete_target "interface" $interface $settings }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if not $server.prov_settings_clean {
|
||||||
|
print $"❗aws provider settings clean ['prov_settings_clean'] set to ($server.prov_settings_clean)"
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if $target == "all" or $target == "pub_sg" {
|
||||||
|
let aws_sg_name = ($prov_settings | get -o main | get -o sg | get -o name | default "")
|
||||||
|
if ($aws_sg_name | is-not-empty) {
|
||||||
|
let res = (^aws ec2 describe-security-groups --query SecurityGroups --out json | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
let aws_sg_id = ($res.stdout | from json | where {|it| ($it.GroupName == $aws_sg_name) } | get -o 0 | get -o GroupId | default "")
|
||||||
|
if ($aws_sg_id | is-not-empty) { aws_delete_target "securityGroup" $aws_sg_id $settings }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if ($aws_priv_cidr_block | is-not-empty) {
|
||||||
|
if $target == "all" or $target == "priv_sg" {
|
||||||
|
let aws_priv_sg_name = ($prov_settings | get -o priv | get -o sg | get -o name | default "")
|
||||||
|
if ($aws_priv_sg_name | is-not-empty) {
|
||||||
|
let res = (^aws ec2 describe-security-groups --query SecurityGroups --out json | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
let aws_priv_sg_id = ($res.stdout | from json | where {|it| ($it.GroupName == $aws_priv_sg_name)} | get -o 0 | get -o GroupId | default "")
|
||||||
|
if ($aws_priv_sg_id | is-not-empty) { aws_delete_target "securityGroup" $aws_priv_sg_id $settings }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if $target == "all" or $target == "priv_subnet" {
|
||||||
|
let res = (^aws ec2 describe-subnets --query Subnets --out json | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
($res.stdout | from json | where { |it| $it.CidrBlock == $aws_priv_cidr_block } | get -o 0 | get -o SubnetId | default [] |
|
||||||
|
each {|it| aws_delete_target "subnet" $it $settings }
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if $target == "all" or $target == "priv_vpc" {
|
||||||
|
let res = (^aws ec2 describe-vpcs --query Vpcs --out json | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
($res.stdout | from json | where { |it| $it.CidrBlock == $aws_priv_cidr_block } | get -o 0 | get -o VpcId | default [] |
|
||||||
|
each {|it| aws_delete_target "vpcs" $it $settings }
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
if (is-debug-enabled) { print $"❗aws_priv_cidr_block not found in (_ansi yellow_bold)($provider_path | path basename)(_ansi reset) " }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def default_vpc [
|
||||||
|
] {
|
||||||
|
let res = (^aws ec2 describe-vpcs --query Vpcs[].VpcId --filters "Name=is-default,Values=true" --out text | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
($res.stdout | str trim)
|
||||||
|
} else {
|
||||||
|
if (is-debug-enabled) { print$"❗Error get (_ansi red)default Vpc(_ansi reset) " }
|
||||||
|
{}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def default_subnet [
|
||||||
|
vpc: string
|
||||||
|
] {
|
||||||
|
let res = (^aws ec2 describe-subnets --query Subnets[] --filters "Name=default-for-az,Values=true" "Name=vpc-id,Values=vpc-0ffea05634122f3fa" --out json | complete)
|
||||||
|
if $res.exit_code == 0 and ($res.stdout | is-not-empty) {
|
||||||
|
($res.stdout | from json | default [] | get -o 0 | default "")
|
||||||
|
} else {
|
||||||
|
if (is-debug-enabled) { print$"❗Error get (_ansi red)default subnet(_ansi reset) VPC (_ansi yellow)($vpc)(_ansi reset)" }
|
||||||
|
""
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_scan_settings [
|
||||||
|
in_task: string
|
||||||
|
provider_path: string
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
check_mode: bool
|
||||||
|
] {
|
||||||
|
let prov_settings = (load_provider_env $settings $server $provider_path)
|
||||||
|
let env_settings = (get_provider_env $settings $server)
|
||||||
|
if (($prov_settings | get -o main ) == ($env_settings | get -o main)
|
||||||
|
and ($prov_settings | get -o priv ) == ($env_settings | get -o priv)
|
||||||
|
and ($prov_settings | get -o main | get -o vpc) != "?") { return }
|
||||||
|
let task = if $prov_settings.main? == null or ($prov_settings | get -o main | get -o vpc) == "?" {
|
||||||
|
"create"
|
||||||
|
} else if $in_task == "create" {
|
||||||
|
if (is-debug-enabled) { print $"❗aws_scan_settings task ($in_task) and ($provider_path) has content "}
|
||||||
|
"scan"
|
||||||
|
} else { $in_task }
|
||||||
|
let data_settings = if $prov_settings.main? == null or ($prov_settings | get -o main | get -o vpc) != "?" {
|
||||||
|
$prov_settings
|
||||||
|
} else { $env_settings }
|
||||||
|
print $"Scanning (_ansi green_bold)AWS(_ansi reset) resources to (_ansi purple_bold)($task)(_ansi reset) settings in (_ansi yellow_bold)($provider_path | path basename)(_ansi reset) ..."
|
||||||
|
let res = (^aws ec2 describe-availability-zones --query AvailabilityZones| complete)
|
||||||
|
if $res.exit_code != 0 {
|
||||||
|
(throw-error $"🛑 Unable lo load ($server.provider) availability zones" $"($res.exit_code) ($res.stdout)" $"server info ($server.hostname)" --span (metadata $res).span)
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
let $aws_vpc = if ($data_settings | get -o main | get -o vpc | length) > 1 { $settings.main.vpc } else { default_vpc }
|
||||||
|
let $aws_subnet_data = if ($data_settings | get -o main | get -o subnet | length) > 1 {
|
||||||
|
let res = (^aws ec2 describe-subnets --query Subnets -SubnetId $settings.main.subnet --out json | complete)
|
||||||
|
if $res.exit_code != 0 {
|
||||||
|
(throw-error $"🛑 Unable lo load ($server.provider) subnet info ($settings.main.subnet)" $"($res.exit_code) ($res.stdout)" $"server info ($server.hostname)" --span (metadata $res).span)
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
($res.stdout | from json)
|
||||||
|
} else {
|
||||||
|
default_subnet $aws_vpc
|
||||||
|
}
|
||||||
|
let aws_subnet = ($aws_subnet_data | get -o SubnetId | default "")
|
||||||
|
if ($aws_subnet | is-empty) {
|
||||||
|
(throw-error $"🛑 Unable lo load ($server.provider) subnet id" $"($aws_subnet_data)" $"server info ($server.hostname)" --span (metadata $aws_subnet_data).span)
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
let aws_avail_zone = ($aws_subnet_data | get -o AvailabilityZone | default "")
|
||||||
|
if ($aws_avail_zone | is-empty) {
|
||||||
|
(throw-error $"🛑 Unable lo load ($server.provider) subnet availability zone" $"($aws_subnet_data)" $"server info ($server.hostname)" --span (metadata $aws_avail_zone).span)
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
let aws_priv_cidr_block = ($server.priv_cidr_block | default "")
|
||||||
|
let priv = if ($aws_priv_cidr_block | is-not-empty) {
|
||||||
|
let aws_priv_vpc = (aws_create_private_vpc $aws_priv_cidr_block $task)
|
||||||
|
if ($aws_priv_vpc | is-empty) {
|
||||||
|
print $"❗ aws_priv_vpc not found in (_ansi yellow_bold)($provider_path | path basename)(_ansi reset) "
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
let aws_priv_subnet_data = (aws_create_private_subnet $aws_priv_cidr_block $aws_priv_vpc $aws_avail_zone $task)
|
||||||
|
if (is-debug-enabled) { print $aws_priv_subnet_data }
|
||||||
|
let aws_priv_subnet = ($aws_priv_subnet_data | get -o SubnetId | default "")
|
||||||
|
if ($aws_priv_subnet | is-empty) {
|
||||||
|
print $"❗aws_priv_subnet not found in (_ansi yellow_bold)($provider_path | path basename)(_ansi reset) "
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
let aws_priv_avail_zone = ($aws_priv_subnet_data | get -o AvailabilityZone | default "")
|
||||||
|
let aws_priv_sg_name = ($data_settings | get -o priv | get -o sg | get -o name | default "sg_priv")
|
||||||
|
if ($aws_priv_sg_name | is-empty) {
|
||||||
|
print $"❗aws_priv_sg.name not found in (_ansi yellow_bold)($provider_path | path basename)(_ansi reset)"
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
let aws_priv_sg_id = (aws_create_sg_id $aws_priv_vpc $aws_priv_sg_name $task)
|
||||||
|
let aws_priv_sg_data = {
|
||||||
|
id: $aws_priv_sg_id,
|
||||||
|
name: $aws_priv_sg_name,
|
||||||
|
perms: ($env_settings | get -o priv | get -o sg | get -o perms | default [])
|
||||||
|
}
|
||||||
|
if $task == "create" or $task == "scan" { aws_add_sg_perms $aws_priv_sg_data $server $check_mode}
|
||||||
|
{
|
||||||
|
vpc: $aws_priv_vpc,
|
||||||
|
subnet: $aws_priv_subnet,
|
||||||
|
cidr_block: $aws_priv_cidr_block,
|
||||||
|
avail_zone: $aws_priv_avail_zone,
|
||||||
|
sg: $aws_priv_sg_data
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
if (is-debug-enabled) { print$"❗aws_priv_cidr_block not found in (_ansi yellow_bold)($provider_path | path basename)(_ansi reset) " }
|
||||||
|
}
|
||||||
|
let aws_sg_name = ($data_settings | get -o sg | get -o name | default "sg_pub")
|
||||||
|
if ($aws_sg_name | is-empty) {
|
||||||
|
print $"aws_sg_name not found in (_ansi yellow_bold)($provider_path | path basename)(_ansi reset) "
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
let aws_vpc_subnet_data = (aws_vpc_subnet $aws_avail_zone $priv.subnet $task)
|
||||||
|
if $task == "create" and ($aws_vpc_subnet_data | is-empty) {
|
||||||
|
print $"❗No availability zone ($aws_avail_zone) "
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
print $aws_vpc_subnet_data
|
||||||
|
let aws_sg_id = (aws_create_sg_id $aws_vpc $aws_sg_name $task)
|
||||||
|
let aws_sg_data = {
|
||||||
|
id: $aws_sg_id,
|
||||||
|
name: $aws_sg_name,
|
||||||
|
perms: ($env_settings | get -o main | get -o sg | get -o perms | default [])
|
||||||
|
}
|
||||||
|
if $task == "create" or $task == "scan" { aws_add_sg_perms $aws_sg_data $server $check_mode }
|
||||||
|
let main = {
|
||||||
|
vpc: $aws_vpc,
|
||||||
|
subnet: $aws_subnet,
|
||||||
|
cidr_block: ($aws_subnet_data | get -o CidrBlock | default ""),
|
||||||
|
avail_zone: $aws_avail_zone,
|
||||||
|
sg: $aws_sg_data,
|
||||||
|
}
|
||||||
|
let data_settings = if ($aws_priv_cidr_block | is-not-empty) {
|
||||||
|
{ main: $main, priv: $priv }
|
||||||
|
} else {
|
||||||
|
{ main: $main }
|
||||||
|
}
|
||||||
|
save_provider_env (load_provider_env $settings $server $provider_path | merge $data_settings) $settings $provider_path
|
||||||
|
print $"✅ (_ansi green_bold)AWS(_ansi reset) (_ansi cyan_bold)settings(_ansi reset) completed in (_ansi yellow_bold)($provider_path | path basename)(_ansi reset)"
|
||||||
|
}
|
||||||
|
export def aws_scan_servers [
|
||||||
|
provider_path: string
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
mut servers = []
|
||||||
|
for server_info in ($settings.data.servers? | default []) {
|
||||||
|
let hostname = ($server_info | get -o hostname | default "" )
|
||||||
|
let network_private_ip = ($server_info | get -o network_private_ip | default "")
|
||||||
|
let res = (^aws ec2 describe-instances --out json --filters $'"Name=tag:hostname,Values=($hostname)"' --filters "Name=instance-state-name,Values=running"
|
||||||
|
--query "Reservations[*].Instances[].{
|
||||||
|
id: InstanceId,
|
||||||
|
priv: NetworkInterfaces[],
|
||||||
|
pub: PublicIpAddress,
|
||||||
|
type: InstanceType,
|
||||||
|
status: State.Name,
|
||||||
|
keyname: KeyName,
|
||||||
|
launchtime: LaunchTime,
|
||||||
|
block_devices: BlockDeviceMappings
|
||||||
|
}"
|
||||||
|
--output json | complete)
|
||||||
|
if $res.exit_code != 0 {
|
||||||
|
print $"❗No data found for ($hostname) in ($server.provider) "
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
for instance_data in ($res.stdout | from json ) {
|
||||||
|
if ($instance_data | get -o status | str contains "erminated") { continue }
|
||||||
|
let instance_id = ($instance_data | get -o id | default "")
|
||||||
|
mut volumes = []
|
||||||
|
for device in ($instance_data | get -o block_devices | default []) {
|
||||||
|
let vol_id = ($device | get -o Ebs | get -o VolumeId | default "")
|
||||||
|
if ($vol_id | is-empty) { continue }
|
||||||
|
let res_vols = (^aws ec2 describe-volumes --volume-id $vol_id --filters $"Name=attachment.instance-id,Values=($instance_id)"
|
||||||
|
--query "Volumes[]" --output=json | complete)
|
||||||
|
if $res_vols.exit_code == 0 {
|
||||||
|
$volumes = ($volumes | append ($res_vols.stdout | from json))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
$servers = ($servers | append {
|
||||||
|
id: $instance_id
|
||||||
|
hostname: $hostname
|
||||||
|
keyname: ($instance_data | get -o keyname | default ""),
|
||||||
|
private_ips: ($instance_data | get -o priv | default []),
|
||||||
|
puplic_ips: ($instance_data | get -o pub | default []),
|
||||||
|
volumes: $volumes,
|
||||||
|
devices: ($instance_data | get -o block_devices | default []),
|
||||||
|
launchtime: ($instance_data | get -o launchtime | default ""),
|
||||||
|
info: $server_info
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
save_provider_env (load_provider_env $settings $server $provider_path | merge { servers: $servers}) $settings $provider_path
|
||||||
|
print $"✅ (_ansi green_bold)AWS(_ansi reset) (_ansi blue_bold)servers settings(_ansi reset)
|
||||||
|
completed in (_ansi yellow_bold)($provider_path | path basename)(_ansi reset)"
|
||||||
|
}
|
||||||
6
providers/aws/nulib/aws/mod.nu
Normal file
6
providers/aws/nulib/aws/mod.nu
Normal file
@ -0,0 +1,6 @@
|
|||||||
|
use env.nu
|
||||||
|
export use lib.nu *
|
||||||
|
export use servers.nu *
|
||||||
|
export use usage.nu *
|
||||||
|
export use prices.nu *
|
||||||
|
export use utils.nu *
|
||||||
6
providers/aws/nulib/aws/mod.nu-e
Normal file
6
providers/aws/nulib/aws/mod.nu-e
Normal file
@ -0,0 +1,6 @@
|
|||||||
|
use env.nu
|
||||||
|
export use lib.nu *
|
||||||
|
export use servers.nu *
|
||||||
|
export use usage.nu *
|
||||||
|
export use prices.nu *
|
||||||
|
export use utils.nu *
|
||||||
251
providers/aws/nulib/aws/prices.nu
Normal file
251
providers/aws/nulib/aws/prices.nu
Normal file
@ -0,0 +1,251 @@
|
|||||||
|
use ../../../../../core/nulib/lib_provisioning/utils/format.nu money_conversion
|
||||||
|
use ../../../../../core/nulib/lib_provisioning/config/accessor.nu *
|
||||||
|
|
||||||
|
def aws_default_store_type [] {
|
||||||
|
"Provisioned IOPS"
|
||||||
|
}
|
||||||
|
export def aws_get_price [
|
||||||
|
all_data: record
|
||||||
|
key: string
|
||||||
|
price_col: string = "pricePerUnit"
|
||||||
|
] {
|
||||||
|
let data = ($all_data | get -o item)
|
||||||
|
let str_price_col = if ($price_col | is-empty) { "pricePerUnit" } else { $price_col }
|
||||||
|
let value = ($data | get -o $str_price_col | get -o "USD" | default "")
|
||||||
|
let val = if ($value | is-empty) {
|
||||||
|
0
|
||||||
|
} else {
|
||||||
|
money_conversion "USD" "EUR" ($value | into float)
|
||||||
|
}
|
||||||
|
let unit = $"($val) ($data | get -o unit | default "")"
|
||||||
|
if ($unit | str contains "Hrs") {
|
||||||
|
match $key {
|
||||||
|
"month" => (($val * 24) * 30),
|
||||||
|
"day" => ($val * 24),
|
||||||
|
"hour" => ($val),
|
||||||
|
"minute" => ($val / 60),
|
||||||
|
"unit" => $unit,
|
||||||
|
}
|
||||||
|
} else if ($unit | str contains "Mo") {
|
||||||
|
match $key {
|
||||||
|
"month" => $val,
|
||||||
|
"day" => ($val / 30),
|
||||||
|
"hour" => (($val / 30) / 24),
|
||||||
|
"minute" => ((($val / 30) / 24) / 60),
|
||||||
|
"unit" => $unit,
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
0
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_get_provider_path [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
let data_path = if ($settings.data.prov_data_dirpath | str starts-with "." ) {
|
||||||
|
($settings.src_path | path join $settings.data.prov_data_dirpath)
|
||||||
|
} else { $settings.data.prov_data_dirpath }
|
||||||
|
if not ($data_path | path exists) { mkdir $data_path }
|
||||||
|
($data_path | path join $"($server.provider)_prices.((get-provisioning-wk-format))")
|
||||||
|
}
|
||||||
|
export def aws_get_item_for_server [
|
||||||
|
server: record
|
||||||
|
settings: record
|
||||||
|
cloud_data: record
|
||||||
|
] {
|
||||||
|
let provider_prices_path = (aws_get_provider_path $settings $server)
|
||||||
|
if not ($provider_prices_path | path exists) { return {} }
|
||||||
|
let pricing_data = (open $provider_prices_path | default [])
|
||||||
|
let memory = $"(($server.reqplan.memory | default 1024) / 1024) GiB"
|
||||||
|
let cores = $"($server.reqplan.cores | default 1)"
|
||||||
|
let current_gen = if ($server.reqplan.gen | default "") == "current" { "Yes" } else { "No" }
|
||||||
|
#let arch = if ($server.reqplan.arch | str contains "x86_64") { "Intel" } else { ""}
|
||||||
|
for item in $pricing_data {
|
||||||
|
if ($item | get -o data | is-empty) or ($item | get -o plan | is-empty) { continue }
|
||||||
|
if ($item.plan != $server.plan and $item.zone != $server.zone) { continue }
|
||||||
|
for it in $item.data {
|
||||||
|
if ($it | get -o product | is-empty) { continue }
|
||||||
|
if ( $it.product.attributes.memory == $memory
|
||||||
|
and $it.product.attributes.vcpu == $cores
|
||||||
|
and $it.product.attributes.currentGeneration == $current_gen
|
||||||
|
and ($it.product.attributes.operatingSystem | str contains "Linux")
|
||||||
|
) {
|
||||||
|
return ($it.on_demand | get -o priceDimensions | default {})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
{}
|
||||||
|
}
|
||||||
|
export def aws_get_item_for_storage [
|
||||||
|
server: record
|
||||||
|
settings: record
|
||||||
|
cloud_data: record
|
||||||
|
] {
|
||||||
|
let provider_prices_path = (aws_get_provider_path $settings $server)
|
||||||
|
if not ($provider_prices_path | path exists) { return [] }
|
||||||
|
let pricing_data = (open $provider_prices_path | default [])
|
||||||
|
if ($pricing_data | length) == 0 { return [] }
|
||||||
|
let default_store_type = aws_default_store_type
|
||||||
|
mut $data = []
|
||||||
|
for store in ($server | get -o storages | default []) {
|
||||||
|
let store_type = ($store | get -o prov_type | default $default_store_type)
|
||||||
|
for item in $pricing_data {
|
||||||
|
let item_type = ($item | get -o store | default "")
|
||||||
|
if ($item_type | is-empty) or $item_type != $store_type and $item.zone != $server.zone { continue }
|
||||||
|
if ($item | get -o data | is-empty) { continue }
|
||||||
|
let item_type = ($item | get -o store | default "")
|
||||||
|
return ($item | get data | get -o 0 | get -o on_demand | get -o priceDimensions | default {})
|
||||||
|
# $data = ($data | append ($item | get data | get -o 0 | get -o on_demand | get -o priceDimensions | default {}))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
{}
|
||||||
|
#$data
|
||||||
|
}
|
||||||
|
export def aws_load_infra_servers_info [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
] {
|
||||||
|
let provider_prices_path = (aws_get_provider_path $settings $server)
|
||||||
|
if ($provider_prices_path | path exists) {
|
||||||
|
let pricing_data = (open $provider_prices_path)
|
||||||
|
for it in $pricing_data {
|
||||||
|
let zone = ($it | get -o zone | default "")
|
||||||
|
let plan = ($it | get -o plan | default "")
|
||||||
|
if $zone == $server.zone and $plan == $server.plan {
|
||||||
|
return {plan: $plan, zone: $zone }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
(aws_load_infra_servers $provider_prices_path $settings $server)
|
||||||
|
}
|
||||||
|
export def aws_load_infra_storages_info [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
] {
|
||||||
|
let provider_prices_path = (aws_get_provider_path $settings $server)
|
||||||
|
if ($provider_prices_path | path exists) {
|
||||||
|
let default_store_type = aws_default_store_type
|
||||||
|
let pricing_data = (open $provider_prices_path)
|
||||||
|
for it in $pricing_data {
|
||||||
|
let zone = ($it | get -o zone | default "")
|
||||||
|
let store = ($it | get -o store | default "")
|
||||||
|
if $zone == $server.zone and $store == $default_store_type {
|
||||||
|
return {zone: $zone, store: $store }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
aws_load_infra_storages $provider_prices_path $settings $server
|
||||||
|
}
|
||||||
|
export def aws_get_price_data [
|
||||||
|
filter: record
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
let res = (^aws pricing get-products --service-code AmazonEC2 --filters
|
||||||
|
$"Type=TERM_MATCH,Field=($filter.field),Value=($filter.value)" $"Type=TERM_MATCH,Field=regionCode,Value=($server.zone)"
|
||||||
|
--query "PriceList[]" --region us-east-1 --out json | complete
|
||||||
|
)
|
||||||
|
if $res.exit_code != 0 {
|
||||||
|
print $"❗ Errors on ($server.hostname) ($server.provider) ($server.plan) in ($server.zone) load cloud price data error: ($res.stdout ) "
|
||||||
|
return
|
||||||
|
}
|
||||||
|
# | str replace '\' ''| str replace '"{' '{' | str replace '}"' '}')
|
||||||
|
mut $data = []
|
||||||
|
for it in ($res.stdout | from json) {
|
||||||
|
let it_data = ($it | from json)
|
||||||
|
|
||||||
|
let product = ($it_data | get -o product | default {})
|
||||||
|
if ($product | is-empty) { continue }
|
||||||
|
|
||||||
|
#let attributes = ($product | get -o attributes | default {})
|
||||||
|
let on_demand_key = ($it_data | get -o terms | get -o OnDemand | columns | first)
|
||||||
|
let on_demand_data = ($it_data | get -o terms | get -o OnDemand | get -o $on_demand_key | default {})
|
||||||
|
let price_dimension = if ($on_demand_data | is-not-empty) {
|
||||||
|
let price_dimension_key = ($on_demand_data | get -o priceDimensions | columns | first | default "")
|
||||||
|
($on_demand_data | get -o priceDimensions | get -o $price_dimension_key | default {})
|
||||||
|
} else {
|
||||||
|
{}
|
||||||
|
}
|
||||||
|
$data = ( $data | append {
|
||||||
|
product: $product,
|
||||||
|
on_demand: {
|
||||||
|
priceDimensions: $price_dimension,
|
||||||
|
sku: ($on_demand_data | get -o sku),
|
||||||
|
effectiveDate: ($on_demand_data | get -o effectiveDate),
|
||||||
|
offerTermCode: ($on_demand_data | get -o offerTermCode),
|
||||||
|
termAttributes: ($on_demand_data | get -o termAttributes)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
$data
|
||||||
|
}
|
||||||
|
export def aws_load_infra_storages [
|
||||||
|
provider_prices_path: string
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
let default_store_type = aws_default_store_type
|
||||||
|
let curr_data = if ($provider_prices_path | path exists) {
|
||||||
|
(open $provider_prices_path)
|
||||||
|
} else {
|
||||||
|
[]
|
||||||
|
}
|
||||||
|
$curr_data | where {|it|
|
||||||
|
if $it.zone == $server.zone and $it.store? != null and $it.store == $default_store_type {
|
||||||
|
print $it
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let filter = {
|
||||||
|
field: "volumeType",
|
||||||
|
value: $default_store_type
|
||||||
|
}
|
||||||
|
let data = (aws_get_price_data $filter $server)
|
||||||
|
let srv_data = { zone: $server.zone, store: $default_store_type, data: $data}
|
||||||
|
let all_data = if ($provider_prices_path | path exists) {
|
||||||
|
(open $provider_prices_path | append $srv_data)
|
||||||
|
} else {
|
||||||
|
[$srv_data]
|
||||||
|
}
|
||||||
|
if (get-provisioning-wk-format) == "json" {
|
||||||
|
$all_data | to json | save -f $provider_prices_path
|
||||||
|
} else {
|
||||||
|
$all_data | to yaml | save -f $provider_prices_path
|
||||||
|
}
|
||||||
|
if (is-debug-enabled) { print $"Storage prices for ($server.provider) in: ($provider_prices_path | path basename) with ($server.zone) saved" }
|
||||||
|
}
|
||||||
|
export def aws_load_infra_servers [
|
||||||
|
provider_prices_path: string
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
let curr_data = if ($provider_prices_path | path exists) {
|
||||||
|
(open $provider_prices_path)
|
||||||
|
} else {
|
||||||
|
[]
|
||||||
|
}
|
||||||
|
$curr_data | where {|it|
|
||||||
|
if $it.zone? != null and $it.zone == $server.zone and $it.plan? != null and $it.plan == $server.plan {
|
||||||
|
return $curr_data
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let filter = {
|
||||||
|
field: "instanceType",
|
||||||
|
value: $server.plan
|
||||||
|
}
|
||||||
|
let data = (aws_get_price_data $filter $server)
|
||||||
|
let srv_data = { zone: $server.zone, plan: $server.plan, data: $data}
|
||||||
|
let all_data = if ($provider_prices_path | path exists) {
|
||||||
|
(open $provider_prices_path | append $srv_data)
|
||||||
|
} else {
|
||||||
|
[$srv_data]
|
||||||
|
}
|
||||||
|
if (get-provisioning-wk-format) == "json" {
|
||||||
|
$all_data | to json | save -f $provider_prices_path
|
||||||
|
} else {
|
||||||
|
$all_data | to yaml | save -f $provider_prices_path
|
||||||
|
}
|
||||||
|
if (is-debug-enabled) { print $"Server prices for ($server.provider) in: ($provider_prices_path | path basename) with ($server.plan)/($server.zone) saved" }
|
||||||
|
{ plan: $server.plan, zone: $server.zone }
|
||||||
|
}
|
||||||
251
providers/aws/nulib/aws/prices.nu-e
Normal file
251
providers/aws/nulib/aws/prices.nu-e
Normal file
@ -0,0 +1,251 @@
|
|||||||
|
use ../../../../core/nulib/lib_provisioning/utils/format.nu money_conversion
|
||||||
|
use ../../../../core/nulib/lib_provisioning/config/accessor.nu *
|
||||||
|
|
||||||
|
def aws_default_store_type [] {
|
||||||
|
"Provisioned IOPS"
|
||||||
|
}
|
||||||
|
export def aws_get_price [
|
||||||
|
all_data: record
|
||||||
|
key: string
|
||||||
|
price_col: string = "pricePerUnit"
|
||||||
|
] {
|
||||||
|
let data = ($all_data | get -o item)
|
||||||
|
let str_price_col = if ($price_col | is-empty) { "pricePerUnit" } else { $price_col }
|
||||||
|
let value = ($data | get -o $str_price_col | get -o "USD" | default "")
|
||||||
|
let val = if ($value | is-empty) {
|
||||||
|
0
|
||||||
|
} else {
|
||||||
|
money_conversion "USD" "EUR" ($value | into float)
|
||||||
|
}
|
||||||
|
let unit = $"($val) ($data | get -o unit | default "")"
|
||||||
|
if ($unit | str contains "Hrs") {
|
||||||
|
match $key {
|
||||||
|
"month" => (($val * 24) * 30),
|
||||||
|
"day" => ($val * 24),
|
||||||
|
"hour" => ($val),
|
||||||
|
"minute" => ($val / 60),
|
||||||
|
"unit" => $unit,
|
||||||
|
}
|
||||||
|
} else if ($unit | str contains "Mo") {
|
||||||
|
match $key {
|
||||||
|
"month" => $val,
|
||||||
|
"day" => ($val / 30),
|
||||||
|
"hour" => (($val / 30) / 24),
|
||||||
|
"minute" => ((($val / 30) / 24) / 60),
|
||||||
|
"unit" => $unit,
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
0
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def aws_get_provider_path [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
let data_path = if ($settings.data.prov_data_dirpath | str starts-with "." ) {
|
||||||
|
($settings.src_path | path join $settings.data.prov_data_dirpath)
|
||||||
|
} else { $settings.data.prov_data_dirpath }
|
||||||
|
if not ($data_path | path exists) { mkdir $data_path }
|
||||||
|
($data_path | path join $"($server.provider)_prices.((get-provisioning-wk-format))")
|
||||||
|
}
|
||||||
|
export def aws_get_item_for_server [
|
||||||
|
server: record
|
||||||
|
settings: record
|
||||||
|
cloud_data: record
|
||||||
|
] {
|
||||||
|
let provider_prices_path = (aws_get_provider_path $settings $server)
|
||||||
|
if not ($provider_prices_path | path exists) { return {} }
|
||||||
|
let pricing_data = (open $provider_prices_path | default [])
|
||||||
|
let memory = $"(($server.reqplan.memory | default 1024) / 1024) GiB"
|
||||||
|
let cores = $"($server.reqplan.cores | default 1)"
|
||||||
|
let current_gen = if ($server.reqplan.gen | default "") == "current" { "Yes" } else { "No" }
|
||||||
|
#let arch = if ($server.reqplan.arch | str contains "x86_64") { "Intel" } else { ""}
|
||||||
|
for item in $pricing_data {
|
||||||
|
if ($item | get -o data | is-empty) or ($item | get -o plan | is-empty) { continue }
|
||||||
|
if ($item.plan != $server.plan and $item.zone != $server.zone) { continue }
|
||||||
|
for it in $item.data {
|
||||||
|
if ($it | get -o product | is-empty) { continue }
|
||||||
|
if ( $it.product.attributes.memory == $memory
|
||||||
|
and $it.product.attributes.vcpu == $cores
|
||||||
|
and $it.product.attributes.currentGeneration == $current_gen
|
||||||
|
and ($it.product.attributes.operatingSystem | str contains "Linux")
|
||||||
|
) {
|
||||||
|
return ($it.on_demand | get -o priceDimensions | default {})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
{}
|
||||||
|
}
|
||||||
|
export def aws_get_item_for_storage [
|
||||||
|
server: record
|
||||||
|
settings: record
|
||||||
|
cloud_data: record
|
||||||
|
] {
|
||||||
|
let provider_prices_path = (aws_get_provider_path $settings $server)
|
||||||
|
if not ($provider_prices_path | path exists) { return [] }
|
||||||
|
let pricing_data = (open $provider_prices_path | default [])
|
||||||
|
if ($pricing_data | length) == 0 { return [] }
|
||||||
|
let default_store_type = aws_default_store_type
|
||||||
|
mut $data = []
|
||||||
|
for store in ($server | get -o storages | default []) {
|
||||||
|
let store_type = ($store | get -o prov_type | default $default_store_type)
|
||||||
|
for item in $pricing_data {
|
||||||
|
let item_type = ($item | get -o store | default "")
|
||||||
|
if ($item_type | is-empty) or $item_type != $store_type and $item.zone != $server.zone { continue }
|
||||||
|
if ($item | get -o data | is-empty) { continue }
|
||||||
|
let item_type = ($item | get -o store | default "")
|
||||||
|
return ($item | get data | get -o 0 | get -o on_demand | get -o priceDimensions | default {})
|
||||||
|
# $data = ($data | append ($item | get data | get -o 0 | get -o on_demand | get -o priceDimensions | default {}))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
{}
|
||||||
|
#$data
|
||||||
|
}
|
||||||
|
export def aws_load_infra_servers_info [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
] {
|
||||||
|
let provider_prices_path = (aws_get_provider_path $settings $server)
|
||||||
|
if ($provider_prices_path | path exists) {
|
||||||
|
let pricing_data = (open $provider_prices_path)
|
||||||
|
for it in $pricing_data {
|
||||||
|
let zone = ($it | get -o zone | default "")
|
||||||
|
let plan = ($it | get -o plan | default "")
|
||||||
|
if $zone == $server.zone and $plan == $server.plan {
|
||||||
|
return {plan: $plan, zone: $zone }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
(aws_load_infra_servers $provider_prices_path $settings $server)
|
||||||
|
}
|
||||||
|
export def aws_load_infra_storages_info [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
] {
|
||||||
|
let provider_prices_path = (aws_get_provider_path $settings $server)
|
||||||
|
if ($provider_prices_path | path exists) {
|
||||||
|
let default_store_type = aws_default_store_type
|
||||||
|
let pricing_data = (open $provider_prices_path)
|
||||||
|
for it in $pricing_data {
|
||||||
|
let zone = ($it | get -o zone | default "")
|
||||||
|
let store = ($it | get -o store | default "")
|
||||||
|
if $zone == $server.zone and $store == $default_store_type {
|
||||||
|
return {zone: $zone, store: $store }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
aws_load_infra_storages $provider_prices_path $settings $server
|
||||||
|
}
|
||||||
|
export def aws_get_price_data [
|
||||||
|
filter: record
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
let res = (^aws pricing get-products --service-code AmazonEC2 --filters
|
||||||
|
$"Type=TERM_MATCH,Field=($filter.field),Value=($filter.value)" $"Type=TERM_MATCH,Field=regionCode,Value=($server.zone)"
|
||||||
|
--query "PriceList[]" --region us-east-1 --out json | complete
|
||||||
|
)
|
||||||
|
if $res.exit_code != 0 {
|
||||||
|
print $"❗ Errors on ($server.hostname) ($server.provider) ($server.plan) in ($server.zone) load cloud price data error: ($res.stdout ) "
|
||||||
|
return
|
||||||
|
}
|
||||||
|
# | str replace '\' ''| str replace '"{' '{' | str replace '}"' '}')
|
||||||
|
mut $data = []
|
||||||
|
for it in ($res.stdout | from json) {
|
||||||
|
let it_data = ($it | from json)
|
||||||
|
|
||||||
|
let product = ($it_data | get -o product | default {})
|
||||||
|
if ($product | is-empty) { continue }
|
||||||
|
|
||||||
|
#let attributes = ($product | get -o attributes | default {})
|
||||||
|
let on_demand_key = ($it_data | get -o terms | get -o OnDemand | columns | first)
|
||||||
|
let on_demand_data = ($it_data | get -o terms | get -o OnDemand | get -o $on_demand_key | default {})
|
||||||
|
let price_dimension = if ($on_demand_data | is-not-empty) {
|
||||||
|
let price_dimension_key = ($on_demand_data | get -o priceDimensions | columns | first | default "")
|
||||||
|
($on_demand_data | get -o priceDimensions | get -o $price_dimension_key | default {})
|
||||||
|
} else {
|
||||||
|
{}
|
||||||
|
}
|
||||||
|
$data = ( $data | append {
|
||||||
|
product: $product,
|
||||||
|
on_demand: {
|
||||||
|
priceDimensions: $price_dimension,
|
||||||
|
sku: ($on_demand_data | get -o sku),
|
||||||
|
effectiveDate: ($on_demand_data | get -o effectiveDate),
|
||||||
|
offerTermCode: ($on_demand_data | get -o offerTermCode),
|
||||||
|
termAttributes: ($on_demand_data | get -o termAttributes)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
$data
|
||||||
|
}
|
||||||
|
export def aws_load_infra_storages [
|
||||||
|
provider_prices_path: string
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
let default_store_type = aws_default_store_type
|
||||||
|
let curr_data = if ($provider_prices_path | path exists) {
|
||||||
|
(open $provider_prices_path)
|
||||||
|
} else {
|
||||||
|
[]
|
||||||
|
}
|
||||||
|
$curr_data | where {|it|
|
||||||
|
if $it.zone == $server.zone and $it.store? != null and $it.store == $default_store_type {
|
||||||
|
print $it
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let filter = {
|
||||||
|
field: "volumeType",
|
||||||
|
value: $default_store_type
|
||||||
|
}
|
||||||
|
let data = (aws_get_price_data $filter $server)
|
||||||
|
let srv_data = { zone: $server.zone, store: $default_store_type, data: $data}
|
||||||
|
let all_data = if ($provider_prices_path | path exists) {
|
||||||
|
(open $provider_prices_path | append $srv_data)
|
||||||
|
} else {
|
||||||
|
[$srv_data]
|
||||||
|
}
|
||||||
|
if (get-provisioning-wk-format) == "json" {
|
||||||
|
$all_data | to json | save -f $provider_prices_path
|
||||||
|
} else {
|
||||||
|
$all_data | to yaml | save -f $provider_prices_path
|
||||||
|
}
|
||||||
|
if (is-debug-enabled) { print $"Storage prices for ($server.provider) in: ($provider_prices_path | path basename) with ($server.zone) saved" }
|
||||||
|
}
|
||||||
|
export def aws_load_infra_servers [
|
||||||
|
provider_prices_path: string
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
let curr_data = if ($provider_prices_path | path exists) {
|
||||||
|
(open $provider_prices_path)
|
||||||
|
} else {
|
||||||
|
[]
|
||||||
|
}
|
||||||
|
$curr_data | where {|it|
|
||||||
|
if $it.zone? != null and $it.zone == $server.zone and $it.plan? != null and $it.plan == $server.plan {
|
||||||
|
return $curr_data
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let filter = {
|
||||||
|
field: "instanceType",
|
||||||
|
value: $server.plan
|
||||||
|
}
|
||||||
|
let data = (aws_get_price_data $filter $server)
|
||||||
|
let srv_data = { zone: $server.zone, plan: $server.plan, data: $data}
|
||||||
|
let all_data = if ($provider_prices_path | path exists) {
|
||||||
|
(open $provider_prices_path | append $srv_data)
|
||||||
|
} else {
|
||||||
|
[$srv_data]
|
||||||
|
}
|
||||||
|
if (get-provisioning-wk-format) == "json" {
|
||||||
|
$all_data | to json | save -f $provider_prices_path
|
||||||
|
} else {
|
||||||
|
$all_data | to yaml | save -f $provider_prices_path
|
||||||
|
}
|
||||||
|
if (is-debug-enabled) { print $"Server prices for ($server.provider) in: ($provider_prices_path | path basename) with ($server.plan)/($server.zone) saved" }
|
||||||
|
{ plan: $server.plan, zone: $server.zone }
|
||||||
|
}
|
||||||
1101
providers/aws/nulib/aws/servers.nu
Normal file
1101
providers/aws/nulib/aws/servers.nu
Normal file
File diff suppressed because it is too large
Load Diff
1101
providers/aws/nulib/aws/servers.nu-e
Normal file
1101
providers/aws/nulib/aws/servers.nu-e
Normal file
File diff suppressed because it is too large
Load Diff
41
providers/aws/nulib/aws/usage.nu
Normal file
41
providers/aws/nulib/aws/usage.nu
Normal file
@ -0,0 +1,41 @@
|
|||||||
|
|
||||||
|
#!/usr/bin/env nu
|
||||||
|
|
||||||
|
# myscript.nu
|
||||||
|
export def usage [provider: string, infra: string] {
|
||||||
|
let info = if ( $env.CURRENT_FILE? | into string ) != "" { (^grep "^# Info:" $env.CURRENT_FILE ) | str replace "# Info: " "" } else { "" }
|
||||||
|
# $(declare -F _usage_options >/dev/null && _usage_options)
|
||||||
|
$"
|
||||||
|
USAGE provisioning ($provider) -k cloud-path file-settings.yaml provider-options
|
||||||
|
DESCRIPTION
|
||||||
|
AWS ($info)
|
||||||
|
OPTIONS
|
||||||
|
-s server-hostname
|
||||||
|
with server-hostname target selection
|
||||||
|
-p provider-name
|
||||||
|
use provider name
|
||||||
|
do not need if 'current directory path basename' is not one of providers available
|
||||||
|
-new | new [provisioning-name]
|
||||||
|
create a new provisioning-directory-name by a copy of ($infra)
|
||||||
|
-k cloud-path-item
|
||||||
|
use cloud-path-item as base directory for settings
|
||||||
|
-x
|
||||||
|
Trace script with 'set -x'
|
||||||
|
providerslist | providers-list | providers list
|
||||||
|
Get available providers list
|
||||||
|
taskslist | tasks-list | tasks list
|
||||||
|
Get available tasks list
|
||||||
|
serviceslist | service-list
|
||||||
|
Get available services list
|
||||||
|
tools
|
||||||
|
Run core/on-tools info
|
||||||
|
-i
|
||||||
|
About this
|
||||||
|
-v
|
||||||
|
Print version
|
||||||
|
-h, --help
|
||||||
|
Print this help and exit.
|
||||||
|
"
|
||||||
|
# ["hello" $name $title]
|
||||||
|
}
|
||||||
|
|
||||||
41
providers/aws/nulib/aws/usage.nu-e
Normal file
41
providers/aws/nulib/aws/usage.nu-e
Normal file
@ -0,0 +1,41 @@
|
|||||||
|
|
||||||
|
#!/usr/bin/env nu
|
||||||
|
|
||||||
|
# myscript.nu
|
||||||
|
export def usage [provider: string, infra: string] {
|
||||||
|
let info = if ( $env.CURRENT_FILE? | into string ) != "" { (^grep "^# Info:" $env.CURRENT_FILE ) | str replace "# Info: " "" } else { "" }
|
||||||
|
# $(declare -F _usage_options >/dev/null && _usage_options)
|
||||||
|
$"
|
||||||
|
USAGE provisioning ($provider) -k cloud-path file-settings.yaml provider-options
|
||||||
|
DESCRIPTION
|
||||||
|
AWS ($info)
|
||||||
|
OPTIONS
|
||||||
|
-s server-hostname
|
||||||
|
with server-hostname target selection
|
||||||
|
-p provider-name
|
||||||
|
use provider name
|
||||||
|
do not need if 'current directory path basename' is not one of providers available
|
||||||
|
-new | new [provisioning-name]
|
||||||
|
create a new provisioning-directory-name by a copy of ($infra)
|
||||||
|
-k cloud-path-item
|
||||||
|
use cloud-path-item as base directory for settings
|
||||||
|
-x
|
||||||
|
Trace script with 'set -x'
|
||||||
|
providerslist | providers-list | providers list
|
||||||
|
Get available providers list
|
||||||
|
taskslist | tasks-list | tasks list
|
||||||
|
Get available tasks list
|
||||||
|
serviceslist | service-list
|
||||||
|
Get available services list
|
||||||
|
tools
|
||||||
|
Run core/on-tools info
|
||||||
|
-i
|
||||||
|
About this
|
||||||
|
-v
|
||||||
|
Print version
|
||||||
|
-h, --help
|
||||||
|
Print this help and exit.
|
||||||
|
"
|
||||||
|
# ["hello" $name $title]
|
||||||
|
}
|
||||||
|
|
||||||
26
providers/aws/nulib/aws/utils.nu
Normal file
26
providers/aws/nulib/aws/utils.nu
Normal file
@ -0,0 +1,26 @@
|
|||||||
|
use ../../../../../core/nulib/lib_provisioning/config/accessor.nu *
|
||||||
|
|
||||||
|
export def aws_check_requirements [
|
||||||
|
settings: record
|
||||||
|
fix_error: bool
|
||||||
|
] {
|
||||||
|
let has_aws = (^bash -c "type -P aws")
|
||||||
|
if ($has_aws | path exists) == false and $fix_error {
|
||||||
|
( ^((get-provisioning-name)) "tools" "install" "aws")
|
||||||
|
}
|
||||||
|
let has_aws = (^bash -c "type -P aws")
|
||||||
|
if ($has_aws | path exists) == false {
|
||||||
|
(throw-error $"🛑 CLI command aws not found"
|
||||||
|
"aws_check_requirements" --span (metadata $has_aws).span)
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
let aws_version = (^aws --version | cut -f1 -d" " | sed 's,aws-cli/,,g')
|
||||||
|
let req_version = (open (get-provisioning-req-versions)).aws?.version? | default ""
|
||||||
|
if ($aws_version != $req_version ) and $fix_error {
|
||||||
|
( ^((get-provisioning-name)) "tools" "update" "aws")
|
||||||
|
}
|
||||||
|
let aws_version = (^aws --version | cut -f1 -d" " | sed 's,aws-cli/,,g')
|
||||||
|
if $aws_version != $req_version {
|
||||||
|
print $"warning❗ aws command as CLI for AWS ($aws_version) with Provisioning is not ($req_version)"
|
||||||
|
}
|
||||||
|
}
|
||||||
26
providers/aws/nulib/aws/utils.nu-e
Normal file
26
providers/aws/nulib/aws/utils.nu-e
Normal file
@ -0,0 +1,26 @@
|
|||||||
|
use ../../../../core/nulib/lib_provisioning/config/accessor.nu *
|
||||||
|
|
||||||
|
export def aws_check_requirements [
|
||||||
|
settings: record
|
||||||
|
fix_error: bool
|
||||||
|
] {
|
||||||
|
let has_aws = (^bash -c "type -P aws")
|
||||||
|
if ($has_aws | path exists) == false and $fix_error {
|
||||||
|
( ^((get-provisioning-name)) "tools" "install" "aws")
|
||||||
|
}
|
||||||
|
let has_aws = (^bash -c "type -P aws")
|
||||||
|
if ($has_aws | path exists) == false {
|
||||||
|
(throw-error $"🛑 CLI command aws not found"
|
||||||
|
"aws_check_requirements" --span (metadata $has_aws).span)
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
let aws_version = (^aws --version | cut -f1 -d" " | sed 's,aws-cli/,,g')
|
||||||
|
let req_version = (open (get-provisioning-req-versions)).aws?.version? | default ""
|
||||||
|
if ($aws_version != $req_version ) and $fix_error {
|
||||||
|
( ^((get-provisioning-name)) "tools" "update" "aws")
|
||||||
|
}
|
||||||
|
let aws_version = (^aws --version | cut -f1 -d" " | sed 's,aws-cli/,,g')
|
||||||
|
if $aws_version != $req_version {
|
||||||
|
print $"warning❗ aws command as CLI for AWS ($aws_version) with Provisioning is not ($req_version)"
|
||||||
|
}
|
||||||
|
}
|
||||||
327
providers/aws/provider.nu
Normal file
327
providers/aws/provider.nu
Normal file
@ -0,0 +1,327 @@
|
|||||||
|
# AWS Provider Adapter
|
||||||
|
# Implements the standard provider interface for AWS
|
||||||
|
|
||||||
|
use nulib/aws/env.nu
|
||||||
|
use nulib/aws/servers.nu *
|
||||||
|
use nulib/aws/cache.nu *
|
||||||
|
use nulib/aws/prices.nu *
|
||||||
|
|
||||||
|
# Provider metadata
|
||||||
|
export def get-provider-metadata []: nothing -> record {
|
||||||
|
{
|
||||||
|
name: "aws"
|
||||||
|
version: "1.0.0"
|
||||||
|
description: "Amazon Web Services provider"
|
||||||
|
capabilities: {
|
||||||
|
server_management: true
|
||||||
|
network_management: true
|
||||||
|
storage_management: true
|
||||||
|
load_balancer: true
|
||||||
|
dns_management: false
|
||||||
|
cdn: true
|
||||||
|
backup_service: true
|
||||||
|
monitoring: true
|
||||||
|
logging: true
|
||||||
|
auto_scaling: true
|
||||||
|
spot_instances: true
|
||||||
|
containers: true
|
||||||
|
serverless: true
|
||||||
|
multi_region: true
|
||||||
|
encryption_at_rest: true
|
||||||
|
encryption_in_transit: true
|
||||||
|
compliance_certifications: ["SOC2", "ISO27001", "PCI-DSS", "HIPAA"]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Server query operations
|
||||||
|
export def query_servers [
|
||||||
|
find?: string
|
||||||
|
cols?: string
|
||||||
|
]: nothing -> list {
|
||||||
|
let str_find = if $find != null { $find } else { "" }
|
||||||
|
let str_cols = if $cols != null { $cols } else { "" }
|
||||||
|
aws_query_servers $str_find $str_cols
|
||||||
|
}
|
||||||
|
|
||||||
|
# Server information operations
|
||||||
|
export def server_info [
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
find?: string
|
||||||
|
cols?: string
|
||||||
|
]: nothing -> record {
|
||||||
|
aws_server_info $server $check
|
||||||
|
}
|
||||||
|
|
||||||
|
# Server existence and status operations
|
||||||
|
export def server_exists [
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
aws_server_exists $server $error_exit
|
||||||
|
}
|
||||||
|
|
||||||
|
export def server_is_running [
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
aws_server_is_running $server $error_exit
|
||||||
|
}
|
||||||
|
|
||||||
|
# Server lifecycle operations
|
||||||
|
export def check_server_requirements [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
aws_check_server_requirements $settings $server $check
|
||||||
|
}
|
||||||
|
|
||||||
|
export def create_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
wait: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
# AWS doesn't have a direct create_server function, it uses server_state
|
||||||
|
# Create server by setting state to "started"
|
||||||
|
try {
|
||||||
|
aws_server_state $server "started" true $wait $settings
|
||||||
|
true
|
||||||
|
} catch {
|
||||||
|
false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export def delete_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
keep_storage: bool
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
aws_delete_server $settings $server $keep_storage $error_exit
|
||||||
|
}
|
||||||
|
|
||||||
|
export def delete_server_storage [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
aws_delete_server_storage $settings $server $error_exit
|
||||||
|
}
|
||||||
|
|
||||||
|
export def post_create_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
aws_post_create_server $settings $server $check
|
||||||
|
}
|
||||||
|
|
||||||
|
export def modify_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
new_values: list
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
aws_modify_server $settings $server $new_values $error_exit
|
||||||
|
}
|
||||||
|
|
||||||
|
# Server state management
|
||||||
|
export def server_state [
|
||||||
|
server: record
|
||||||
|
new_state: string
|
||||||
|
error_exit: bool
|
||||||
|
wait: bool
|
||||||
|
settings: record
|
||||||
|
]: nothing -> bool {
|
||||||
|
aws_server_state $server $new_state $error_exit $wait $settings
|
||||||
|
}
|
||||||
|
|
||||||
|
# Network operations
|
||||||
|
export def get_ip [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
ip_type: string
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> string {
|
||||||
|
let use_type = match $ip_type {
|
||||||
|
"$network_public_ip" => "public"
|
||||||
|
"$network_private_ip" => "private"
|
||||||
|
_ => $ip_type
|
||||||
|
}
|
||||||
|
|
||||||
|
let result = (aws_server [ "get_ip", $use_type ] --server $server --settings $settings)
|
||||||
|
$result | str trim
|
||||||
|
}
|
||||||
|
|
||||||
|
export def servers_ips [
|
||||||
|
settings: record
|
||||||
|
data: list
|
||||||
|
prov?: string
|
||||||
|
serverpos?: int
|
||||||
|
]: nothing -> list {
|
||||||
|
# AWS-specific implementation for getting IPs from multiple servers
|
||||||
|
mut result = []
|
||||||
|
mut index = -1
|
||||||
|
|
||||||
|
for srv in $data {
|
||||||
|
$index += 1
|
||||||
|
let settings_server = ($settings.data.servers | where {|it| $it.hostname == $srv.hostname})
|
||||||
|
if ($settings_server | length) == 0 { continue }
|
||||||
|
let provider = ($settings_server | get -o 0 | get -o provider | default "")
|
||||||
|
if $prov != null and $provider != "aws" { continue }
|
||||||
|
if $serverpos != null and $serverpos != $index { continue }
|
||||||
|
|
||||||
|
if $srv.ip_addresses? != null {
|
||||||
|
$result = ($result | append ($srv.ip_addresses? |
|
||||||
|
each {|it| { hostname: $srv.hostname, ip: $it.address, access: $it.access, family: $it.family }} |
|
||||||
|
flatten
|
||||||
|
))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
$result
|
||||||
|
}
|
||||||
|
|
||||||
|
# Infrastructure operations
|
||||||
|
export def load_infra_servers_info [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> record {
|
||||||
|
aws_load_infra_servers_info $settings $server $error_exit
|
||||||
|
}
|
||||||
|
|
||||||
|
export def load_infra_storages_info [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> record {
|
||||||
|
aws_load_infra_storages_info $settings $server $error_exit
|
||||||
|
}
|
||||||
|
|
||||||
|
export def get_infra_storage [
|
||||||
|
server: record
|
||||||
|
settings: record
|
||||||
|
cloud_data: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> list {
|
||||||
|
aws_get_item_for_storage $server $settings $cloud_data
|
||||||
|
}
|
||||||
|
|
||||||
|
export def get_infra_item [
|
||||||
|
server: record
|
||||||
|
settings: record
|
||||||
|
cloud_data: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> record {
|
||||||
|
aws_get_item_for_server $server $settings $cloud_data
|
||||||
|
}
|
||||||
|
|
||||||
|
export def get_infra_price [
|
||||||
|
server: record
|
||||||
|
data: record
|
||||||
|
key: string
|
||||||
|
error_exit: bool
|
||||||
|
price_col?: string
|
||||||
|
]: nothing -> float {
|
||||||
|
if ($data | get -o item | is-empty) { return 0.0 }
|
||||||
|
aws_get_price $data $key $price_col
|
||||||
|
}
|
||||||
|
|
||||||
|
# Cache operations
|
||||||
|
export def start_cache_info [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
]: nothing -> nothing {
|
||||||
|
aws_start_cache_info $settings $server
|
||||||
|
}
|
||||||
|
|
||||||
|
export def create_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> nothing {
|
||||||
|
aws_create_cache $settings $server $error_exit
|
||||||
|
}
|
||||||
|
|
||||||
|
export def read_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> nothing {
|
||||||
|
aws_read_cache $settings $server $error_exit
|
||||||
|
}
|
||||||
|
|
||||||
|
export def clean_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> nothing {
|
||||||
|
aws_clean_cache $settings $server $error_exit
|
||||||
|
}
|
||||||
|
|
||||||
|
export def ip_from_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> nothing {
|
||||||
|
aws_ip_from_cache $settings $server $error_exit
|
||||||
|
}
|
||||||
|
|
||||||
|
# Provider metadata operations
|
||||||
|
export def on_prov_server [
|
||||||
|
server: record
|
||||||
|
]: nothing -> string {
|
||||||
|
aws_on_prov_server $server
|
||||||
|
}
|
||||||
|
|
||||||
|
# AWS-specific extensions (beyond standard interface)
|
||||||
|
export def create_private_network [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
]: nothing -> record {
|
||||||
|
aws_create_private_network $settings $server $check
|
||||||
|
}
|
||||||
|
|
||||||
|
export def get_ssh_key [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
]: nothing -> string {
|
||||||
|
aws_get_ssh_key $server
|
||||||
|
}
|
||||||
|
|
||||||
|
export def create_ssh_key [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
]: nothing -> bool {
|
||||||
|
aws_create_ssh_key $server
|
||||||
|
}
|
||||||
|
|
||||||
|
export def make_settings [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
]: nothing -> record {
|
||||||
|
aws_make_settings $settings $server
|
||||||
|
}
|
||||||
|
|
||||||
|
export def delete_settings [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
]: nothing -> nothing {
|
||||||
|
aws_delete_settings $settings $server
|
||||||
|
}
|
||||||
|
|
||||||
|
# Provider validation
|
||||||
|
export def validate_provider []: nothing -> record {
|
||||||
|
{
|
||||||
|
provider: "aws"
|
||||||
|
valid: true
|
||||||
|
interface_version: "1.0.0"
|
||||||
|
capabilities: (get-provider-metadata).capabilities
|
||||||
|
last_validated: (date now)
|
||||||
|
}
|
||||||
|
}
|
||||||
9
providers/aws/provisioning.yaml
Normal file
9
providers/aws/provisioning.yaml
Normal file
@ -0,0 +1,9 @@
|
|||||||
|
version: 1.0
|
||||||
|
info: AWS provisioning
|
||||||
|
site: https://docs.aws.amazon.com/cli/
|
||||||
|
tools:
|
||||||
|
aws:
|
||||||
|
version: 2.17.7
|
||||||
|
source: "https://awscli.amazonaws.com/awscli-exe-${OS}-${ORG_ARCH}.zip"
|
||||||
|
tags: https://github.com/aws/aws-cli/tags
|
||||||
|
site: https://docs.aws.amazon.com/cli/
|
||||||
94
providers/aws/templates/aws_servers.j2
Normal file
94
providers/aws/templates/aws_servers.j2
Normal file
@ -0,0 +1,94 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# provisioning {{provisioning_vers}} aws server creation: {{now}}
|
||||||
|
{% if use_debug %} set -x {% endif %}
|
||||||
|
aws_version=$(aws --version | cut -f1 -d" " | sed 's,aws-cli/,,g')
|
||||||
|
[ -z "$aws_version" ] && echo "Error❗: aws command as not found" && exit 1
|
||||||
|
if [ -z "$(aws configure get aws_access_key_id 2>/dev/null)" ] ; then
|
||||||
|
echo "Error❗ AWS credentials not found for command. Review $HOME/.aws/credentials and/or environment variables for settings"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
out_path={{runset.output_path}}
|
||||||
|
if [ -n "$out_path" ] ; then
|
||||||
|
out_path=${out_path//NOW/{{now}}}
|
||||||
|
[ ! -d "$out_path" ] && mkdir -p "$out_path"
|
||||||
|
else
|
||||||
|
out_path=/tmp
|
||||||
|
fi
|
||||||
|
{%- if server.hostname %}
|
||||||
|
instance_data=$(aws ec2 describe-instances --filter "Name=tag-value,Values={{server.hostname}}" "Name=instance-state-name,Values=running" \
|
||||||
|
--query "Reservations[*].Instances[].{\
|
||||||
|
__tags: Tags[?Key=='Name'].Value[],\
|
||||||
|
__id: InstanceId,\
|
||||||
|
__priv: NetworkInterfaces[*].PrivateIpAddresses[*].PrivateIpAddress,\
|
||||||
|
__pub: PublicIpAddress,\
|
||||||
|
__type: InstanceType,\
|
||||||
|
__status: State.Name\
|
||||||
|
}"\
|
||||||
|
--output yaml
|
||||||
|
)
|
||||||
|
instance_id=$(echo $instance_data | tr "__" "\n" | grep "^id: " | cut -f2 -d":" | sed "s/ //g")
|
||||||
|
if [ -n "$instance_id" ] ; then
|
||||||
|
instance_type=$(echo $instance_data | tr "__" "\n" | grep "^type: " | cut -f2 -d":" | sed "s/ //g")
|
||||||
|
status=$(echo $instance_data | tr "__" "\n" | grep "^status: " | cut -f2 -d":" | sed "s/ //g")
|
||||||
|
public_ip=$(echo $instance_data | tr "__" "\n" | grep "^pub: " | cut -f2 -d":" | sed "s/ //g")
|
||||||
|
echo -e "Server {{server.hostname}} already created \nid: $instance_id\ntype: $instance_type\nstate: $status\nip: $public_ip "
|
||||||
|
else
|
||||||
|
interface=$(aws ec2 describe-network-interfaces --query "NetworkInterfaces[][NetworkInterfaceId,PrivateIpAddress]" --output text | grep "{{server.network_private_ip}}" | awk '{print $1}')
|
||||||
|
if [ -n "$interface" ] ; then
|
||||||
|
echo "Try to delete interface $interface already using {{server.network_private_ip}} ..."
|
||||||
|
aws ec2 delete-network-interface --network-interface-id "$interface"
|
||||||
|
interface=$(aws ec2 describe-network-interfaces --query "NetworkInterfaces[][NetworkInterfaceId,PrivateIpAddress]" --output text | grep "{{server.network_private_ip}}" | awk '{print $1}')
|
||||||
|
fi
|
||||||
|
[ -n "$interface" ] && echo "interface $interface is already using {{server.network_private_ip}}" && exit 1
|
||||||
|
{% if use_time and use_time == 'true' %} time {%- endif -%}
|
||||||
|
aws ec2 run-instances \
|
||||||
|
{%- if provider and provider.main and provider.main.subnet %}
|
||||||
|
--subnet-id {{provider.main.subnet}} \
|
||||||
|
{%- endif -%}
|
||||||
|
{%- if provider and provider.main and provider.main.sg and provider.main.sg.id %}
|
||||||
|
--security-group-ids {{provider.main.sg.id}} \
|
||||||
|
{%- endif -%}
|
||||||
|
{%- if server.ssh_key_name %}
|
||||||
|
--key-name {{server.ssh_key_name}} \
|
||||||
|
{%- elif defaults.ssh_key_name and defaults.ssh_key_name != '' %}
|
||||||
|
--key-name {{defaults.ssh_key_name}} \
|
||||||
|
{%- endif -%}
|
||||||
|
{%- if server.plan %}
|
||||||
|
--instance-type {{server.plan}} \
|
||||||
|
{%- endif -%}
|
||||||
|
{%- if server.storage_os %}
|
||||||
|
--image-id {{server.storage_os}} \
|
||||||
|
{%- endif -%}
|
||||||
|
{%- if server.storages %}
|
||||||
|
--block-device-mappings '[
|
||||||
|
{%- for storage in server.storages %}{%- if loop.index0 == 0 -%}{%- continue %}{%- endif -%}{%- if loop.index0 > 1 -%},{%- endif -%}
|
||||||
|
{"DeviceName":"/dev/{{storage.voldevice}}","Ebs":{"VolumeSize":
|
||||||
|
{%- if storage.size > 0 -%}{{storage.size}}{%- elif storage.parts and storage.parts[0] -%}{{storage.parts[0].size}}{%- endif -%},
|
||||||
|
{%- if storage.encrypted -%}"Encrypted":{{storage.encrypted}},{%- endif -%}
|
||||||
|
{%- if storage.kms_id and storage.kms_id != "" -%}"KmsKeyId":{{storage.kms_id}},{%- endif -%}
|
||||||
|
"VolumeType":"{{storage.voltype}}","DeleteOnTermination":{{storage.deletetermination}}},"NoDevice":""}
|
||||||
|
{%- endfor -%}]' \
|
||||||
|
{%- endif -%}
|
||||||
|
{%- if server.user_data %}
|
||||||
|
--user-data {{server.user_data}} \
|
||||||
|
{%- endif -%}
|
||||||
|
{%- if server.disable_stop %}
|
||||||
|
--disable-api-stop \
|
||||||
|
{%- endif -%}
|
||||||
|
{%- if server.zone %}
|
||||||
|
--region {{server.zone}} \
|
||||||
|
{%- endif %}
|
||||||
|
--tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value={{server.hostname}}},{Key=hostname,Value={{server.hostname}}}]' 'ResourceType=volume,Tags=[{Key=Name,Value={{server.hostname}}}]' \
|
||||||
|
--output yaml > $out_path/{{server.hostname}}.yaml
|
||||||
|
instance_id=$(grep "InstanceId:" $out_path/{{server.hostname}}.yaml | cut -f2 -d':')
|
||||||
|
[ -z "$instance_id" ] && echo "❗ Error: no instance id found for {{server.hostname}} " && exit 1
|
||||||
|
{%- if provider and provider.priv and provider.priv.subnet and server.network_private_ip != '' %}
|
||||||
|
while [ "$(aws ec2 describe-instance-status --instance-id $instance_id --query "InstanceStatuses[].InstanceState.Name" --out text)" != "running" ] ; do sleep 10; echo "wait {{server.hostname}} running ..."; done
|
||||||
|
interface=$(aws ec2 create-network-interface --subnet-id "{{provider.priv.subnet}}" --description "private_ip {{server.hostname}}" \
|
||||||
|
--private-ip-address "{{server.network_private_ip}}" --query "NetworkInterface.NetworkInterfaceId" \
|
||||||
|
{% if provider and provider.priv and provider.priv.sg and provider.priv.sg.id %}--groups {{provider.priv.sg.id}} {%- endif -%} \
|
||||||
|
--output text)
|
||||||
|
[ -n "$interface" ] && [ -n "$instance_id" ] && aws ec2 attach-network-interface --network-interface-id $interface --instance-id $instance_id --device-index 1
|
||||||
|
{% endif %}
|
||||||
|
fi
|
||||||
|
{%- endif -%}
|
||||||
20
providers/aws/templates/aws_sg.j2
Normal file
20
providers/aws/templates/aws_sg.j2
Normal file
@ -0,0 +1,20 @@
|
|||||||
|
{% for perm in curr_perms -%}
|
||||||
|
{% set_global ranges = "" -%}
|
||||||
|
{% if perm.IpRanges -%}
|
||||||
|
{% for rng in perm.IpRanges -%}
|
||||||
|
{% if ranges != "" -%} {% set_global ranges = ranges ~ "," -%} {% endif -%}
|
||||||
|
{% set_global ranges = ranges ~ "{CidrIp=" ~ rng.CidrIp ~ "}" -%}
|
||||||
|
{% endfor -%}
|
||||||
|
{% endif -%}
|
||||||
|
aws ec2 revoke-security-group-ingress \
|
||||||
|
--group-id "{{sg_id}}" \
|
||||||
|
--ip-permissions "IpProtocol={{perm.IpProtocol}},FromPort={{perm.FromPort}},ToPort={{perm.ToPort}},IpRanges=[{{ranges}}]" \
|
||||||
|
--out json
|
||||||
|
{% endfor -%}
|
||||||
|
{% for perm in perms -%}
|
||||||
|
aws ec2 authorize-security-group-ingress \
|
||||||
|
--group-id "{{sg_id}}" \
|
||||||
|
--tag-specifications 'ResourceType=security-group-rule,Tags=[{Key=Name,Value={{perm.name}}}]' \
|
||||||
|
--ip-permissions "IpProtocol={{perm.protocol}},FromPort={{perm.fromPort}},ToPort={{perm.toPort}},IpRanges={{perm.ranges}}" \
|
||||||
|
--out json
|
||||||
|
{% endfor -%}
|
||||||
4
providers/aws/versions
Normal file
4
providers/aws/versions
Normal file
@ -0,0 +1,4 @@
|
|||||||
|
AWS_AWS_VERSION="2.17.7"
|
||||||
|
AWS_AWS_SOURCE="https://awscli.amazonaws.com/awscli-exe-${OS}-${ORG_ARCH}.zip"
|
||||||
|
AWS_AWS_TAGS="https://github.com/aws/aws-cli/tags"
|
||||||
|
AWS_AWS_SITE="https://docs.aws.amazon.com/cli/"
|
||||||
12
providers/aws/versions.yaml
Normal file
12
providers/aws/versions.yaml
Normal file
@ -0,0 +1,12 @@
|
|||||||
|
aws:
|
||||||
|
version: 2.17.7
|
||||||
|
fixed: false
|
||||||
|
source: https://github.com/aws/aws-cli/releases
|
||||||
|
tags: https://github.com/aws/aws-cli/tags
|
||||||
|
site: https://docs.aws.amazon.com/cli/
|
||||||
|
detector:
|
||||||
|
method: command
|
||||||
|
command: aws --version
|
||||||
|
pattern: aws-cli/(\d+\.\d+\.\d+)
|
||||||
|
capture: capture0
|
||||||
|
comparison: semantic
|
||||||
51
providers/local/README.md
Normal file
51
providers/local/README.md
Normal file
@ -0,0 +1,51 @@
|
|||||||
|
# Local Declarative Provision via scripts & templates
|
||||||
|
|
||||||
|
Part of [Cloud Native zone Provision](/CloudNativeZone/cnz-provision)
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
Install [Python](https://es.wikipedia.org/wiki/Python)
|
||||||
|
|
||||||
|
For [Ubuntu](https://ubuntu.com/)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo apt install wget build-essential libncursesw5-dev libssl-dev \
|
||||||
|
libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev libffi-dev zlib1g-dev
|
||||||
|
|
||||||
|
sudo add-apt-repository ppa:deadsnakes/ppa
|
||||||
|
|
||||||
|
sudo apt install python3.13
|
||||||
|
sudo apt-get -y install python3-pip
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
Install [Jinja2 engine](https://jinja.palletsprojects.com/en/3.1.x/)
|
||||||
|
|
||||||
|
```python
|
||||||
|
pip3 install Jinja2
|
||||||
|
```
|
||||||
|
|
||||||
|
Install [Python YAML](https://pypi.org/project/PyYAML/)
|
||||||
|
|
||||||
|
```python
|
||||||
|
pip3 install PyYAML
|
||||||
|
```
|
||||||
|
|
||||||
|
[Install YQ](https://github.com/mikefarah/yq/#install)
|
||||||
|
|
||||||
|
[Install JQ](https://jqlang.github.io/jq/download/)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
apt install jq
|
||||||
|
```
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
[YAML org](https://yaml.org/)
|
||||||
|
|
||||||
|
[YQ](https://github.com/mikefarah/yq)
|
||||||
|
|
||||||
|
[YQ Documentation](https://mikefarah.gitbook.io/yq/)
|
||||||
|
|
||||||
|
[Jinja2 Tempalte engine](https://jinja.palletsprojects.com/en/3.1.x/)
|
||||||
|
|
||||||
102
providers/local/bin/install.sh
Executable file
102
providers/local/bin/install.sh
Executable file
@ -0,0 +1,102 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Info: Script to install provider
|
||||||
|
# Author: JesusPerezLorenzo
|
||||||
|
# Release: 1.0
|
||||||
|
# Date: 15-04-2024
|
||||||
|
|
||||||
|
[ "$DEBUG" == "-x" ] && set -x
|
||||||
|
|
||||||
|
USAGE="install [ tool-name: upctl, etc | all | info] [--update]
|
||||||
|
As alternative use environment var TOOL_TO_INSTALL with a list-of-tools (separeted with spaces)
|
||||||
|
Versions are set in ./versions file
|
||||||
|
|
||||||
|
This can be called by directly with an argumet or from an other script
|
||||||
|
"
|
||||||
|
|
||||||
|
ORG=$(pwd)
|
||||||
|
function _info_tools {
|
||||||
|
local match=$1
|
||||||
|
local info_keys
|
||||||
|
info_keys="info version site"
|
||||||
|
|
||||||
|
if [ -z "$match" ] || [ "$match" == "all" ] || [ "$match" == "-" ]; then
|
||||||
|
match="all"
|
||||||
|
fi
|
||||||
|
echo "$PROVIDER_TITLE"
|
||||||
|
[ ! -r "$PROVIDERS_PATH/$PROVIDER_NAME/provisioning.yaml" ] && return
|
||||||
|
echo "-------------------------------------------------------"
|
||||||
|
case "$match" in
|
||||||
|
"i" | "?" | "info")
|
||||||
|
for key in $info_keys
|
||||||
|
do
|
||||||
|
echo -n "$key:"
|
||||||
|
[ "$key" != "version" ] && echo -ne "\t"
|
||||||
|
echo " $(grep "^$key:" "$PROVIDERS_PATH/$PROVIDER_NAME/provisioning.yaml" | sed "s/$key: //g")"
|
||||||
|
done
|
||||||
|
;;
|
||||||
|
"all")
|
||||||
|
cat "$PROVIDERS_PATH/$PROVIDER_NAME/provisioning.yaml"
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
echo -e "$match:\t $(grep "^$match:" "$PROVIDERS_PATH/$PROVIDER_NAME/provisioning.yaml" | sed "s/$match: //g")"
|
||||||
|
esac
|
||||||
|
echo "________________________________________________________"
|
||||||
|
}
|
||||||
|
function _install_tools {
|
||||||
|
local match=$1
|
||||||
|
shift
|
||||||
|
local options
|
||||||
|
options="$*"
|
||||||
|
local has_tool
|
||||||
|
local tool_version
|
||||||
|
|
||||||
|
OS="$(uname | tr '[:upper:]' '[:lower:]')"
|
||||||
|
ORG_OS=$(uname)
|
||||||
|
ARCH="$(uname -m | sed -e 's/x86_64/amd64/' -e 's/\(arm\)\(64\)\?.*/\1\2/' -e 's/aarch64$/arm64/')"
|
||||||
|
ORG_ARCH="$(uname -m)"
|
||||||
|
|
||||||
|
}
|
||||||
|
function _on_tools {
|
||||||
|
local tools_list=$1
|
||||||
|
[ -z "$tools_list" ] || [[ "$tools_list" == -* ]] && tools_list=${TOOL_TO_INSTALL:-all}
|
||||||
|
case $tools_list in
|
||||||
|
"all")
|
||||||
|
_install_tools "all" "$@"
|
||||||
|
;;
|
||||||
|
"info" | "i" | "?")
|
||||||
|
shift
|
||||||
|
_info_tools "$@"
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
for tool in $tools_list
|
||||||
|
do
|
||||||
|
[[ "$tool" == -* ]] && continue
|
||||||
|
_install_tools "$tool" "${*//$tool/}"
|
||||||
|
done
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
|
||||||
|
set -o allexport
|
||||||
|
## shellcheck disable=SC1090
|
||||||
|
[ -n "$PROVISIONING_ENV" ] && [ -r "$PROVISIONING_ENV" ] && source "$PROVISIONING_ENV"
|
||||||
|
[ -r "../env-provisioning" ] && source ../env-provisioning
|
||||||
|
[ -r "env-provisioning" ] && source ./env-provisioning
|
||||||
|
#[ -r ".env" ] && source .env set
|
||||||
|
set +o allexport
|
||||||
|
|
||||||
|
export PROVISIONING=${PROVISIONING:-/usr/local/provisioning}
|
||||||
|
|
||||||
|
PROVIDERS_PATH=${PROVIDERS_PATH:-"$PROVISIONING/providers"}
|
||||||
|
|
||||||
|
PROVIDER_NAME="local"
|
||||||
|
PROVIDER_TITLE="Local"
|
||||||
|
|
||||||
|
if [ -r "$(dirname "$0")/../versions" ] ; then
|
||||||
|
. "$(dirname "$0")"/../versions
|
||||||
|
elif [ -r "$(dirname "$0")/versions" ] ; then
|
||||||
|
. "$(dirname "$0")"/versions
|
||||||
|
fi
|
||||||
|
[ "$1" == "-h" ] && echo "$USAGE" && shift
|
||||||
|
[ "$1" == "check" ] && CHECK_ONLY="yes" && shift
|
||||||
|
[ -n "$1" ] && cd /tmp && _on_tools "$@"
|
||||||
|
[ -z "$1" ] && _on_tools "$@"
|
||||||
0
providers/local/generate/local_defaults.k.j2
Normal file
0
providers/local/generate/local_defaults.k.j2
Normal file
0
providers/local/generate/servers.k.j2
Normal file
0
providers/local/generate/servers.k.j2
Normal file
8
providers/local/kcl/kcl.mod
Normal file
8
providers/local/kcl/kcl.mod
Normal file
@ -0,0 +1,8 @@
|
|||||||
|
[package]
|
||||||
|
name = "local_prov"
|
||||||
|
edition = "0.0.1"
|
||||||
|
version = "0.0.1"
|
||||||
|
|
||||||
|
[dependencies]
|
||||||
|
provisioning = { path = "../../../../kcl", version = "0.0.1" }
|
||||||
|
providers = { path = "../..", version = "0.0.1" }
|
||||||
10
providers/local/kcl/kcl.mod.lock
Normal file
10
providers/local/kcl/kcl.mod.lock
Normal file
@ -0,0 +1,10 @@
|
|||||||
|
[dependencies]
|
||||||
|
[dependencies.providers]
|
||||||
|
name = "providers"
|
||||||
|
full_name = "vPkg_415e4ae0-6183-4f8e-8123-e69f70e9de7e_0.0.1"
|
||||||
|
version = "0.0.1"
|
||||||
|
[dependencies.provisioning]
|
||||||
|
name = "provisioning"
|
||||||
|
full_name = "provisioning_0.0.1"
|
||||||
|
version = "0.0.1"
|
||||||
|
sum = "BiCbdv6gSdLNik8n6Mc8mv5ATy0Ea2sk1Z6zRMcNNV0="
|
||||||
5
providers/local/nulib/local/env.nu
Normal file
5
providers/local/nulib/local/env.nu
Normal file
@ -0,0 +1,5 @@
|
|||||||
|
export-env {
|
||||||
|
$env.LOCAL_API_URL = ($env | get -o LOCAL_API_URL | default "")
|
||||||
|
$env.LOCAL_AUTH = ($env | get -o LOCAL_AUTH | default "")
|
||||||
|
$env.LOCAL_INTERFACE = ($env | get -o LOCAL_INTERFACE | default "CLI") # API or CLI
|
||||||
|
}
|
||||||
5
providers/local/nulib/local/env.nu-e
Normal file
5
providers/local/nulib/local/env.nu-e
Normal file
@ -0,0 +1,5 @@
|
|||||||
|
export-env {
|
||||||
|
$env.LOCAL_API_URL = ($env | get -o LOCAL_API_URL | default "")
|
||||||
|
$env.LOCAL_AUTH = ($env | get -o LOCAL_AUTH | default "")
|
||||||
|
$env.LOCAL_INTERFACE = ($env | get -o LOCAL_INTERFACE | default "CLI") # API or CLI
|
||||||
|
}
|
||||||
4
providers/local/nulib/local/mod.nu
Normal file
4
providers/local/nulib/local/mod.nu
Normal file
@ -0,0 +1,4 @@
|
|||||||
|
use env.nu
|
||||||
|
export use servers.nu *
|
||||||
|
export use usage.nu *
|
||||||
|
export use utils.nu *
|
||||||
4
providers/local/nulib/local/mod.nu-e
Normal file
4
providers/local/nulib/local/mod.nu-e
Normal file
@ -0,0 +1,4 @@
|
|||||||
|
use env.nu
|
||||||
|
export use servers.nu *
|
||||||
|
export use usage.nu *
|
||||||
|
export use utils.nu *
|
||||||
576
providers/local/nulib/local/servers.nu
Normal file
576
providers/local/nulib/local/servers.nu
Normal file
@ -0,0 +1,576 @@
|
|||||||
|
#!/usr/bin/env nu
|
||||||
|
use std
|
||||||
|
use ../../../../../core/nulib/lib_provisioning/config/accessor.nu *
|
||||||
|
|
||||||
|
export def local_query_servers [
|
||||||
|
find: string
|
||||||
|
cols: string
|
||||||
|
] {
|
||||||
|
# TODO FIX
|
||||||
|
let res = (^upctl server list -o json err> (if $nu.os-info.name == "windows" { "NUL" } else { "/dev/null" }) | complete)
|
||||||
|
if $res.exit_code == 0 {
|
||||||
|
$res.stdout | from json | get servers
|
||||||
|
} else {
|
||||||
|
if (is-debug-enabled) {
|
||||||
|
(throw-error "🛑 local server list " $"($res.exit_code) ($res.stdout)" "local query server" --span (metadata $res).span)
|
||||||
|
} else {
|
||||||
|
print $"🛑 Error local server list: ($res.exit_code) ($res.stdout | ^grep 'error')"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def local_server_info [
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
] {
|
||||||
|
let hostname = $server.hostname
|
||||||
|
# TODO FIX
|
||||||
|
let res = (^upctl server show $hostname -o json err> (if $nu.os-info.name == "windows" { "NUL" } else { "/dev/null" }) | complete)
|
||||||
|
if $res.exit_code == 0 {
|
||||||
|
$res.stdout | from json
|
||||||
|
} else if $check {
|
||||||
|
{}
|
||||||
|
} else {
|
||||||
|
if (is-debug-enabled) {
|
||||||
|
(throw-error "🛑 local server show" $"($res.exit_code) ($res.stdout)" $"local server info ($hostname)" --span (metadata $res).span)
|
||||||
|
} else {
|
||||||
|
print $"🛑 local server show ($hostname):($res.stdout | ^grep 'error')"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def local_on_prov_server [
|
||||||
|
server?: record
|
||||||
|
infra?: string
|
||||||
|
] {
|
||||||
|
let info = if ( $env.CURRENT_FILE? | into string ) != "" { (^grep "^# Info:" $env.CURRENT_FILE ) | str replace "# Info: " "" } else { "" }
|
||||||
|
print $env.CURRENT_FILE
|
||||||
|
$" From LOCAL ($info) "
|
||||||
|
}
|
||||||
|
# infrastructure and services
|
||||||
|
export def local [
|
||||||
|
args: list<string> # Args for create command
|
||||||
|
--server(-s): record
|
||||||
|
#hostname?: string # Server hostname in settings
|
||||||
|
--serverpos (-p): int # Server position in settings
|
||||||
|
--check (-c) # Only check mode no servers will be created
|
||||||
|
--wait (-w) # Wait servers to be created
|
||||||
|
--infra (-i): string # Infra path
|
||||||
|
--settings (-s): string # Settings path
|
||||||
|
--outfile (-o): string # Output file
|
||||||
|
--debug (-x) # Use Debug mode
|
||||||
|
] {
|
||||||
|
if $debug { set-debug-enabled true }
|
||||||
|
let target = ($args | get -o 0 | default "")
|
||||||
|
let task = ($args | get -o 1 | default "")
|
||||||
|
let cmd_args = if ($args | length) > 1 { ($args | drop nth ..1) } else { [] }
|
||||||
|
match ($task) {
|
||||||
|
"help" | "h" | "" => {
|
||||||
|
print "TODO local help"
|
||||||
|
if not (is-debug-enabled) { end_run "" }
|
||||||
|
exit
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
if ($args | find "help" | length) > 0 {
|
||||||
|
match $task {
|
||||||
|
"server" => {
|
||||||
|
print "SERVER "
|
||||||
|
local_server ($args | drop nth ..0)
|
||||||
|
#local_server ($args | drop nth ..1) --server $server
|
||||||
|
},
|
||||||
|
"inventory" => {
|
||||||
|
local_server ($args | drop nth ..0)
|
||||||
|
},
|
||||||
|
"ssh" => {
|
||||||
|
local_server ($args | drop nth ..0)
|
||||||
|
},
|
||||||
|
"delete" => {
|
||||||
|
local_server ($args | drop nth ..0)
|
||||||
|
# ($args | drop nth ..1) --server $server
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
option_undefined "local" ""
|
||||||
|
print "TODO local help"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if not (is-debug-enabled) { end_run "" }
|
||||||
|
exit
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
#use utils/settings.nu [ load_settings ]
|
||||||
|
let curr_settings = if $infra != null {
|
||||||
|
if $settings != null {
|
||||||
|
(load_settings --infra $infra --settings $settings)
|
||||||
|
} else {
|
||||||
|
(load_settings --infra $infra)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
if $settings != null {
|
||||||
|
(load_settings --settings $settings)
|
||||||
|
} else {
|
||||||
|
(load_settings)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
match ($task) {
|
||||||
|
"get_ip" => {
|
||||||
|
local_get_ip $curr_settings $server ($cmd_args | get -o 0 | default "")
|
||||||
|
},
|
||||||
|
"server" => {
|
||||||
|
print (
|
||||||
|
local_server $cmd_args --server $server --settings $curr_settings --error_exit
|
||||||
|
)
|
||||||
|
},
|
||||||
|
"inventory" => {
|
||||||
|
},
|
||||||
|
"ssh" => {
|
||||||
|
},
|
||||||
|
"delete" => {
|
||||||
|
# ($args | drop nth ..1) --server $server
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
option_undefined "local" ""
|
||||||
|
if not (is-debug-enabled) { end_run "" }
|
||||||
|
exit
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def local_get_ip [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
ip_type: string
|
||||||
|
] {
|
||||||
|
match $ip_type {
|
||||||
|
"private" | "prv" | "priv" => {
|
||||||
|
return $"($server.network_private_ip)"
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
let ip = ($server.network_public_ip | default "")
|
||||||
|
# TODO FIX add NOT FOUND ERRORS
|
||||||
|
return $ip
|
||||||
|
#let result = (^upctl "server" "show" $server.hostname "-o" "json" | complete)
|
||||||
|
#if $result.exit_code == 0 {
|
||||||
|
# let data = ($result.stdout | from json)
|
||||||
|
# #let id = ($data.id? | default "")
|
||||||
|
# let ip_addresses = ($data.networking?.interfaces? | where {|it| ($it.type | str contains "public") }).ip_addresses?
|
||||||
|
# return $"(($ip_addresses | get -o 0).address? | get -o 0 | default '')"
|
||||||
|
#} else { "" }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
# To create infrastructure and services
|
||||||
|
export def local_server [
|
||||||
|
args: list<string> # Args for create command
|
||||||
|
--server(-s): record
|
||||||
|
--error_exit
|
||||||
|
--status
|
||||||
|
#hostname?: string # Server hostname in settings
|
||||||
|
--serverpos (-p): int # Server position in settings
|
||||||
|
--check (-c) # Only check mode no servers will be created
|
||||||
|
--wait (-w) # Wait servers to be created
|
||||||
|
--infra (-i): string # Infra path
|
||||||
|
--settings (-s): record # Settings path
|
||||||
|
--outfile (-o): string # Output file
|
||||||
|
--debug (-x) # Use Debug mode
|
||||||
|
] {
|
||||||
|
let task = ($args | get -o 0)
|
||||||
|
let target = if ($args | length) > 1 { ($args | get -o 1) } else { "" }
|
||||||
|
let cmd_args = if ($args | length) > 1 { ($args | drop nth ..1) } else { [] }
|
||||||
|
match ($task) {
|
||||||
|
"help" | "h" | "" => {
|
||||||
|
print "TODO local server help"
|
||||||
|
if not (is-debug-enabled) { end_run "" }
|
||||||
|
exit
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
if $target == "" or ($args | find "help" | length) > 0 {
|
||||||
|
match $task {
|
||||||
|
"server" => {
|
||||||
|
local_server $cmd_args
|
||||||
|
},
|
||||||
|
"status" => {
|
||||||
|
print $server
|
||||||
|
print $error_exit
|
||||||
|
}
|
||||||
|
"inventory" => {
|
||||||
|
print "TODO local server inventory help"
|
||||||
|
},
|
||||||
|
"ssh" => {
|
||||||
|
print "TODO local server ssh help"
|
||||||
|
},
|
||||||
|
"delete" => {
|
||||||
|
# ($args | drop nth ..1) --server $server
|
||||||
|
#local_delete_server $cmd_args true
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
option_undefined "local" "server"
|
||||||
|
print "TODO local server help"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if not (is-debug-enabled) { end_run "" }
|
||||||
|
exit
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let server_target = if $server != null {
|
||||||
|
$server
|
||||||
|
} else if $settings != null {
|
||||||
|
($settings.data.servers | where {|it| $it.hostname == $target } | get -o 0)
|
||||||
|
} else {
|
||||||
|
null
|
||||||
|
}
|
||||||
|
if $server_target == null {
|
||||||
|
if $error_exit {
|
||||||
|
let text = $"($args | str join ' ')"
|
||||||
|
(throw-error "🛑 local server" $text "" --span (metadata $server_target).span)
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
if $status or $task == "status" {
|
||||||
|
print "local server status "
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
match $task {
|
||||||
|
"get_ip" => {
|
||||||
|
local_get_ip $settings $server_target ($cmd_args | get -o 0 | default "")
|
||||||
|
},
|
||||||
|
"stop" => {
|
||||||
|
print "TODO local server stop"
|
||||||
|
},
|
||||||
|
"start" => {
|
||||||
|
print "TODO local server start"
|
||||||
|
},
|
||||||
|
"restart" => {
|
||||||
|
print "TODO local server restart"
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
option_undefined "local" "server"
|
||||||
|
if not (is-debug-enabled) { end_run "" }
|
||||||
|
exit
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def local_create_private_network [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
] {
|
||||||
|
if $server == null {
|
||||||
|
print $"❗ No server found in settings "
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
# new_upctl network list -o json |
|
||||||
|
# let net_id = ($data.networks | get -o 0 ).uuid)
|
||||||
|
let zone = ( $server.zone? | default "")
|
||||||
|
if $zone == "" {
|
||||||
|
print $"($server.hostname) No zone found to CREATE network_privat_id"
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
let network_private_name = ($server.network_private_name? | default "")
|
||||||
|
if $network_private_name == "" {
|
||||||
|
print $"($server.hostname) No network_private_name found to CREATE network_privat_id"
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
let priv_cidr_block = ($server.priv_cidr_block | default "")
|
||||||
|
if $network_private_name == "" {
|
||||||
|
print $"($server.hostname) No priv_cidr_block found to CREATE network_privat_id"
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
# EXAMPLE_BASH private_net_id=$(upctl network list -o yaml | $YQ '.networks[] | select(.ip_networks.ip_network[].address == "'"$priv_cidr_block"'") | .uuid' 2>/dev/null | sed 's,",,g')
|
||||||
|
let result = (^upctl "network" "list" "-o" "json" | complete)
|
||||||
|
let private_net_id = if $result.exit_code == 0 {
|
||||||
|
let data = ($result.stdout | from json )
|
||||||
|
($data.networks? | find $priv_cidr_block | get -o 0 | get -o uuid | default "")
|
||||||
|
} else {
|
||||||
|
""
|
||||||
|
}
|
||||||
|
if $check and $private_net_id == "" {
|
||||||
|
print $"❗private_network will be register in a real creation request not in check state"
|
||||||
|
return ""
|
||||||
|
} else if $private_net_id == "" {
|
||||||
|
let result = (^upctl network create --name $network_private_name --zone $zone --ip-network $"address='($priv_cidr_block)',dhcp=true" -o json ) | complete
|
||||||
|
let new_net_id = if $result.exit_code == 0 {
|
||||||
|
(($result.stdout | from json | find $priv_cidr_block | get -o 0).uuid? | default "")
|
||||||
|
} else { "" }
|
||||||
|
if $new_net_id == "" {
|
||||||
|
(throw-error $"🛑 no private network ($network_private_name) found"
|
||||||
|
$"for server ($server.hostname) ip ($server.network_private_ip)"
|
||||||
|
$"local_check_requirements" --span (metadata $new_net_id.span))
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
# Save changes ...
|
||||||
|
#use utils/settings.nu [ save_servers_settings save_settings_file ]
|
||||||
|
let match_text = " network_private_id = "
|
||||||
|
let defs_provider_path = $"($settings.data.server_path | get -o 0 | path dirname)/local_defaults"
|
||||||
|
save_servers_setings $settings $match_text $new_net_id
|
||||||
|
save_settings_file $settings $"($settings.src_path)/($settings.src)" $match_text $new_net_id
|
||||||
|
save_settings_file $settings $"($defs_provider_path)" $match_text $new_net_id
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
export def local_check_server_requirements [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
] {
|
||||||
|
if $server.provider != "local" { return false }
|
||||||
|
print ($"✅ (_ansi blue_bold)($server.hostname)(_ansi reset) with provider " +
|
||||||
|
$"(_ansi green_bold)($server.provider)(_ansi reset) ($server.zone) does not require creation !" )
|
||||||
|
true
|
||||||
|
}
|
||||||
|
|
||||||
|
export def local_make_settings [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
|
||||||
|
# # _delete_settings
|
||||||
|
let out_settings_path = $"($settings.infra_fullpath)/($server.provider)_settings.yaml"
|
||||||
|
let data = if ($out_settings_path | path exists ) {
|
||||||
|
(open $out_settings_path | from yaml)
|
||||||
|
} else {
|
||||||
|
null
|
||||||
|
}
|
||||||
|
let task = if $data != null { "update" } else { "create" }
|
||||||
|
let uuid = (^upctl server show $server.hostname "-o" "json" | from json).uuid? | default ""
|
||||||
|
# echo "settings:" > "$out_settings"
|
||||||
|
|
||||||
|
# for server in $(_settings_hosts)
|
||||||
|
# do
|
||||||
|
if $uuid == "" {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
let ip_pub = (local_get_ip $settings $server "public")
|
||||||
|
let ip_priv = (local_get_ip $settings $server "private")
|
||||||
|
|
||||||
|
let server_settings = {
|
||||||
|
name: $server.hostname,
|
||||||
|
id: $uuid,
|
||||||
|
private_net: {
|
||||||
|
id: $server.network_private_id
|
||||||
|
name: $server.network_private_name
|
||||||
|
},
|
||||||
|
zone: $server.zone,
|
||||||
|
datetime: $env.NOW,
|
||||||
|
ip_addresses: {
|
||||||
|
pub: $ip_pub, priv: $ip_priv
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let new_data = if $data != null and $data.servers? != null {
|
||||||
|
( $data.servers | each { |srv|
|
||||||
|
where {|it| $it.name != $server.hostname }
|
||||||
|
}) | append $server_settings
|
||||||
|
} else {
|
||||||
|
## create data record
|
||||||
|
{
|
||||||
|
servers: [ $server_settings ]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
$new_data | to yaml | save --force $out_settings_path
|
||||||
|
print $"✅ local settings ($task) -> ($out_settings_path)"
|
||||||
|
true
|
||||||
|
}
|
||||||
|
export def local_delete_settings [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
}
|
||||||
|
export def local_post_create_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
] {
|
||||||
|
if $server != null {
|
||||||
|
return (local_storage_fix_size $settings $server 0)
|
||||||
|
}
|
||||||
|
true
|
||||||
|
}
|
||||||
|
export def local_modify_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
new_values: list
|
||||||
|
error_exit: bool
|
||||||
|
] {
|
||||||
|
# TODO LOCAL
|
||||||
|
return
|
||||||
|
let res = (^upctl server $server.hostname modify ...($new_values) | complete)
|
||||||
|
if $res.exit_code != 0 {
|
||||||
|
print $"❗ Server ($server.hostname) modify ($new_values | str join ' ') errors ($res.stdout ) "
|
||||||
|
if $error_exit {
|
||||||
|
exit 1
|
||||||
|
} else {
|
||||||
|
return "error"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def local_storage_fix_size [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
storage_pos: int
|
||||||
|
] {
|
||||||
|
# TODO LOCAL
|
||||||
|
return
|
||||||
|
let total_size = ($server | get -o storages | get $storage_pos | get -o total | default 0)
|
||||||
|
if $total_size == 0 { return 0 }
|
||||||
|
let storage = (^upctl server show $server.hostname "-o" "json" | from json).storage_devices | (get -o $storage_pos)
|
||||||
|
if $storage == null { return 0 }
|
||||||
|
let curr_size = $storage.storage_size? | default 0
|
||||||
|
if $curr_size == 0 { return 0 }
|
||||||
|
if $curr_size != $total_size {
|
||||||
|
print (
|
||||||
|
$"Stop (_ansi blue_bold)($server.hostname)(_ansi reset) for storage (_ansi yellow_bold)($storage.storage)(_ansi reset)" +
|
||||||
|
$" from (_ansi purple_bold)($curr_size)(_ansi reset) to (_ansi green_bold)($total_size)(_ansi reset) ... "
|
||||||
|
)
|
||||||
|
if (local_change_server_state $settings $server "stop" "") == false {
|
||||||
|
print $"❗ Stop ($server.hostname) errors "
|
||||||
|
return "error"
|
||||||
|
}
|
||||||
|
let res = (^upctl storage modify --size $total_size $storage.storage | complete)
|
||||||
|
if $res.exit_code != 0 {
|
||||||
|
print $"❗ Storage modify errors ($res.stdout ) "
|
||||||
|
return "error"
|
||||||
|
}
|
||||||
|
let new_storage = (^upctl server show $server.hostname "-o" "json" | from json).storage_devices | (get -o $storage_pos)
|
||||||
|
let new_curr_size = $new_storage.storage_size? | default 0
|
||||||
|
print $"Start (_ansi blue_bold)($server.hostname)(_ansi reset) with new size (_ansi green_bold)($new_curr_size)(_ansi reset) ... "
|
||||||
|
if (local_change_server_state $settings $server "start" "") == false {
|
||||||
|
print $"❗ Errors to start ($server.hostname): ($res.stdout ) "
|
||||||
|
return "error"
|
||||||
|
}
|
||||||
|
return "storage"
|
||||||
|
}
|
||||||
|
""
|
||||||
|
}
|
||||||
|
export def local_status_server [
|
||||||
|
hostname: string
|
||||||
|
] {
|
||||||
|
let res = (^upctl server show $hostname "-o" "json" | complete)
|
||||||
|
if $res.exit_code != 0 {
|
||||||
|
print $"❗ status ($hostname) errors ($res.stdout ) "
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
return (($res.stdout | from json).state | default "")
|
||||||
|
}
|
||||||
|
export def local_server_exists [
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
] {
|
||||||
|
let res = (^upctl server show $server.hostname "-o" "json" err> (if $nu.os-info.name == "windows" { "NUL" } else { "/dev/null" }) | complete)
|
||||||
|
if $res.exit_code != 0 {
|
||||||
|
if $error_exit {
|
||||||
|
print $"❗ status ($server.hostname) errors ($res.stdout ) "
|
||||||
|
exit 1
|
||||||
|
} else {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
true
|
||||||
|
}
|
||||||
|
export def local_server_state [
|
||||||
|
server: record
|
||||||
|
new_state: string
|
||||||
|
error_exit: bool
|
||||||
|
wait: bool
|
||||||
|
settings: record
|
||||||
|
] {
|
||||||
|
}
|
||||||
|
export def local_server_is_running [
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
] {
|
||||||
|
true
|
||||||
|
#TODO FIX
|
||||||
|
# let res = (^upctl server show $server.hostname "-o" "json" err> (if $nu.os-info.name == "windows" { "NUL" } else { "/dev/null" }) | complete)
|
||||||
|
# if $res.exit_code != 0 {
|
||||||
|
# print $"❗ status ($server.hostname) errors ($res.stdout ) "
|
||||||
|
# if $error_exit {
|
||||||
|
# exit 1
|
||||||
|
# } else {
|
||||||
|
# return false
|
||||||
|
# }
|
||||||
|
# }
|
||||||
|
# (($res.stdout | from json).state? | str contains "started" | default false)
|
||||||
|
}
|
||||||
|
export def local_change_server_state [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
new_state: string
|
||||||
|
ops: string
|
||||||
|
] {
|
||||||
|
let state = (local_status_server $server.hostname)
|
||||||
|
if $state == "" { return false }
|
||||||
|
if ($state | str contains $new_state) { return true }
|
||||||
|
print $"Checking (_ansi blue_bold)($server.hostname)(_ansi reset) state (_ansi yellow_bold)($new_state)(_ansi reset) ..."
|
||||||
|
let val_timeout = if $server.running_timeout? != null { $server.running_timeout } else { 60 }
|
||||||
|
let wait = if $server.running_wait? != null { $server.running_wait } else { 10 }
|
||||||
|
let wait_duration = ($"($wait)sec"| into duration)
|
||||||
|
mut num = 0
|
||||||
|
while true {
|
||||||
|
let status = (local_status_server $server.hostname)
|
||||||
|
if $status == "" {
|
||||||
|
return false
|
||||||
|
} else if ($status | str contains "maintenance") == false {
|
||||||
|
print " "
|
||||||
|
break
|
||||||
|
} else if $val_timeout > 0 and $num > $val_timeout {
|
||||||
|
print $"\n🛑 (_ansi red)Timeout(_ansi reset) ($val_timeout) (_ansi blue)($server.hostname)(_ansi reset) (_ansi blue_bold)($new_state)(_ansi reset) (_ansi red_bold)failed(_ansi reset) "
|
||||||
|
return false
|
||||||
|
} else {
|
||||||
|
$num = $num + $wait
|
||||||
|
if (is-debug-enabled) {
|
||||||
|
print -n $"(_ansi blue_bold) 🌥 (_ansi reset)(_ansi green)($server.hostname)(_ansi reset)->($status) "
|
||||||
|
} else {
|
||||||
|
print -n $"(_ansi blue_bold) 🌥 (_ansi reset)"
|
||||||
|
}
|
||||||
|
sleep $wait_duration
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let res = if ($ops | str contains "--type" ) {
|
||||||
|
(^upctl server $new_state --type ($ops | str replace "--type " "") $server.hostname | complete)
|
||||||
|
} else if $ops != "" {
|
||||||
|
(^upctl server $new_state $ops $server.hostname | complete)
|
||||||
|
} else {
|
||||||
|
(^upctl server $new_state $server.hostname | complete)
|
||||||
|
}
|
||||||
|
if $res.exit_code != 0 {
|
||||||
|
print $"❗Errors ($server.hostname) to ($new_state) ($res.stdout ) "
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
$num = 0
|
||||||
|
while true {
|
||||||
|
let status = (local_status_server $server.hostname)
|
||||||
|
if ($status | str contains $new_state) {
|
||||||
|
print " "
|
||||||
|
return true
|
||||||
|
} else if $val_timeout > 0 and $num > $val_timeout {
|
||||||
|
print $"\n🛑 (_ansi red)Timeout(_ansi reset) ($val_timeout) (_ansi blue)($server.hostname)(_ansi reset) (_ansi blue_bold)($new_state)(_ansi reset) (_ansi red_bold)failed(_ansi reset) "
|
||||||
|
return false
|
||||||
|
} else {
|
||||||
|
$num = $num + $wait
|
||||||
|
if (is-debug-enabled) {
|
||||||
|
print -n $"(_ansi blue_bold) 🌥 (_ansi reset)(_ansi green)($server.hostname)(_ansi reset)->($status) "
|
||||||
|
} else {
|
||||||
|
print -n $"(_ansi blue_bold) 🌥 (_ansi reset)"
|
||||||
|
}
|
||||||
|
sleep $wait_duration
|
||||||
|
}
|
||||||
|
}
|
||||||
|
false
|
||||||
|
}
|
||||||
|
export def local_delete_server_storage [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
] {
|
||||||
|
print ($"✅ (_ansi blue_bold)($server.hostname)(_ansi reset) with provider " +
|
||||||
|
$"(_ansi green_bold)($server.provider)(_ansi reset) ($server.zone) does not require delete storage !" )
|
||||||
|
true
|
||||||
|
}
|
||||||
|
export def local_delete_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
keep_storage: bool
|
||||||
|
error_exit: bool
|
||||||
|
] {
|
||||||
|
print ($"✅ (_ansi blue_bold)($server.hostname)(_ansi reset) with provider " +
|
||||||
|
$"(_ansi green_bold)($server.provider)(_ansi reset) ($server.zone) does not require delete !" )
|
||||||
|
true
|
||||||
|
}
|
||||||
576
providers/local/nulib/local/servers.nu-e
Normal file
576
providers/local/nulib/local/servers.nu-e
Normal file
@ -0,0 +1,576 @@
|
|||||||
|
#!/usr/bin/env nu
|
||||||
|
use std
|
||||||
|
use ../../../../core/nulib/lib_provisioning/config/accessor.nu *
|
||||||
|
|
||||||
|
export def local_query_servers [
|
||||||
|
find: string
|
||||||
|
cols: string
|
||||||
|
] {
|
||||||
|
# TODO FIX
|
||||||
|
let res = (^upctl server list -o json err> (if $nu.os-info.name == "windows" { "NUL" } else { "/dev/null" }) | complete)
|
||||||
|
if $res.exit_code == 0 {
|
||||||
|
$res.stdout | from json | get servers
|
||||||
|
} else {
|
||||||
|
if (is-debug-enabled) {
|
||||||
|
(throw-error "🛑 local server list " $"($res.exit_code) ($res.stdout)" "local query server" --span (metadata $res).span)
|
||||||
|
} else {
|
||||||
|
print $"🛑 Error local server list: ($res.exit_code) ($res.stdout | ^grep 'error')"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def local_server_info [
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
] {
|
||||||
|
let hostname = $server.hostname
|
||||||
|
# TODO FIX
|
||||||
|
let res = (^upctl server show $hostname -o json err> (if $nu.os-info.name == "windows" { "NUL" } else { "/dev/null" }) | complete)
|
||||||
|
if $res.exit_code == 0 {
|
||||||
|
$res.stdout | from json
|
||||||
|
} else if $check {
|
||||||
|
{}
|
||||||
|
} else {
|
||||||
|
if (is-debug-enabled) {
|
||||||
|
(throw-error "🛑 local server show" $"($res.exit_code) ($res.stdout)" $"local server info ($hostname)" --span (metadata $res).span)
|
||||||
|
} else {
|
||||||
|
print $"🛑 local server show ($hostname):($res.stdout | ^grep 'error')"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def local_on_prov_server [
|
||||||
|
server?: record
|
||||||
|
infra?: string
|
||||||
|
] {
|
||||||
|
let info = if ( $env.CURRENT_FILE? | into string ) != "" { (^grep "^# Info:" $env.CURRENT_FILE ) | str replace "# Info: " "" } else { "" }
|
||||||
|
print $env.CURRENT_FILE
|
||||||
|
$" From LOCAL ($info) "
|
||||||
|
}
|
||||||
|
# infrastructure and services
|
||||||
|
export def local [
|
||||||
|
args: list<string> # Args for create command
|
||||||
|
--server(-s): record
|
||||||
|
#hostname?: string # Server hostname in settings
|
||||||
|
--serverpos (-p): int # Server position in settings
|
||||||
|
--check (-c) # Only check mode no servers will be created
|
||||||
|
--wait (-w) # Wait servers to be created
|
||||||
|
--infra (-i): string # Infra path
|
||||||
|
--settings (-s): string # Settings path
|
||||||
|
--outfile (-o): string # Output file
|
||||||
|
--debug (-x) # Use Debug mode
|
||||||
|
] {
|
||||||
|
if $debug { set-debug-enabled true }
|
||||||
|
let target = ($args | get -o 0 | default "")
|
||||||
|
let task = ($args | get -o 1 | default "")
|
||||||
|
let cmd_args = if ($args | length) > 1 { ($args | drop nth ..1) } else { [] }
|
||||||
|
match ($task) {
|
||||||
|
"help" | "h" | "" => {
|
||||||
|
print "TODO local help"
|
||||||
|
if not (is-debug-enabled) { end_run "" }
|
||||||
|
exit
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
if ($args | find "help" | length) > 0 {
|
||||||
|
match $task {
|
||||||
|
"server" => {
|
||||||
|
print "SERVER "
|
||||||
|
local_server ($args | drop nth ..0)
|
||||||
|
#local_server ($args | drop nth ..1) --server $server
|
||||||
|
},
|
||||||
|
"inventory" => {
|
||||||
|
local_server ($args | drop nth ..0)
|
||||||
|
},
|
||||||
|
"ssh" => {
|
||||||
|
local_server ($args | drop nth ..0)
|
||||||
|
},
|
||||||
|
"delete" => {
|
||||||
|
local_server ($args | drop nth ..0)
|
||||||
|
# ($args | drop nth ..1) --server $server
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
option_undefined "local" ""
|
||||||
|
print "TODO local help"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if not (is-debug-enabled) { end_run "" }
|
||||||
|
exit
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
#use utils/settings.nu [ load_settings ]
|
||||||
|
let curr_settings = if $infra != null {
|
||||||
|
if $settings != null {
|
||||||
|
(load_settings --infra $infra --settings $settings)
|
||||||
|
} else {
|
||||||
|
(load_settings --infra $infra)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
if $settings != null {
|
||||||
|
(load_settings --settings $settings)
|
||||||
|
} else {
|
||||||
|
(load_settings)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
match ($task) {
|
||||||
|
"get_ip" => {
|
||||||
|
local_get_ip $curr_settings $server ($cmd_args | get -o 0 | default "")
|
||||||
|
},
|
||||||
|
"server" => {
|
||||||
|
print (
|
||||||
|
local_server $cmd_args --server $server --settings $curr_settings --error_exit
|
||||||
|
)
|
||||||
|
},
|
||||||
|
"inventory" => {
|
||||||
|
},
|
||||||
|
"ssh" => {
|
||||||
|
},
|
||||||
|
"delete" => {
|
||||||
|
# ($args | drop nth ..1) --server $server
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
option_undefined "local" ""
|
||||||
|
if not (is-debug-enabled) { end_run "" }
|
||||||
|
exit
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def local_get_ip [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
ip_type: string
|
||||||
|
] {
|
||||||
|
match $ip_type {
|
||||||
|
"private" | "prv" | "priv" => {
|
||||||
|
return $"($server.network_private_ip)"
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
let ip = ($server.network_public_ip | default "")
|
||||||
|
# TODO FIX add NOT FOUND ERRORS
|
||||||
|
return $ip
|
||||||
|
#let result = (^upctl "server" "show" $server.hostname "-o" "json" | complete)
|
||||||
|
#if $result.exit_code == 0 {
|
||||||
|
# let data = ($result.stdout | from json)
|
||||||
|
# #let id = ($data.id? | default "")
|
||||||
|
# let ip_addresses = ($data.networking?.interfaces? | where {|it| ($it.type | str contains "public") }).ip_addresses?
|
||||||
|
# return $"(($ip_addresses | get -o 0).address? | get -o 0 | default '')"
|
||||||
|
#} else { "" }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
# To create infrastructure and services
|
||||||
|
export def local_server [
|
||||||
|
args: list<string> # Args for create command
|
||||||
|
--server(-s): record
|
||||||
|
--error_exit
|
||||||
|
--status
|
||||||
|
#hostname?: string # Server hostname in settings
|
||||||
|
--serverpos (-p): int # Server position in settings
|
||||||
|
--check (-c) # Only check mode no servers will be created
|
||||||
|
--wait (-w) # Wait servers to be created
|
||||||
|
--infra (-i): string # Infra path
|
||||||
|
--settings (-s): record # Settings path
|
||||||
|
--outfile (-o): string # Output file
|
||||||
|
--debug (-x) # Use Debug mode
|
||||||
|
] {
|
||||||
|
let task = ($args | get -o 0)
|
||||||
|
let target = if ($args | length) > 1 { ($args | get -o 1) } else { "" }
|
||||||
|
let cmd_args = if ($args | length) > 1 { ($args | drop nth ..1) } else { [] }
|
||||||
|
match ($task) {
|
||||||
|
"help" | "h" | "" => {
|
||||||
|
print "TODO local server help"
|
||||||
|
if not (is-debug-enabled) { end_run "" }
|
||||||
|
exit
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
if $target == "" or ($args | find "help" | length) > 0 {
|
||||||
|
match $task {
|
||||||
|
"server" => {
|
||||||
|
local_server $cmd_args
|
||||||
|
},
|
||||||
|
"status" => {
|
||||||
|
print $server
|
||||||
|
print $error_exit
|
||||||
|
}
|
||||||
|
"inventory" => {
|
||||||
|
print "TODO local server inventory help"
|
||||||
|
},
|
||||||
|
"ssh" => {
|
||||||
|
print "TODO local server ssh help"
|
||||||
|
},
|
||||||
|
"delete" => {
|
||||||
|
# ($args | drop nth ..1) --server $server
|
||||||
|
#local_delete_server $cmd_args true
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
option_undefined "local" "server"
|
||||||
|
print "TODO local server help"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if not (is-debug-enabled) { end_run "" }
|
||||||
|
exit
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let server_target = if $server != null {
|
||||||
|
$server
|
||||||
|
} else if $settings != null {
|
||||||
|
($settings.data.servers | where {|it| $it.hostname == $target } | get -o 0)
|
||||||
|
} else {
|
||||||
|
null
|
||||||
|
}
|
||||||
|
if $server_target == null {
|
||||||
|
if $error_exit {
|
||||||
|
let text = $"($args | str join ' ')"
|
||||||
|
(throw-error "🛑 local server" $text "" --span (metadata $server_target).span)
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
if $status or $task == "status" {
|
||||||
|
print "local server status "
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
match $task {
|
||||||
|
"get_ip" => {
|
||||||
|
local_get_ip $settings $server_target ($cmd_args | get -o 0 | default "")
|
||||||
|
},
|
||||||
|
"stop" => {
|
||||||
|
print "TODO local server stop"
|
||||||
|
},
|
||||||
|
"start" => {
|
||||||
|
print "TODO local server start"
|
||||||
|
},
|
||||||
|
"restart" => {
|
||||||
|
print "TODO local server restart"
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
option_undefined "local" "server"
|
||||||
|
if not (is-debug-enabled) { end_run "" }
|
||||||
|
exit
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def local_create_private_network [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
] {
|
||||||
|
if $server == null {
|
||||||
|
print $"❗ No server found in settings "
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
# new_upctl network list -o json |
|
||||||
|
# let net_id = ($data.networks | get -o 0 ).uuid)
|
||||||
|
let zone = ( $server.zone? | default "")
|
||||||
|
if $zone == "" {
|
||||||
|
print $"($server.hostname) No zone found to CREATE network_privat_id"
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
let network_private_name = ($server.network_private_name? | default "")
|
||||||
|
if $network_private_name == "" {
|
||||||
|
print $"($server.hostname) No network_private_name found to CREATE network_privat_id"
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
let priv_cidr_block = ($server.priv_cidr_block | default "")
|
||||||
|
if $network_private_name == "" {
|
||||||
|
print $"($server.hostname) No priv_cidr_block found to CREATE network_privat_id"
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
# EXAMPLE_BASH private_net_id=$(upctl network list -o yaml | $YQ '.networks[] | select(.ip_networks.ip_network[].address == "'"$priv_cidr_block"'") | .uuid' 2>/dev/null | sed 's,",,g')
|
||||||
|
let result = (^upctl "network" "list" "-o" "json" | complete)
|
||||||
|
let private_net_id = if $result.exit_code == 0 {
|
||||||
|
let data = ($result.stdout | from json )
|
||||||
|
($data.networks? | find $priv_cidr_block | get -o 0 | get -o uuid | default "")
|
||||||
|
} else {
|
||||||
|
""
|
||||||
|
}
|
||||||
|
if $check and $private_net_id == "" {
|
||||||
|
print $"❗private_network will be register in a real creation request not in check state"
|
||||||
|
return ""
|
||||||
|
} else if $private_net_id == "" {
|
||||||
|
let result = (^upctl network create --name $network_private_name --zone $zone --ip-network $"address='($priv_cidr_block)',dhcp=true" -o json ) | complete
|
||||||
|
let new_net_id = if $result.exit_code == 0 {
|
||||||
|
(($result.stdout | from json | find $priv_cidr_block | get -o 0).uuid? | default "")
|
||||||
|
} else { "" }
|
||||||
|
if $new_net_id == "" {
|
||||||
|
(throw-error $"🛑 no private network ($network_private_name) found"
|
||||||
|
$"for server ($server.hostname) ip ($server.network_private_ip)"
|
||||||
|
$"local_check_requirements" --span (metadata $new_net_id.span))
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
# Save changes ...
|
||||||
|
#use utils/settings.nu [ save_servers_settings save_settings_file ]
|
||||||
|
let match_text = " network_private_id = "
|
||||||
|
let defs_provider_path = $"($settings.data.server_path | get -o 0 | path dirname)/local_defaults"
|
||||||
|
save_servers_setings $settings $match_text $new_net_id
|
||||||
|
save_settings_file $settings $"($settings.src_path)/($settings.src)" $match_text $new_net_id
|
||||||
|
save_settings_file $settings $"($defs_provider_path)" $match_text $new_net_id
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
export def local_check_server_requirements [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
] {
|
||||||
|
if $server.provider != "local" { return false }
|
||||||
|
print ($"✅ (_ansi blue_bold)($server.hostname)(_ansi reset) with provider " +
|
||||||
|
$"(_ansi green_bold)($server.provider)(_ansi reset) ($server.zone) does not require creation !" )
|
||||||
|
true
|
||||||
|
}
|
||||||
|
|
||||||
|
export def local_make_settings [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
|
||||||
|
# # _delete_settings
|
||||||
|
let out_settings_path = $"($settings.infra_fullpath)/($server.provider)_settings.yaml"
|
||||||
|
let data = if ($out_settings_path | path exists ) {
|
||||||
|
(open $out_settings_path | from yaml)
|
||||||
|
} else {
|
||||||
|
null
|
||||||
|
}
|
||||||
|
let task = if $data != null { "update" } else { "create" }
|
||||||
|
let uuid = (^upctl server show $server.hostname "-o" "json" | from json).uuid? | default ""
|
||||||
|
# echo "settings:" > "$out_settings"
|
||||||
|
|
||||||
|
# for server in $(_settings_hosts)
|
||||||
|
# do
|
||||||
|
if $uuid == "" {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
let ip_pub = (local_get_ip $settings $server "public")
|
||||||
|
let ip_priv = (local_get_ip $settings $server "private")
|
||||||
|
|
||||||
|
let server_settings = {
|
||||||
|
name: $server.hostname,
|
||||||
|
id: $uuid,
|
||||||
|
private_net: {
|
||||||
|
id: $server.network_private_id
|
||||||
|
name: $server.network_private_name
|
||||||
|
},
|
||||||
|
zone: $server.zone,
|
||||||
|
datetime: $env.NOW,
|
||||||
|
ip_addresses: {
|
||||||
|
pub: $ip_pub, priv: $ip_priv
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let new_data = if $data != null and $data.servers? != null {
|
||||||
|
( $data.servers | each { |srv|
|
||||||
|
where {|it| $it.name != $server.hostname }
|
||||||
|
}) | append $server_settings
|
||||||
|
} else {
|
||||||
|
## create data record
|
||||||
|
{
|
||||||
|
servers: [ $server_settings ]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
$new_data | to yaml | save --force $out_settings_path
|
||||||
|
print $"✅ local settings ($task) -> ($out_settings_path)"
|
||||||
|
true
|
||||||
|
}
|
||||||
|
export def local_delete_settings [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
}
|
||||||
|
export def local_post_create_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
] {
|
||||||
|
if $server != null {
|
||||||
|
return (local_storage_fix_size $settings $server 0)
|
||||||
|
}
|
||||||
|
true
|
||||||
|
}
|
||||||
|
export def local_modify_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
new_values: list
|
||||||
|
error_exit: bool
|
||||||
|
] {
|
||||||
|
# TODO LOCAL
|
||||||
|
return
|
||||||
|
let res = (^upctl server $server.hostname modify ...($new_values) | complete)
|
||||||
|
if $res.exit_code != 0 {
|
||||||
|
print $"❗ Server ($server.hostname) modify ($new_values | str join ' ') errors ($res.stdout ) "
|
||||||
|
if $error_exit {
|
||||||
|
exit 1
|
||||||
|
} else {
|
||||||
|
return "error"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def local_storage_fix_size [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
storage_pos: int
|
||||||
|
] {
|
||||||
|
# TODO LOCAL
|
||||||
|
return
|
||||||
|
let total_size = ($server | get -o storages | get $storage_pos | get -o total | default 0)
|
||||||
|
if $total_size == 0 { return 0 }
|
||||||
|
let storage = (^upctl server show $server.hostname "-o" "json" | from json).storage_devices | (get -o $storage_pos)
|
||||||
|
if $storage == null { return 0 }
|
||||||
|
let curr_size = $storage.storage_size? | default 0
|
||||||
|
if $curr_size == 0 { return 0 }
|
||||||
|
if $curr_size != $total_size {
|
||||||
|
print (
|
||||||
|
$"Stop (_ansi blue_bold)($server.hostname)(_ansi reset) for storage (_ansi yellow_bold)($storage.storage)(_ansi reset)" +
|
||||||
|
$" from (_ansi purple_bold)($curr_size)(_ansi reset) to (_ansi green_bold)($total_size)(_ansi reset) ... "
|
||||||
|
)
|
||||||
|
if (local_change_server_state $settings $server "stop" "") == false {
|
||||||
|
print $"❗ Stop ($server.hostname) errors "
|
||||||
|
return "error"
|
||||||
|
}
|
||||||
|
let res = (^upctl storage modify --size $total_size $storage.storage | complete)
|
||||||
|
if $res.exit_code != 0 {
|
||||||
|
print $"❗ Storage modify errors ($res.stdout ) "
|
||||||
|
return "error"
|
||||||
|
}
|
||||||
|
let new_storage = (^upctl server show $server.hostname "-o" "json" | from json).storage_devices | (get -o $storage_pos)
|
||||||
|
let new_curr_size = $new_storage.storage_size? | default 0
|
||||||
|
print $"Start (_ansi blue_bold)($server.hostname)(_ansi reset) with new size (_ansi green_bold)($new_curr_size)(_ansi reset) ... "
|
||||||
|
if (local_change_server_state $settings $server "start" "") == false {
|
||||||
|
print $"❗ Errors to start ($server.hostname): ($res.stdout ) "
|
||||||
|
return "error"
|
||||||
|
}
|
||||||
|
return "storage"
|
||||||
|
}
|
||||||
|
""
|
||||||
|
}
|
||||||
|
export def local_status_server [
|
||||||
|
hostname: string
|
||||||
|
] {
|
||||||
|
let res = (^upctl server show $hostname "-o" "json" | complete)
|
||||||
|
if $res.exit_code != 0 {
|
||||||
|
print $"❗ status ($hostname) errors ($res.stdout ) "
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
return (($res.stdout | from json).state | default "")
|
||||||
|
}
|
||||||
|
export def local_server_exists [
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
] {
|
||||||
|
let res = (^upctl server show $server.hostname "-o" "json" err> (if $nu.os-info.name == "windows" { "NUL" } else { "/dev/null" }) | complete)
|
||||||
|
if $res.exit_code != 0 {
|
||||||
|
if $error_exit {
|
||||||
|
print $"❗ status ($server.hostname) errors ($res.stdout ) "
|
||||||
|
exit 1
|
||||||
|
} else {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
true
|
||||||
|
}
|
||||||
|
export def local_server_state [
|
||||||
|
server: record
|
||||||
|
new_state: string
|
||||||
|
error_exit: bool
|
||||||
|
wait: bool
|
||||||
|
settings: record
|
||||||
|
] {
|
||||||
|
}
|
||||||
|
export def local_server_is_running [
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
] {
|
||||||
|
true
|
||||||
|
#TODO FIX
|
||||||
|
# let res = (^upctl server show $server.hostname "-o" "json" err> (if $nu.os-info.name == "windows" { "NUL" } else { "/dev/null" }) | complete)
|
||||||
|
# if $res.exit_code != 0 {
|
||||||
|
# print $"❗ status ($server.hostname) errors ($res.stdout ) "
|
||||||
|
# if $error_exit {
|
||||||
|
# exit 1
|
||||||
|
# } else {
|
||||||
|
# return false
|
||||||
|
# }
|
||||||
|
# }
|
||||||
|
# (($res.stdout | from json).state? | str contains "started" | default false)
|
||||||
|
}
|
||||||
|
export def local_change_server_state [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
new_state: string
|
||||||
|
ops: string
|
||||||
|
] {
|
||||||
|
let state = (local_status_server $server.hostname)
|
||||||
|
if $state == "" { return false }
|
||||||
|
if ($state | str contains $new_state) { return true }
|
||||||
|
print $"Checking (_ansi blue_bold)($server.hostname)(_ansi reset) state (_ansi yellow_bold)($new_state)(_ansi reset) ..."
|
||||||
|
let val_timeout = if $server.running_timeout? != null { $server.running_timeout } else { 60 }
|
||||||
|
let wait = if $server.running_wait? != null { $server.running_wait } else { 10 }
|
||||||
|
let wait_duration = ($"($wait)sec"| into duration)
|
||||||
|
mut num = 0
|
||||||
|
while true {
|
||||||
|
let status = (local_status_server $server.hostname)
|
||||||
|
if $status == "" {
|
||||||
|
return false
|
||||||
|
} else if ($status | str contains "maintenance") == false {
|
||||||
|
print " "
|
||||||
|
break
|
||||||
|
} else if $val_timeout > 0 and $num > $val_timeout {
|
||||||
|
print $"\n🛑 (_ansi red)Timeout(_ansi reset) ($val_timeout) (_ansi blue)($server.hostname)(_ansi reset) (_ansi blue_bold)($new_state)(_ansi reset) (_ansi red_bold)failed(_ansi reset) "
|
||||||
|
return false
|
||||||
|
} else {
|
||||||
|
$num = $num + $wait
|
||||||
|
if (is-debug-enabled) {
|
||||||
|
print -n $"(_ansi blue_bold) 🌥 (_ansi reset)(_ansi green)($server.hostname)(_ansi reset)->($status) "
|
||||||
|
} else {
|
||||||
|
print -n $"(_ansi blue_bold) 🌥 (_ansi reset)"
|
||||||
|
}
|
||||||
|
sleep $wait_duration
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let res = if ($ops | str contains "--type" ) {
|
||||||
|
(^upctl server $new_state --type ($ops | str replace "--type " "") $server.hostname | complete)
|
||||||
|
} else if $ops != "" {
|
||||||
|
(^upctl server $new_state $ops $server.hostname | complete)
|
||||||
|
} else {
|
||||||
|
(^upctl server $new_state $server.hostname | complete)
|
||||||
|
}
|
||||||
|
if $res.exit_code != 0 {
|
||||||
|
print $"❗Errors ($server.hostname) to ($new_state) ($res.stdout ) "
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
$num = 0
|
||||||
|
while true {
|
||||||
|
let status = (local_status_server $server.hostname)
|
||||||
|
if ($status | str contains $new_state) {
|
||||||
|
print " "
|
||||||
|
return true
|
||||||
|
} else if $val_timeout > 0 and $num > $val_timeout {
|
||||||
|
print $"\n🛑 (_ansi red)Timeout(_ansi reset) ($val_timeout) (_ansi blue)($server.hostname)(_ansi reset) (_ansi blue_bold)($new_state)(_ansi reset) (_ansi red_bold)failed(_ansi reset) "
|
||||||
|
return false
|
||||||
|
} else {
|
||||||
|
$num = $num + $wait
|
||||||
|
if (is-debug-enabled) {
|
||||||
|
print -n $"(_ansi blue_bold) 🌥 (_ansi reset)(_ansi green)($server.hostname)(_ansi reset)->($status) "
|
||||||
|
} else {
|
||||||
|
print -n $"(_ansi blue_bold) 🌥 (_ansi reset)"
|
||||||
|
}
|
||||||
|
sleep $wait_duration
|
||||||
|
}
|
||||||
|
}
|
||||||
|
false
|
||||||
|
}
|
||||||
|
export def local_delete_server_storage [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
] {
|
||||||
|
print ($"✅ (_ansi blue_bold)($server.hostname)(_ansi reset) with provider " +
|
||||||
|
$"(_ansi green_bold)($server.provider)(_ansi reset) ($server.zone) does not require delete storage !" )
|
||||||
|
true
|
||||||
|
}
|
||||||
|
export def local_delete_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
keep_storage: bool
|
||||||
|
error_exit: bool
|
||||||
|
] {
|
||||||
|
print ($"✅ (_ansi blue_bold)($server.hostname)(_ansi reset) with provider " +
|
||||||
|
$"(_ansi green_bold)($server.provider)(_ansi reset) ($server.zone) does not require delete !" )
|
||||||
|
true
|
||||||
|
}
|
||||||
41
providers/local/nulib/local/usage.nu
Normal file
41
providers/local/nulib/local/usage.nu
Normal file
@ -0,0 +1,41 @@
|
|||||||
|
|
||||||
|
#!/usr/bin/env nu
|
||||||
|
|
||||||
|
# myscript.nu
|
||||||
|
export def usage [provider: string, infra: string] {
|
||||||
|
let info = if ( $env.CURRENT_FILE? | into string ) != "" { (^grep "^# Info:" $env.CURRENT_FILE ) | str replace "# Info: " "" } else { "" }
|
||||||
|
# $(declare -F _usage_options >/dev/null && _usage_options)
|
||||||
|
$"
|
||||||
|
USAGE provisioning ($provider) -k cloud-path file-settings.yaml provider-options
|
||||||
|
DESCRIPTION
|
||||||
|
LOCAL ($info)
|
||||||
|
OPTIONS
|
||||||
|
-s server-hostname
|
||||||
|
with server-hostname target selection
|
||||||
|
-p provider-name
|
||||||
|
use provider name
|
||||||
|
do not need if 'current directory path basename' is not one of providers available
|
||||||
|
-new | new [provisioning-name]
|
||||||
|
create a new provisioning-directory-name by a copy of ($infra)
|
||||||
|
-k cloud-path-item
|
||||||
|
use cloud-path-item as base directory for settings
|
||||||
|
-x
|
||||||
|
Trace script with 'set -x'
|
||||||
|
providerslist | providers-list | providers list
|
||||||
|
Get available providers list
|
||||||
|
taskslist | tasks-list | tasks list
|
||||||
|
Get available tasks list
|
||||||
|
serviceslist | service-list
|
||||||
|
Get available services list
|
||||||
|
tools
|
||||||
|
Run core/on-tools info
|
||||||
|
-i
|
||||||
|
About this
|
||||||
|
-v
|
||||||
|
Print version
|
||||||
|
-h, --help
|
||||||
|
Print this help and exit.
|
||||||
|
"
|
||||||
|
# ["hello" $name $title]
|
||||||
|
}
|
||||||
|
|
||||||
41
providers/local/nulib/local/usage.nu-e
Normal file
41
providers/local/nulib/local/usage.nu-e
Normal file
@ -0,0 +1,41 @@
|
|||||||
|
|
||||||
|
#!/usr/bin/env nu
|
||||||
|
|
||||||
|
# myscript.nu
|
||||||
|
export def usage [provider: string, infra: string] {
|
||||||
|
let info = if ( $env.CURRENT_FILE? | into string ) != "" { (^grep "^# Info:" $env.CURRENT_FILE ) | str replace "# Info: " "" } else { "" }
|
||||||
|
# $(declare -F _usage_options >/dev/null && _usage_options)
|
||||||
|
$"
|
||||||
|
USAGE provisioning ($provider) -k cloud-path file-settings.yaml provider-options
|
||||||
|
DESCRIPTION
|
||||||
|
LOCAL ($info)
|
||||||
|
OPTIONS
|
||||||
|
-s server-hostname
|
||||||
|
with server-hostname target selection
|
||||||
|
-p provider-name
|
||||||
|
use provider name
|
||||||
|
do not need if 'current directory path basename' is not one of providers available
|
||||||
|
-new | new [provisioning-name]
|
||||||
|
create a new provisioning-directory-name by a copy of ($infra)
|
||||||
|
-k cloud-path-item
|
||||||
|
use cloud-path-item as base directory for settings
|
||||||
|
-x
|
||||||
|
Trace script with 'set -x'
|
||||||
|
providerslist | providers-list | providers list
|
||||||
|
Get available providers list
|
||||||
|
taskslist | tasks-list | tasks list
|
||||||
|
Get available tasks list
|
||||||
|
serviceslist | service-list
|
||||||
|
Get available services list
|
||||||
|
tools
|
||||||
|
Run core/on-tools info
|
||||||
|
-i
|
||||||
|
About this
|
||||||
|
-v
|
||||||
|
Print version
|
||||||
|
-h, --help
|
||||||
|
Print this help and exit.
|
||||||
|
"
|
||||||
|
# ["hello" $name $title]
|
||||||
|
}
|
||||||
|
|
||||||
0
providers/local/nulib/local/utils.nu
Normal file
0
providers/local/nulib/local/utils.nu
Normal file
0
providers/local/nulib/local/utils.nu-e
Normal file
0
providers/local/nulib/local/utils.nu-e
Normal file
304
providers/local/provider.nu
Normal file
304
providers/local/provider.nu
Normal file
@ -0,0 +1,304 @@
|
|||||||
|
# Local Provider Adapter
|
||||||
|
# Implements the standard provider interface for Local (bare metal/existing servers)
|
||||||
|
|
||||||
|
use nulib/local/env.nu
|
||||||
|
use nulib/local/servers.nu *
|
||||||
|
|
||||||
|
# Provider metadata
|
||||||
|
export def get-provider-metadata []: nothing -> record {
|
||||||
|
{
|
||||||
|
name: "local"
|
||||||
|
version: "1.0.0"
|
||||||
|
description: "Local provider for bare metal and existing servers"
|
||||||
|
capabilities: {
|
||||||
|
server_management: true
|
||||||
|
network_management: false # Limited network management
|
||||||
|
storage_management: true
|
||||||
|
load_balancer: false
|
||||||
|
dns_management: false
|
||||||
|
cdn: false
|
||||||
|
backup_service: false
|
||||||
|
monitoring: false
|
||||||
|
logging: false
|
||||||
|
auto_scaling: false
|
||||||
|
spot_instances: false
|
||||||
|
containers: true
|
||||||
|
serverless: false
|
||||||
|
multi_region: false
|
||||||
|
encryption_at_rest: false
|
||||||
|
encryption_in_transit: false
|
||||||
|
compliance_certifications: []
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Server query operations
|
||||||
|
export def query_servers [
|
||||||
|
find?: string
|
||||||
|
cols?: string
|
||||||
|
]: nothing -> list {
|
||||||
|
let str_find = if $find != null { $find } else { "" }
|
||||||
|
let str_cols = if $cols != null { $cols } else { "" }
|
||||||
|
local_query_servers $str_find $str_cols
|
||||||
|
}
|
||||||
|
|
||||||
|
# Server information operations
|
||||||
|
export def server_info [
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
find?: string
|
||||||
|
cols?: string
|
||||||
|
]: nothing -> record {
|
||||||
|
local_server_info $server $check
|
||||||
|
}
|
||||||
|
|
||||||
|
# Server existence and status operations
|
||||||
|
export def server_exists [
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
local_server_exists $server $error_exit
|
||||||
|
}
|
||||||
|
|
||||||
|
export def server_is_running [
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
local_server_is_running $server $error_exit
|
||||||
|
}
|
||||||
|
|
||||||
|
# Server lifecycle operations
|
||||||
|
export def check_server_requirements [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
local_check_server_requirements $settings $server $check
|
||||||
|
}
|
||||||
|
|
||||||
|
export def create_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
wait: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
# Local provider doesn't "create" servers, they already exist
|
||||||
|
# This just validates the server is accessible
|
||||||
|
try {
|
||||||
|
local_server_state $server "started" true $wait $settings
|
||||||
|
true
|
||||||
|
} catch {
|
||||||
|
false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export def delete_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
keep_storage: bool
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
local_delete_server $settings $server $keep_storage $error_exit
|
||||||
|
}
|
||||||
|
|
||||||
|
export def delete_server_storage [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
local_delete_server_storage $settings $server $error_exit
|
||||||
|
}
|
||||||
|
|
||||||
|
export def post_create_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
local_post_create_server $settings $server $check
|
||||||
|
}
|
||||||
|
|
||||||
|
export def modify_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
new_values: list
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
local_modify_server $settings $server $new_values $error_exit
|
||||||
|
}
|
||||||
|
|
||||||
|
# Server state management
|
||||||
|
export def server_state [
|
||||||
|
server: record
|
||||||
|
new_state: string
|
||||||
|
error_exit: bool
|
||||||
|
wait: bool
|
||||||
|
settings: record
|
||||||
|
]: nothing -> bool {
|
||||||
|
local_server_state $server $new_state $error_exit $wait $settings
|
||||||
|
}
|
||||||
|
|
||||||
|
# Network operations
|
||||||
|
export def get_ip [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
ip_type: string
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> string {
|
||||||
|
let use_type = match $ip_type {
|
||||||
|
"$network_public_ip" => "public"
|
||||||
|
"$network_private_ip" => "private"
|
||||||
|
_ => $ip_type
|
||||||
|
}
|
||||||
|
|
||||||
|
let result = (local_server [ "get_ip", $use_type ] --server $server --settings $settings)
|
||||||
|
$result | str trim
|
||||||
|
}
|
||||||
|
|
||||||
|
export def servers_ips [
|
||||||
|
settings: record
|
||||||
|
data: list
|
||||||
|
prov?: string
|
||||||
|
serverpos?: int
|
||||||
|
]: nothing -> list {
|
||||||
|
mut result = []
|
||||||
|
mut index = -1
|
||||||
|
|
||||||
|
for srv in $data {
|
||||||
|
$index += 1
|
||||||
|
let settings_server = ($settings.data.servers | where {|it| $it.hostname == $srv.hostname})
|
||||||
|
if ($settings_server | length) == 0 { continue }
|
||||||
|
let provider = ($settings_server | get -o 0 | get -o provider | default "")
|
||||||
|
if $prov != null and $provider != "local" { continue }
|
||||||
|
if $serverpos != null and $serverpos != $index { continue }
|
||||||
|
|
||||||
|
if $srv.ip_addresses? != null {
|
||||||
|
$result = ($result | append ($srv.ip_addresses? |
|
||||||
|
each {|it| { hostname: $srv.hostname, ip: $it.address, access: $it.access, family: $it.family }} |
|
||||||
|
flatten
|
||||||
|
))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
$result
|
||||||
|
}
|
||||||
|
|
||||||
|
# Infrastructure operations (simplified for local)
|
||||||
|
export def load_infra_servers_info [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> record {
|
||||||
|
# Local provider doesn't have cloud infrastructure info
|
||||||
|
{
|
||||||
|
provider: "local"
|
||||||
|
server: $server.hostname
|
||||||
|
info: "Local servers don't have cloud infrastructure info"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export def load_infra_storages_info [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> record {
|
||||||
|
# Local storage info is basic
|
||||||
|
{
|
||||||
|
provider: "local"
|
||||||
|
server: $server.hostname
|
||||||
|
storage: "Local storage managed directly on server"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export def get_infra_storage [
|
||||||
|
server: record
|
||||||
|
settings: record
|
||||||
|
cloud_data: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> list {
|
||||||
|
# Return empty list for local provider
|
||||||
|
[]
|
||||||
|
}
|
||||||
|
|
||||||
|
export def get_infra_item [
|
||||||
|
server: record
|
||||||
|
settings: record
|
||||||
|
cloud_data: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> record {
|
||||||
|
# Return basic server info
|
||||||
|
{
|
||||||
|
provider: "local"
|
||||||
|
hostname: $server.hostname
|
||||||
|
type: "local-server"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export def get_infra_price [
|
||||||
|
server: record
|
||||||
|
data: record
|
||||||
|
key: string
|
||||||
|
error_exit: bool
|
||||||
|
price_col?: string
|
||||||
|
]: nothing -> float {
|
||||||
|
# Local servers don't have pricing
|
||||||
|
0.0
|
||||||
|
}
|
||||||
|
|
||||||
|
# Cache operations (local doesn't use cloud caching)
|
||||||
|
export def start_cache_info [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
]: nothing -> nothing {
|
||||||
|
# No-op for local provider
|
||||||
|
}
|
||||||
|
|
||||||
|
export def create_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> nothing {
|
||||||
|
# No-op for local provider
|
||||||
|
}
|
||||||
|
|
||||||
|
export def read_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> nothing {
|
||||||
|
# No-op for local provider
|
||||||
|
}
|
||||||
|
|
||||||
|
export def clean_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> nothing {
|
||||||
|
# No-op for local provider
|
||||||
|
}
|
||||||
|
|
||||||
|
export def ip_from_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> nothing {
|
||||||
|
# For local, just return the configured IP
|
||||||
|
$server | get -o network_public_ip | default ""
|
||||||
|
}
|
||||||
|
|
||||||
|
# Provider metadata operations
|
||||||
|
export def on_prov_server [
|
||||||
|
server: record
|
||||||
|
]: nothing -> string {
|
||||||
|
local_on_prov_server $server
|
||||||
|
}
|
||||||
|
|
||||||
|
# Provider validation
|
||||||
|
export def validate_provider []: nothing -> record {
|
||||||
|
{
|
||||||
|
provider: "local"
|
||||||
|
valid: true
|
||||||
|
interface_version: "1.0.0"
|
||||||
|
capabilities: (get-provider-metadata).capabilities
|
||||||
|
last_validated: (date now)
|
||||||
|
}
|
||||||
|
}
|
||||||
4
providers/local/provisioning.yaml
Normal file
4
providers/local/provisioning.yaml
Normal file
@ -0,0 +1,4 @@
|
|||||||
|
version: 1.0
|
||||||
|
info: Local provisioning
|
||||||
|
site: ""
|
||||||
|
tools: []
|
||||||
89
providers/local/templates/local_servers.j2
Normal file
89
providers/local/templates/local_servers.j2
Normal file
@ -0,0 +1,89 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# on_local_server creation: {{now}}
|
||||||
|
|
||||||
|
{%- for server in servers %}
|
||||||
|
{%- if server.hostname %}
|
||||||
|
{#
|
||||||
|
if upctl server show {{server.hostname}} >/dev/null 2>/dev/null ; then
|
||||||
|
echo "Server {{server.hostname}} already created."
|
||||||
|
else
|
||||||
|
{% if use_time and use_time == 'true' %} time {%- endif -%}
|
||||||
|
upctl server create \
|
||||||
|
--hostname {{server.hostname}} \
|
||||||
|
{%- if server.title and server.title != '' %}
|
||||||
|
--title "{{server.title}}" \
|
||||||
|
{%- endif -%}
|
||||||
|
{%- if server.plan %}
|
||||||
|
--plan {{server.plan}} \
|
||||||
|
{%- elif defaults.plan and defaults.plan != '' %}
|
||||||
|
--plan {{defaults.plan}} \
|
||||||
|
{%- endif -%}
|
||||||
|
{%- if server.zone %}
|
||||||
|
--zone {{server.zone}} \
|
||||||
|
{%- elif defaults.zone and defaults.zone != '' %}
|
||||||
|
--zone {{defaults.zone}} \
|
||||||
|
{%- endif -%}
|
||||||
|
{%- if server.ssh_key_path %}
|
||||||
|
--ssh-keys {{server.ssh_key_path}} \
|
||||||
|
{%- elif defaults.ssh_key_path and defaults.ssh_key_path != '' %}
|
||||||
|
--ssh-keys {{defaults.ssh_key_path}} \
|
||||||
|
{%- endif -%}
|
||||||
|
{%- if server.storage_os %}
|
||||||
|
--os "{{server.storage_os}}" \
|
||||||
|
{%- elif defaults.storage_os and defaults.storage_os != '' %}
|
||||||
|
--os "{{defaults.storage_os}}" \
|
||||||
|
{%- endif -%}
|
||||||
|
{%- if server.storage_size %}
|
||||||
|
--os-storage-size {{server.storage_size}} \
|
||||||
|
{%- elif defaults.storage_size and defaults.storage_size > 0 %}
|
||||||
|
--os-storage-size {{defaults.storage_size}} \
|
||||||
|
{%- endif -%}
|
||||||
|
{%- if server.network_public_ipv4 %}
|
||||||
|
--network family=IPv4,type=public \
|
||||||
|
{%- elif defaults.network_public_ipv4 and defaults.network_public_ipv4 != '' %}
|
||||||
|
--network family=IPv4,type=public \
|
||||||
|
{%- endif -%}
|
||||||
|
{%- if server.network_public_ipv6 %}
|
||||||
|
--network family=IPv6,type=public \
|
||||||
|
{%- elif defaults.network_public_ipv6 and defaults.network_public_ipv6 != '' %}
|
||||||
|
--network family=IPv6,type=public \
|
||||||
|
{%- endif -%}
|
||||||
|
{%- if server.network_utility_ipv4 %}
|
||||||
|
--network family=IPv4,type=utility \
|
||||||
|
{%- elif defaults.network_utility_ipv4 and defaults.network_utility_ipv4 != '' %}
|
||||||
|
--network family=IPv4,type=utility \
|
||||||
|
{%- endif -%}
|
||||||
|
{%- if server.network_utility_ipv6 %}
|
||||||
|
--network family=IPv6,type=utility \
|
||||||
|
{%- elif defaults.network_utility_ipv6 and defaults.network_utility_ipv6 != '' %}
|
||||||
|
--network family=IPv6,type=utility \
|
||||||
|
{%- endif -%}
|
||||||
|
{%- if server.network_private_ip %}
|
||||||
|
{%- if server.network_private_id %}
|
||||||
|
--network type=private,network={{server.network_private_id}},ip-address={{server.network_private_ip}} \
|
||||||
|
{%- elif defaults.network_private_id %}
|
||||||
|
--network type=private,network={{defaults.network_private_id}},ip-address={{server.network_private_ip}} \
|
||||||
|
{%- endif -%}
|
||||||
|
{%- endif -%}
|
||||||
|
{%- if server.time_zone %}
|
||||||
|
--time-zone {{server.time_zone}} \
|
||||||
|
{%- elif defaults.time_zone and defaults.time_zone != '' %}
|
||||||
|
--time-zone {{defaults.time_zone}} \
|
||||||
|
{%- endif -%}
|
||||||
|
{%- if server.labels %}
|
||||||
|
--label {{server.labels}} \
|
||||||
|
{%- endif -%}
|
||||||
|
{%- if defaults.labels and defaults.labels != '' %}
|
||||||
|
--labels {{defaults.labels}} \
|
||||||
|
{%- endif -%}
|
||||||
|
{%- if wait %}
|
||||||
|
--wait \
|
||||||
|
{%- endif -%}
|
||||||
|
{%- if runset.output_format and runset.output_format != '' %}
|
||||||
|
-o {{runset.output_format}} \
|
||||||
|
{%- endif %}
|
||||||
|
--enable-metadata >> {{wk_file}}
|
||||||
|
fi
|
||||||
|
#}
|
||||||
|
{%- endif -%}
|
||||||
|
{%- endfor %}
|
||||||
0
providers/local/versions
Normal file
0
providers/local/versions
Normal file
882
providers/prov_lib/create_middleware.nu
Normal file
882
providers/prov_lib/create_middleware.nu
Normal file
@ -0,0 +1,882 @@
|
|||||||
|
def provider_lib_has_method [
|
||||||
|
providers_path: string
|
||||||
|
prov: string
|
||||||
|
method: string
|
||||||
|
]: nothing -> bool {
|
||||||
|
let prov_root = ($providers_path | path join $prov | path join "nulib" | path join $prov)
|
||||||
|
let res = (^grep $method ...(glob ($prov_root | path join "*")) err> /dev/null | complete)
|
||||||
|
($res.stdout | is-not-empty)
|
||||||
|
}
|
||||||
|
|
||||||
|
def make_provider_undefined [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
'def provider_undefined [
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
#use defs/lists.nu providers_list
|
||||||
|
let str_providers_list = (providers_list | each { |it| $it.name} | str join " ")
|
||||||
|
print ($"(_ansi blue_bold)($server.hostname)(_ansi reset) with provider " +
|
||||||
|
$"(_ansi green_bold)($server.provider)(_ansi reset) ($server.zone) ")
|
||||||
|
let text = ( $"expected to be one of available providers [(_ansi green_italic)($str_providers_list)(_ansi reset)], " +
|
||||||
|
$"got (_ansi green_bold)($server.provider)(_ansi reset)")
|
||||||
|
print $"Error 🛑 provider ($text)"
|
||||||
|
}'
|
||||||
|
}
|
||||||
|
def make_mw_query_servers [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_query_servers [
|
||||||
|
settings: record
|
||||||
|
find?: string
|
||||||
|
cols?: string
|
||||||
|
--prov: string
|
||||||
|
--serverpos: int
|
||||||
|
] {
|
||||||
|
let str_find = if $find != null { $find } else { "" }
|
||||||
|
let str_cols = if $cols != null { $cols } else { "" }
|
||||||
|
$settings.data.servers | enumerate | each { |it|
|
||||||
|
#let res = for idx in ..($settings.data.servers | length) {
|
||||||
|
#let srv = ($settings.data.servers | get -o $idx)
|
||||||
|
if $prov == null or $it.item.provider == $prov {
|
||||||
|
if $serverpos == null or $serverpos == $it.index {
|
||||||
|
let res = match $it.item.provider {'
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_query_servers"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $str_find $str_cols)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $it.item
|
||||||
|
[]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if ($res | length) > 0 {
|
||||||
|
let result = if $str_find != "" {
|
||||||
|
$res | find $str_find
|
||||||
|
} else {
|
||||||
|
$res
|
||||||
|
}
|
||||||
|
if $str_cols != "" {
|
||||||
|
let field_list = ($str_cols | split row ",")
|
||||||
|
($result | select -o $field_list)
|
||||||
|
} else {
|
||||||
|
$result
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
# $list | append $srv
|
||||||
|
} | flatten
|
||||||
|
}' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_servers_ips [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_servers_ips [
|
||||||
|
settings: record
|
||||||
|
data: list
|
||||||
|
prov?: string
|
||||||
|
serverpos?: int
|
||||||
|
]: nothing -> list {
|
||||||
|
mut index = -1
|
||||||
|
mut result = []
|
||||||
|
for srv in $data {
|
||||||
|
$index += 1
|
||||||
|
let settings_server = ($settings.data.servers | where {|it| $it.hostname == $srv.hostname})
|
||||||
|
if ($settings_server | length) == 0 { continue }
|
||||||
|
let provider = ($settings_server | get -o provider | default "")
|
||||||
|
if $prov != null and $provider != $prov { continue }
|
||||||
|
if $serverpos != null and $serverpos != $index { continue }
|
||||||
|
match $provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {' | append '
|
||||||
|
if $srv.ip_addresses? != null {
|
||||||
|
$result = ($result | append ($srv.ip_addresses? |
|
||||||
|
each {|it| { hostname: $srv.hostname, ip: $it.address, access: $it.access, family: $it.family }} |
|
||||||
|
flatten
|
||||||
|
))
|
||||||
|
}
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $srv.provider
|
||||||
|
[]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
$result
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_server_info [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_server_info [
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
find?: string
|
||||||
|
cols?: string
|
||||||
|
]: nothing -> record {
|
||||||
|
let str_find = if $find != null { $find } else { "" }
|
||||||
|
let str_cols = if $cols != null { $cols } else { "" }
|
||||||
|
let res = match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_server_info"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $server $check)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output = ($output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server.hostname
|
||||||
|
[]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if $res.hostname? != null {
|
||||||
|
let result = if $str_find != "" {
|
||||||
|
$res | find $str_find
|
||||||
|
} else {
|
||||||
|
$res
|
||||||
|
}
|
||||||
|
let info = if $str_cols != "" {
|
||||||
|
let field_list = ($str_cols | split row ",")
|
||||||
|
($result | select -o $field_list)
|
||||||
|
} else {
|
||||||
|
($result)
|
||||||
|
}
|
||||||
|
let priv = match $server.provider {' | str join "")
|
||||||
|
for prov in $providers_list {
|
||||||
|
if $prov == "aws" {
|
||||||
|
$output = ($output | append '
|
||||||
|
"aws" => {
|
||||||
|
($info | get -o private_ips | default [] | each {|it| ($it | select Description PrivateIpAddress VpcId SubnetId Groups) })
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => ($info | get -o priv | default "")
|
||||||
|
}
|
||||||
|
let full_info = if ($priv | length) > 0 {
|
||||||
|
($info | merge { private_ips: $priv })
|
||||||
|
} else {
|
||||||
|
$info
|
||||||
|
}
|
||||||
|
if not $check {
|
||||||
|
print ($full_info | table -e)
|
||||||
|
}
|
||||||
|
$full_info
|
||||||
|
} else {
|
||||||
|
$res
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_servers_info [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
'
|
||||||
|
export def mw_servers_info [
|
||||||
|
settings: record
|
||||||
|
find?: string
|
||||||
|
cols?: string
|
||||||
|
--prov: string
|
||||||
|
--serverpos: int
|
||||||
|
--check
|
||||||
|
]: nothing -> nothing {
|
||||||
|
let str_find = if $find != null { $find } else { "" }
|
||||||
|
let str_cols = if $cols != null { $cols } else { "" }
|
||||||
|
$settings.data.servers | enumerate | each { |it|
|
||||||
|
if $prov == null or $it.item.provider == $prov {
|
||||||
|
if $serverpos == null or $serverpos == $it.index {
|
||||||
|
mw_server_info $it.item $check $str_find $str_cols
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}'
|
||||||
|
}
|
||||||
|
def make_mw_create_server [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_create_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
let zone = $server.zone? | default ""
|
||||||
|
let res = match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_check_server_requirements"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
print ($"\(($prov)_on_prov_server $server)")
|
||||||
|
($"\(($method) $settings $server $check)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { false }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if not $res {
|
||||||
|
(throw-error $"🛑 ($server.provider) check requirements error"
|
||||||
|
$"for server ($server.hostname)"
|
||||||
|
"create_server" --span (metadata $server.provider).span)
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
print ($"Create (_ansi blue_bold)($server.hostname)(_ansi reset) with provider " +
|
||||||
|
$"(_ansi green_bold)($server.provider)(_ansi reset) ($zone) ")
|
||||||
|
return true
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_server_state [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_server_state [
|
||||||
|
server: record
|
||||||
|
new_state: string
|
||||||
|
error_exit: bool
|
||||||
|
wait: bool
|
||||||
|
settings: record
|
||||||
|
]: nothing -> bool {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_server_state"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $server $new_state $error_exit $wait $settings)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { return false }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
true
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_server_exists [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_server_exists [
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
match $server.provider {'
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_server_exists"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $server $error_exit)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { false }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_server_is_running [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_server_is_running [
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_server_is_running"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $server $error_exit)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { false }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_get_ip [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_get_ip [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
ip_type: string
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> string {
|
||||||
|
let use_type = match $ip_type {
|
||||||
|
"$network_public_ip" => "public",
|
||||||
|
"$network_private_ip" => "private",
|
||||||
|
_ => $ip_type
|
||||||
|
}
|
||||||
|
let res = match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_server"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) [ "get_ip", $use_type ] --server $server --settings $settings)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { "" }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
$"($res)" | str trim
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_wait_storage [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_wait_storage [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
new_state: str
|
||||||
|
id: str
|
||||||
|
]: nothing -> record {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_wait_storage"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
let str_it = "$server"
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $settings $server $new_state $id)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server.provider
|
||||||
|
true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_create_storage [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_create_storage [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
server_info: record
|
||||||
|
storage: record
|
||||||
|
volumes: list
|
||||||
|
total_size: int
|
||||||
|
]: nothing -> record {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_create_storage"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
let str_it = "$server"
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $settings $server $server_info $storage $volumes $total_size)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server.provider
|
||||||
|
true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_post_create_server [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_post_create_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_post_create_server"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
let str_it = "$server"
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $settings ($str_it) $check)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server.provider
|
||||||
|
true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_modify_server [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_modify_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
new_values: list
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_modify_server"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
let str_it = "$server"
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $settings ($str_it) $new_values $error_exit)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server.provider
|
||||||
|
true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_delete_server_storage [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_delete_server_storage [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
let zone = $server.zone? | default ""
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_delete_server_storage"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
print ($"\(($prov)_on_prov_server $server)")
|
||||||
|
($"\(($method) $settings $server $error_exit)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { false }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
#print ($"Delete storage (_ansi blue_bold)($server.hostname)(_ansi reset) with provider " +
|
||||||
|
# $"(_ansi green_bold)($server.provider)(_ansi reset) ($zone) ")
|
||||||
|
#true
|
||||||
|
}
|
||||||
|
def make_mw_delete_server [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_delete_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
keep_storage: bool
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
let zone = $server.zone? | default ""
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_delete_server"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
print ($"\(($prov)_on_prov_server $server)")
|
||||||
|
($"\(($method) $settings $server $keep_storage $error_exit)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { false }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
#print ($"Delete (_ansi blue_bold)($server.hostname)(_ansi reset) with provider " +
|
||||||
|
#$"(_ansi green_bold)($server.provider)(_ansi reset) ($zone) ")
|
||||||
|
#true
|
||||||
|
}
|
||||||
|
def make_mw_load_infra_servers_info [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_load_infra_servers_info [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> record {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_load_infra_servers_info"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $settings $server $error_exit)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { {} }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join "\n"
|
||||||
|
}
|
||||||
|
def make_mw_load_infra_storages_info [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_load_infra_storages_info [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> record {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_load_infra_storages_info"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $settings $server $error_exit)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { {} }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join "\n"
|
||||||
|
}
|
||||||
|
def make_mw_get_infra_storage [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_get_infra_storage [
|
||||||
|
server: record
|
||||||
|
settings: record
|
||||||
|
cloud_data: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> list {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_get_item_for_storage"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $server $settings $cloud_data)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { [] }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_get_infra_item [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_get_infra_item [
|
||||||
|
server: record
|
||||||
|
settings: record
|
||||||
|
cloud_data: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> record {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_get_item_for_server"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $server $settings $cloud_data)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { return {} }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_get_infra_price [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_get_infra_price [
|
||||||
|
server: record
|
||||||
|
data: record
|
||||||
|
key: string
|
||||||
|
error_exit: bool
|
||||||
|
price_col?: string
|
||||||
|
]: nothing -> float {
|
||||||
|
if ($data | get -o item | is-empty) { return {} }
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_get_price"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $data $key $price_col)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { return 0 }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_start_cache_info [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_start_cache_info [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
]: nothing -> nothing {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_start_cache_info"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $settings $server)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_create_cache [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_create_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> nothing {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_create_cache"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $settings $server $error_exit)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { return 0 }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_read_cache [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_read_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> nothing {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_read_cache"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $settings $server $error_exit)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { return }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_clean_cache [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_clean_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> nothing {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_clean_cache"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $settings $server $error_exit)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { return }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_ip_from_cache [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_ip_from_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> nothing {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_ip_from_cache"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $settings $server $error_exit)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
"local" => {
|
||||||
|
($server | get -o network_public_ip | default "")
|
||||||
|
#(local_ip_from_cache $settings $server $error_exit)
|
||||||
|
}
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { return }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
# - > Make middleware (middleware.nu env_middleware.nu) for existing providers
|
||||||
|
export def make_middleware [
|
||||||
|
] {
|
||||||
|
use ../../core/nulib/lib_provisioning/config/accessor.nu get-providers-path
|
||||||
|
let providers_path = (get-providers-path)
|
||||||
|
if not ($providers_path | path exists) {
|
||||||
|
print $"🛑 providers path (ansi red_bold)($providers_path)(ansi reset) not found"
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
let middleware_path = ($providers_path | path join "prov_lib" | path join "middleware.nu" )
|
||||||
|
let env_middleware_path = ($providers_path | path join "prov_lib" | path join "env_middleware.nu" )
|
||||||
|
let providers_list = (ls -s $providers_path | where {|it| (
|
||||||
|
($it.name | str starts-with "_") == false
|
||||||
|
and ($providers_path | path join $it.name | path type) == "dir"
|
||||||
|
and ($providers_path | path join $it.name | path join "templates" | path exists)
|
||||||
|
)
|
||||||
|
} | select name | values | flatten )
|
||||||
|
let use_list = [ servers.nu, cache.nu, prices.nu, utils.nu ]
|
||||||
|
mut output = $"# CNPROV middleware generated by 'make_middleware' on (date now | format date '%Y-%m-%d %H:%M:%S')"
|
||||||
|
mut env_output = ($output | append "\nexport-env {" | str join "\n")
|
||||||
|
for prov in $providers_list {
|
||||||
|
let prov_root = ($providers_path | path join $prov | path join "nulib" | path join $prov)
|
||||||
|
if not ($prov_root | path exists ) { continue }
|
||||||
|
if ($prov_root | path join "env.nu" | path exists ) { $env_output = ($env_output | append $" use ($prov)/env.nu" | str join "\n") }
|
||||||
|
for $item in $use_list {
|
||||||
|
if ($prov_root | path join $item | path exists ) { $output = ($output | append $"use ($prov)/($item) *" | str join "\n") }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
$env_output | append "}" | str join "" | save --force $env_middleware_path
|
||||||
|
$output | append (make_provider_undefined $providers_path $providers_list) | str join "\n"
|
||||||
|
| append (make_mw_query_servers $providers_path $providers_list)
|
||||||
|
| append (make_mw_servers_ips $providers_path $providers_list)
|
||||||
|
| append (make_mw_server_info $providers_path $providers_list)
|
||||||
|
| append (make_mw_servers_info $providers_path $providers_list)
|
||||||
|
| append (make_mw_create_server $providers_path $providers_list)
|
||||||
|
| append (make_mw_server_state $providers_path $providers_list)
|
||||||
|
| append (make_mw_server_exists $providers_path $providers_list)
|
||||||
|
| append (make_mw_server_is_running $providers_path $providers_list)
|
||||||
|
| append (make_mw_get_ip $providers_path $providers_list)
|
||||||
|
| append (make_mw_wait_storage $providers_path $providers_list)
|
||||||
|
| append (make_mw_create_storage $providers_path $providers_list)
|
||||||
|
| append (make_mw_post_create_server $providers_path $providers_list)
|
||||||
|
| append (make_mw_modify_server $providers_path $providers_list)
|
||||||
|
| append (make_mw_delete_server_storage $providers_path $providers_list)
|
||||||
|
| append (make_mw_delete_server $providers_path $providers_list)
|
||||||
|
| append (make_mw_load_infra_servers_info $providers_path $providers_list)
|
||||||
|
| append (make_mw_load_infra_storages_info $providers_path $providers_list)
|
||||||
|
| append (make_mw_get_infra_storage $providers_path $providers_list)
|
||||||
|
| append (make_mw_get_infra_item $providers_path $providers_list)
|
||||||
|
| append (make_mw_get_infra_price $providers_path $providers_list)
|
||||||
|
| append (make_mw_start_cache_info $providers_path $providers_list)
|
||||||
|
| append (make_mw_create_cache $providers_path $providers_list)
|
||||||
|
| append (make_mw_read_cache $providers_path $providers_list)
|
||||||
|
| append (make_mw_clean_cache $providers_path $providers_list)
|
||||||
|
| str join ""
|
||||||
|
| save --force $middleware_path
|
||||||
|
}
|
||||||
882
providers/prov_lib/create_middleware.nu-e
Normal file
882
providers/prov_lib/create_middleware.nu-e
Normal file
@ -0,0 +1,882 @@
|
|||||||
|
def provider_lib_has_method [
|
||||||
|
providers_path: string
|
||||||
|
prov: string
|
||||||
|
method: string
|
||||||
|
]: nothing -> bool {
|
||||||
|
let prov_root = ($providers_path | path join $prov | path join "nulib" | path join $prov)
|
||||||
|
let res = (^grep $method ...(glob ($prov_root | path join "*")) err> /dev/null | complete)
|
||||||
|
($res.stdout | is-not-empty)
|
||||||
|
}
|
||||||
|
|
||||||
|
def make_provider_undefined [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
'def provider_undefined [
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
#use defs/lists.nu providers_list
|
||||||
|
let str_providers_list = (providers_list | each { |it| $it.name} | str join " ")
|
||||||
|
print ($"(_ansi blue_bold)($server.hostname)(_ansi reset) with provider " +
|
||||||
|
$"(_ansi green_bold)($server.provider)(_ansi reset) ($server.zone) ")
|
||||||
|
let text = ( $"expected to be one of available providers [(_ansi green_italic)($str_providers_list)(_ansi reset)], " +
|
||||||
|
$"got (_ansi green_bold)($server.provider)(_ansi reset)")
|
||||||
|
print $"Error 🛑 provider ($text)"
|
||||||
|
}'
|
||||||
|
}
|
||||||
|
def make_mw_query_servers [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_query_servers [
|
||||||
|
settings: record
|
||||||
|
find?: string
|
||||||
|
cols?: string
|
||||||
|
--prov: string
|
||||||
|
--serverpos: int
|
||||||
|
] {
|
||||||
|
let str_find = if $find != null { $find } else { "" }
|
||||||
|
let str_cols = if $cols != null { $cols } else { "" }
|
||||||
|
$settings.data.servers | enumerate | each { |it|
|
||||||
|
#let res = for idx in ..($settings.data.servers | length) {
|
||||||
|
#let srv = ($settings.data.servers | get -o $idx)
|
||||||
|
if $prov == null or $it.item.provider == $prov {
|
||||||
|
if $serverpos == null or $serverpos == $it.index {
|
||||||
|
let res = match $it.item.provider {'
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_query_servers"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $str_find $str_cols)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $it.item
|
||||||
|
[]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if ($res | length) > 0 {
|
||||||
|
let result = if $str_find != "" {
|
||||||
|
$res | find $str_find
|
||||||
|
} else {
|
||||||
|
$res
|
||||||
|
}
|
||||||
|
if $str_cols != "" {
|
||||||
|
let field_list = ($str_cols | split row ",")
|
||||||
|
($result | select -o $field_list)
|
||||||
|
} else {
|
||||||
|
$result
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
# $list | append $srv
|
||||||
|
} | flatten
|
||||||
|
}' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_servers_ips [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_servers_ips [
|
||||||
|
settings: record
|
||||||
|
data: list
|
||||||
|
prov?: string
|
||||||
|
serverpos?: int
|
||||||
|
]: nothing -> list {
|
||||||
|
mut index = -1
|
||||||
|
mut result = []
|
||||||
|
for srv in $data {
|
||||||
|
$index += 1
|
||||||
|
let settings_server = ($settings.data.servers | where {|it| $it.hostname == $srv.hostname})
|
||||||
|
if ($settings_server | length) == 0 { continue }
|
||||||
|
let provider = ($settings_server | get -o provider | default "")
|
||||||
|
if $prov != null and $provider != $prov { continue }
|
||||||
|
if $serverpos != null and $serverpos != $index { continue }
|
||||||
|
match $provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {' | append '
|
||||||
|
if $srv.ip_addresses? != null {
|
||||||
|
$result = ($result | append ($srv.ip_addresses? |
|
||||||
|
each {|it| { hostname: $srv.hostname, ip: $it.address, access: $it.access, family: $it.family }} |
|
||||||
|
flatten
|
||||||
|
))
|
||||||
|
}
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $srv.provider
|
||||||
|
[]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
$result
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_server_info [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_server_info [
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
find?: string
|
||||||
|
cols?: string
|
||||||
|
]: nothing -> record {
|
||||||
|
let str_find = if $find != null { $find } else { "" }
|
||||||
|
let str_cols = if $cols != null { $cols } else { "" }
|
||||||
|
let res = match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_server_info"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $server $check)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output = ($output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server.hostname
|
||||||
|
[]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if $res.hostname? != null {
|
||||||
|
let result = if $str_find != "" {
|
||||||
|
$res | find $str_find
|
||||||
|
} else {
|
||||||
|
$res
|
||||||
|
}
|
||||||
|
let info = if $str_cols != "" {
|
||||||
|
let field_list = ($str_cols | split row ",")
|
||||||
|
($result | select -o $field_list)
|
||||||
|
} else {
|
||||||
|
($result)
|
||||||
|
}
|
||||||
|
let priv = match $server.provider {' | str join "")
|
||||||
|
for prov in $providers_list {
|
||||||
|
if $prov == "aws" {
|
||||||
|
$output = ($output | append '
|
||||||
|
"aws" => {
|
||||||
|
($info | get -o private_ips | default [] | each {|it| ($it | select Description PrivateIpAddress VpcId SubnetId Groups) })
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => ($info | get -o priv | default "")
|
||||||
|
}
|
||||||
|
let full_info = if ($priv | length) > 0 {
|
||||||
|
($info | merge { private_ips: $priv })
|
||||||
|
} else {
|
||||||
|
$info
|
||||||
|
}
|
||||||
|
if not $check {
|
||||||
|
print ($full_info | table -e)
|
||||||
|
}
|
||||||
|
$full_info
|
||||||
|
} else {
|
||||||
|
$res
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_servers_info [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
'
|
||||||
|
export def mw_servers_info [
|
||||||
|
settings: record
|
||||||
|
find?: string
|
||||||
|
cols?: string
|
||||||
|
--prov: string
|
||||||
|
--serverpos: int
|
||||||
|
--check
|
||||||
|
]: nothing -> nothing {
|
||||||
|
let str_find = if $find != null { $find } else { "" }
|
||||||
|
let str_cols = if $cols != null { $cols } else { "" }
|
||||||
|
$settings.data.servers | enumerate | each { |it|
|
||||||
|
if $prov == null or $it.item.provider == $prov {
|
||||||
|
if $serverpos == null or $serverpos == $it.index {
|
||||||
|
mw_server_info $it.item $check $str_find $str_cols
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}'
|
||||||
|
}
|
||||||
|
def make_mw_create_server [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_create_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
let zone = $server.zone? | default ""
|
||||||
|
let res = match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_check_server_requirements"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
print ($"\(($prov)_on_prov_server $server)")
|
||||||
|
($"\(($method) $settings $server $check)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { false }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if not $res {
|
||||||
|
(throw-error $"🛑 ($server.provider) check requirements error"
|
||||||
|
$"for server ($server.hostname)"
|
||||||
|
"create_server" --span (metadata $server.provider).span)
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
print ($"Create (_ansi blue_bold)($server.hostname)(_ansi reset) with provider " +
|
||||||
|
$"(_ansi green_bold)($server.provider)(_ansi reset) ($zone) ")
|
||||||
|
return true
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_server_state [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_server_state [
|
||||||
|
server: record
|
||||||
|
new_state: string
|
||||||
|
error_exit: bool
|
||||||
|
wait: bool
|
||||||
|
settings: record
|
||||||
|
]: nothing -> bool {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_server_state"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $server $new_state $error_exit $wait $settings)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { return false }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
true
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_server_exists [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_server_exists [
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
match $server.provider {'
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_server_exists"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $server $error_exit)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { false }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_server_is_running [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_server_is_running [
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_server_is_running"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $server $error_exit)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { false }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_get_ip [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_get_ip [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
ip_type: string
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> string {
|
||||||
|
let use_type = match $ip_type {
|
||||||
|
"$network_public_ip" => "public",
|
||||||
|
"$network_private_ip" => "private",
|
||||||
|
_ => $ip_type
|
||||||
|
}
|
||||||
|
let res = match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_server"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) [ "get_ip", $use_type ] --server $server --settings $settings)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { "" }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
$"($res)" | str trim
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_wait_storage [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_wait_storage [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
new_state: str
|
||||||
|
id: str
|
||||||
|
]: nothing -> record {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_wait_storage"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
let str_it = "$server"
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $settings $server $new_state $id)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server.provider
|
||||||
|
true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_create_storage [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_create_storage [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
server_info: record
|
||||||
|
storage: record
|
||||||
|
volumes: list
|
||||||
|
total_size: int
|
||||||
|
]: nothing -> record {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_create_storage"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
let str_it = "$server"
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $settings $server $server_info $storage $volumes $total_size)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server.provider
|
||||||
|
true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_post_create_server [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_post_create_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_post_create_server"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
let str_it = "$server"
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $settings ($str_it) $check)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server.provider
|
||||||
|
true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_modify_server [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_modify_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
new_values: list
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_modify_server"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
let str_it = "$server"
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $settings ($str_it) $new_values $error_exit)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server.provider
|
||||||
|
true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_delete_server_storage [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_delete_server_storage [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
let zone = $server.zone? | default ""
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_delete_server_storage"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
print ($"\(($prov)_on_prov_server $server)")
|
||||||
|
($"\(($method) $settings $server $error_exit)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { false }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
#print ($"Delete storage (_ansi blue_bold)($server.hostname)(_ansi reset) with provider " +
|
||||||
|
# $"(_ansi green_bold)($server.provider)(_ansi reset) ($zone) ")
|
||||||
|
#true
|
||||||
|
}
|
||||||
|
def make_mw_delete_server [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_delete_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
keep_storage: bool
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
let zone = $server.zone? | default ""
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_delete_server"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
print ($"\(($prov)_on_prov_server $server)")
|
||||||
|
($"\(($method) $settings $server $keep_storage $error_exit)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { false }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
#print ($"Delete (_ansi blue_bold)($server.hostname)(_ansi reset) with provider " +
|
||||||
|
#$"(_ansi green_bold)($server.provider)(_ansi reset) ($zone) ")
|
||||||
|
#true
|
||||||
|
}
|
||||||
|
def make_mw_load_infra_servers_info [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_load_infra_servers_info [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> record {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_load_infra_servers_info"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $settings $server $error_exit)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { {} }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join "\n"
|
||||||
|
}
|
||||||
|
def make_mw_load_infra_storages_info [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_load_infra_storages_info [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> record {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_load_infra_storages_info"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $settings $server $error_exit)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { {} }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join "\n"
|
||||||
|
}
|
||||||
|
def make_mw_get_infra_storage [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_get_infra_storage [
|
||||||
|
server: record
|
||||||
|
settings: record
|
||||||
|
cloud_data: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> list {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_get_item_for_storage"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $server $settings $cloud_data)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { [] }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_get_infra_item [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_get_infra_item [
|
||||||
|
server: record
|
||||||
|
settings: record
|
||||||
|
cloud_data: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> record {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_get_item_for_server"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $server $settings $cloud_data)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { return {} }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_get_infra_price [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_get_infra_price [
|
||||||
|
server: record
|
||||||
|
data: record
|
||||||
|
key: string
|
||||||
|
error_exit: bool
|
||||||
|
price_col?: string
|
||||||
|
]: nothing -> float {
|
||||||
|
if ($data | get -o item | is-empty) { return {} }
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_get_price"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $data $key $price_col)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { return 0 }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_start_cache_info [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_start_cache_info [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
]: nothing -> nothing {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_start_cache_info"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $settings $server)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_create_cache [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_create_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> nothing {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_create_cache"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $settings $server $error_exit)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { return 0 }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_read_cache [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_read_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> nothing {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_read_cache"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $settings $server $error_exit)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { return }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_clean_cache [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_clean_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> nothing {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_clean_cache"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $settings $server $error_exit)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { return }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
def make_mw_ip_from_cache [
|
||||||
|
providers_path: string
|
||||||
|
providers_list: list<string>
|
||||||
|
]: nothing -> string {
|
||||||
|
mut output = '
|
||||||
|
export def mw_ip_from_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> nothing {
|
||||||
|
match $server.provider { '
|
||||||
|
for prov in $providers_list {
|
||||||
|
let method = $"($prov)_ip_from_cache"
|
||||||
|
if not (provider_lib_has_method $providers_path $prov $method) { continue }
|
||||||
|
$output = ($output | append $'
|
||||||
|
"($prov)" => {
|
||||||
|
($"\(($method) $settings $server $error_exit)")
|
||||||
|
},' | str join "")
|
||||||
|
}
|
||||||
|
$output | append '
|
||||||
|
"local" => {
|
||||||
|
($server | get -o network_public_ip | default "")
|
||||||
|
#(local_ip_from_cache $settings $server $error_exit)
|
||||||
|
}
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { return }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} ' | str join ""
|
||||||
|
}
|
||||||
|
# - > Make middleware (middleware.nu env_middleware.nu) for existing providers
|
||||||
|
export def make_middleware [
|
||||||
|
] {
|
||||||
|
use ../../core/nulib/lib_provisioning/config/accessor.nu get-providers-path
|
||||||
|
let providers_path = (get-providers-path)
|
||||||
|
if not ($providers_path | path exists) {
|
||||||
|
print $"🛑 providers path (ansi red_bold)($providers_path)(ansi reset) not found"
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
let middleware_path = ($providers_path | path join "prov_lib" | path join "middleware.nu" )
|
||||||
|
let env_middleware_path = ($providers_path | path join "prov_lib" | path join "env_middleware.nu" )
|
||||||
|
let providers_list = (ls -s $providers_path | where {|it| (
|
||||||
|
($it.name | str starts-with "_") == false
|
||||||
|
and ($providers_path | path join $it.name | path type) == "dir"
|
||||||
|
and ($providers_path | path join $it.name | path join "templates" | path exists)
|
||||||
|
)
|
||||||
|
} | select name | values | flatten )
|
||||||
|
let use_list = [ servers.nu, cache.nu, prices.nu, utils.nu ]
|
||||||
|
mut output = $"# CNPROV middleware generated by 'make_middleware' on (date now | format date '%Y-%m-%d %H:%M:%S')"
|
||||||
|
mut env_output = ($output | append "\nexport-env {" | str join "\n")
|
||||||
|
for prov in $providers_list {
|
||||||
|
let prov_root = ($providers_path | path join $prov | path join "nulib" | path join $prov)
|
||||||
|
if not ($prov_root | path exists ) { continue }
|
||||||
|
if ($prov_root | path join "env.nu" | path exists ) { $env_output = ($env_output | append $" use ($prov)/env.nu" | str join "\n") }
|
||||||
|
for $item in $use_list {
|
||||||
|
if ($prov_root | path join $item | path exists ) { $output = ($output | append $"use ($prov)/($item) *" | str join "\n") }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
$env_output | append "}" | str join "" | save --force $env_middleware_path
|
||||||
|
$output | append (make_provider_undefined $providers_path $providers_list) | str join "\n"
|
||||||
|
| append (make_mw_query_servers $providers_path $providers_list)
|
||||||
|
| append (make_mw_servers_ips $providers_path $providers_list)
|
||||||
|
| append (make_mw_server_info $providers_path $providers_list)
|
||||||
|
| append (make_mw_servers_info $providers_path $providers_list)
|
||||||
|
| append (make_mw_create_server $providers_path $providers_list)
|
||||||
|
| append (make_mw_server_state $providers_path $providers_list)
|
||||||
|
| append (make_mw_server_exists $providers_path $providers_list)
|
||||||
|
| append (make_mw_server_is_running $providers_path $providers_list)
|
||||||
|
| append (make_mw_get_ip $providers_path $providers_list)
|
||||||
|
| append (make_mw_wait_storage $providers_path $providers_list)
|
||||||
|
| append (make_mw_create_storage $providers_path $providers_list)
|
||||||
|
| append (make_mw_post_create_server $providers_path $providers_list)
|
||||||
|
| append (make_mw_modify_server $providers_path $providers_list)
|
||||||
|
| append (make_mw_delete_server_storage $providers_path $providers_list)
|
||||||
|
| append (make_mw_delete_server $providers_path $providers_list)
|
||||||
|
| append (make_mw_load_infra_servers_info $providers_path $providers_list)
|
||||||
|
| append (make_mw_load_infra_storages_info $providers_path $providers_list)
|
||||||
|
| append (make_mw_get_infra_storage $providers_path $providers_list)
|
||||||
|
| append (make_mw_get_infra_item $providers_path $providers_list)
|
||||||
|
| append (make_mw_get_infra_price $providers_path $providers_list)
|
||||||
|
| append (make_mw_start_cache_info $providers_path $providers_list)
|
||||||
|
| append (make_mw_create_cache $providers_path $providers_list)
|
||||||
|
| append (make_mw_read_cache $providers_path $providers_list)
|
||||||
|
| append (make_mw_clean_cache $providers_path $providers_list)
|
||||||
|
| str join ""
|
||||||
|
| save --force $middleware_path
|
||||||
|
}
|
||||||
6
providers/prov_lib/env_middleware.nu
Normal file
6
providers/prov_lib/env_middleware.nu
Normal file
@ -0,0 +1,6 @@
|
|||||||
|
# CNPROV middleware generated by 'make_middleware' on 2024-04-08_21:24:42
|
||||||
|
export-env {
|
||||||
|
use aws/env.nu
|
||||||
|
use local/env.nu
|
||||||
|
use upcloud/env.nu
|
||||||
|
}
|
||||||
6
providers/prov_lib/env_middleware.nu-e
Normal file
6
providers/prov_lib/env_middleware.nu-e
Normal file
@ -0,0 +1,6 @@
|
|||||||
|
# CNPROV middleware generated by 'make_middleware' on 2024-04-08_21:24:42
|
||||||
|
export-env {
|
||||||
|
use aws/env.nu
|
||||||
|
use local/env.nu
|
||||||
|
use upcloud/env.nu
|
||||||
|
}
|
||||||
544
providers/prov_lib/middleware.nu
Normal file
544
providers/prov_lib/middleware.nu
Normal file
@ -0,0 +1,544 @@
|
|||||||
|
# Provider-Agnostic Middleware
|
||||||
|
# Uses dynamic provider loading and interface-based dispatch
|
||||||
|
# Supports multi-provider infrastructure deployments
|
||||||
|
|
||||||
|
use ../../../core/nulib/lib_provisioning/config/accessor.nu *
|
||||||
|
use ../../../core/nulib/lib_provisioning/extensions/profiles.nu *
|
||||||
|
use ../../../core/nulib/lib_provisioning/providers/registry.nu *
|
||||||
|
use ../../../core/nulib/lib_provisioning/providers/loader.nu *
|
||||||
|
use ../../../core/nulib/lib_provisioning/providers/interface.nu *
|
||||||
|
use ../../../core/nulib/lib_provisioning/utils/logging.nu *
|
||||||
|
|
||||||
|
# Initialize middleware
|
||||||
|
export def init-middleware []: nothing -> nothing {
|
||||||
|
init-provider-registry
|
||||||
|
log-info "Provider-agnostic middleware initialized" "middleware"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Check if provider is allowed by profile (safer version)
|
||||||
|
def is_provider_allowed [provider_name: string]: nothing -> bool {
|
||||||
|
# Disabled - causes | complete issues
|
||||||
|
true
|
||||||
|
}
|
||||||
|
|
||||||
|
def provider_undefined [
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
let server_prov = ($server | get -o provider | default "")
|
||||||
|
let available_providers = (list-providers --available-only | get name | str join ", ")
|
||||||
|
# This error is misleading - it's called when function dispatch fails, not when provider isn't found
|
||||||
|
# The actual issue is that the provider function call failed
|
||||||
|
log-error $"Provider function returned null for provider ($server_prov)" "middleware"
|
||||||
|
print $"(_ansi blue_bold)($server.hostname)(_ansi reset) with provider " +
|
||||||
|
$"(_ansi green_bold)($server_prov)(_ansi reset) ($server.zone?) "
|
||||||
|
print $"Note: Provider was loaded but function call failed or returned empty result"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Dynamic provider dispatch helper
|
||||||
|
def dispatch_provider_function [
|
||||||
|
provider_name: string
|
||||||
|
function_name: string
|
||||||
|
...args
|
||||||
|
]: nothing -> any {
|
||||||
|
# Check if provider is allowed
|
||||||
|
if not (is_provider_allowed $provider_name) {
|
||||||
|
log-error $"Provider ($provider_name) blocked by profile" "middleware"
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
|
||||||
|
# Load provider if not already loaded
|
||||||
|
let provider = (get-provider $provider_name)
|
||||||
|
if ($provider | is-empty) {
|
||||||
|
log-error $"Failed to load provider ($provider_name)" "middleware"
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
|
||||||
|
# Call provider function
|
||||||
|
call-provider-function $provider_name $function_name ...$args
|
||||||
|
}
|
||||||
|
|
||||||
|
# === CORE MIDDLEWARE FUNCTIONS ===
|
||||||
|
|
||||||
|
# Query servers (supports multi-provider)
|
||||||
|
export def mw_query_servers [
|
||||||
|
settings: record
|
||||||
|
find?: string
|
||||||
|
cols?: string
|
||||||
|
--prov: string
|
||||||
|
--serverpos: int
|
||||||
|
]: nothing -> list {
|
||||||
|
if not ($env.PROVIDER_REGISTRY_INITIALIZED? | default false) {
|
||||||
|
init-middleware | ignore
|
||||||
|
}
|
||||||
|
|
||||||
|
let str_find = if $find != null { $find } else { "" }
|
||||||
|
let str_cols = if $cols != null { $cols } else { "" }
|
||||||
|
|
||||||
|
$settings.data.servers | enumerate | each { |it|
|
||||||
|
if $prov == null or $it.item.provider == $prov {
|
||||||
|
if $serverpos == null or $serverpos == $it.index {
|
||||||
|
let provider_name = $it.item.provider
|
||||||
|
|
||||||
|
# Dynamic provider dispatch
|
||||||
|
let res = (dispatch_provider_function $provider_name "query_servers" $str_find $str_cols )
|
||||||
|
if $res == null {
|
||||||
|
provider_undefined $it.item
|
||||||
|
[]
|
||||||
|
} else {
|
||||||
|
$res
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($res != null) and ($res | length) > 0 {
|
||||||
|
let result = if $str_find != "" {
|
||||||
|
$res | find $str_find
|
||||||
|
} else {
|
||||||
|
$res
|
||||||
|
}
|
||||||
|
if $str_cols != "" {
|
||||||
|
let field_list = ($str_cols | split row ",")
|
||||||
|
($result | select -o $field_list)
|
||||||
|
} else {
|
||||||
|
$result
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
[]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} | flatten
|
||||||
|
}
|
||||||
|
|
||||||
|
# Server information (provider-agnostic)
|
||||||
|
export def mw_server_info [
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
find?: string
|
||||||
|
cols?: string
|
||||||
|
]: nothing -> record {
|
||||||
|
let str_find = if $find != null { $find } else { "" }
|
||||||
|
let str_cols = if $cols != null { $cols } else { "" }
|
||||||
|
|
||||||
|
let res = (dispatch_provider_function $server.provider "server_info" $server $check $str_find $str_cols )
|
||||||
|
if $res == null {
|
||||||
|
provider_undefined $server
|
||||||
|
{}
|
||||||
|
} else {
|
||||||
|
$res
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($res | describe | str starts-with "record") and $res.hostname? != null {
|
||||||
|
let result = if $str_find != "" {
|
||||||
|
$res | find $str_find
|
||||||
|
} else {
|
||||||
|
$res
|
||||||
|
}
|
||||||
|
let info = if $str_cols != "" {
|
||||||
|
let field_list = ($str_cols | split row ",")
|
||||||
|
($result | select -o $field_list)
|
||||||
|
} else {
|
||||||
|
$result
|
||||||
|
}
|
||||||
|
|
||||||
|
# Handle provider-specific private IP formats
|
||||||
|
let priv = if $server.provider == "aws" {
|
||||||
|
($info | get -o private_ips | default [] | each {|it| ($it | select Description PrivateIpAddress VpcId SubnetId Groups) })
|
||||||
|
} else {
|
||||||
|
($info | get -o priv | default [])
|
||||||
|
}
|
||||||
|
|
||||||
|
let full_info = if ($priv | length) > 0 {
|
||||||
|
($info | merge { private_ips: $priv })
|
||||||
|
} else {
|
||||||
|
$info
|
||||||
|
}
|
||||||
|
|
||||||
|
let out = (get-provisioning-out)
|
||||||
|
if ($out | is-empty) {
|
||||||
|
print ($full_info | table -e)
|
||||||
|
}
|
||||||
|
if (not $check) {
|
||||||
|
($full_info | table -e)
|
||||||
|
}
|
||||||
|
$full_info
|
||||||
|
} else {
|
||||||
|
$res
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Server creation (provider-agnostic with profile checking)
|
||||||
|
export def mw_create_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
# Check if provider is allowed by profile
|
||||||
|
if not (is_provider_allowed $server.provider) {
|
||||||
|
log-error $"Provider ($server.provider) blocked by current profile" "middleware"
|
||||||
|
if $error_exit { exit } else { return false }
|
||||||
|
}
|
||||||
|
|
||||||
|
let zone = $server.zone? | default ""
|
||||||
|
|
||||||
|
# Dynamic provider dispatch for requirements check
|
||||||
|
let res = (dispatch_provider_function $server.provider "check_server_requirements" $settings $server $check)
|
||||||
|
if $res == null {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { return false }
|
||||||
|
}
|
||||||
|
|
||||||
|
if not $res {
|
||||||
|
log-error $"($server.provider) check requirements error for server ($server.hostname)" "middleware"
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
# Provider-specific message (most providers don't implement this, so we skip it)
|
||||||
|
# If needed in future, provider can override the default "Create..." message below
|
||||||
|
|
||||||
|
_print ($"Create (_ansi blue_bold)($server.hostname)(_ansi reset) with provider " +
|
||||||
|
$"(_ansi green_bold)($server.provider)(_ansi reset) ($zone) ")
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
# Server state management (provider-agnostic)
|
||||||
|
export def mw_server_state [
|
||||||
|
server: record
|
||||||
|
new_state: string
|
||||||
|
error_exit: bool
|
||||||
|
wait: bool
|
||||||
|
settings: record
|
||||||
|
]: nothing -> bool {
|
||||||
|
# Check if provider is allowed by profile
|
||||||
|
if not (is_provider_allowed $server.provider) {
|
||||||
|
log-error $"Provider ($server.provider) blocked by current profile" "middleware"
|
||||||
|
if $error_exit { exit } else { return false }
|
||||||
|
}
|
||||||
|
|
||||||
|
let res = (dispatch_provider_function $server.provider "server_state" $server $new_state $error_exit $wait $settings )
|
||||||
|
if $res == null {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { return false }
|
||||||
|
} else {
|
||||||
|
let result = $res
|
||||||
|
$result != null
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Server existence check (provider-agnostic)
|
||||||
|
export def mw_server_exists [
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
let res = (dispatch_provider_function $server.provider "server_exists" $server $error_exit )
|
||||||
|
if $res == null {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { false }
|
||||||
|
} else {
|
||||||
|
let result = $res
|
||||||
|
$result != null
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Server running status (provider-agnostic)
|
||||||
|
export def mw_server_is_running [
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
let res = (dispatch_provider_function $server.provider "server_is_running" $server $error_exit )
|
||||||
|
if $res == null {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { false }
|
||||||
|
} else {
|
||||||
|
let result = $res
|
||||||
|
$result != null
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Get IP (provider-agnostic)
|
||||||
|
export def mw_get_ip [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
ip_type: string
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> string {
|
||||||
|
let res = (dispatch_provider_function $server.provider "get_ip" $settings $server $ip_type $error_exit )
|
||||||
|
if $res == null {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { "" }
|
||||||
|
} else {
|
||||||
|
let result = $res
|
||||||
|
if $result != null {
|
||||||
|
$result | str trim
|
||||||
|
} else {
|
||||||
|
""
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Multi-provider infrastructure operations
|
||||||
|
export def mw_servers_ips [
|
||||||
|
settings: record
|
||||||
|
data: list
|
||||||
|
prov?: string
|
||||||
|
serverpos?: int
|
||||||
|
]: nothing -> list {
|
||||||
|
mut result = []
|
||||||
|
|
||||||
|
for srv in ($data | enumerate) {
|
||||||
|
let settings_server = ($settings.data.servers | where {|it| $it.hostname == $srv.item.hostname})
|
||||||
|
if ($settings_server | length) == 0 { continue }
|
||||||
|
|
||||||
|
let provider = ($settings_server | get -o 0 | get -o provider | default "")
|
||||||
|
if $prov != null and $provider != $prov { continue }
|
||||||
|
if $serverpos != null and $serverpos != $srv.index { continue }
|
||||||
|
|
||||||
|
let res = (dispatch_provider_function $provider "servers_ips" $settings [$srv.item] $prov $serverpos )
|
||||||
|
if $res == null {
|
||||||
|
provider_undefined { provider: $provider, hostname: $srv.item.hostname }
|
||||||
|
} else {
|
||||||
|
let provider_result = $res
|
||||||
|
if $provider_result != null {
|
||||||
|
$result = ($result | append $provider_result)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
$result
|
||||||
|
}
|
||||||
|
|
||||||
|
# Multi-provider server deletion
|
||||||
|
export def mw_delete_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
keep_storage: bool
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
let zone = $server.zone? | default ""
|
||||||
|
|
||||||
|
# Show provider-specific server message
|
||||||
|
let msg_result = (dispatch_provider_function $server.provider "on_prov_server" $server )
|
||||||
|
if ($msg_result != null) {
|
||||||
|
print $msg_result
|
||||||
|
}
|
||||||
|
|
||||||
|
# Delete server
|
||||||
|
let res = (dispatch_provider_function $server.provider "delete_server" $settings $server $keep_storage $error_exit )
|
||||||
|
if $res == null {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { false }
|
||||||
|
} else {
|
||||||
|
let result = $res
|
||||||
|
$result != null
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Load infrastructure servers info (provider-agnostic)
|
||||||
|
export def mw_load_infra_servers_info [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> record {
|
||||||
|
let res = (dispatch_provider_function $server.provider "load_infra_servers_info" $settings $server $error_exit)
|
||||||
|
if $res == null {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { {} }
|
||||||
|
} else {
|
||||||
|
$res
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Load infrastructure storages info (provider-agnostic)
|
||||||
|
export def mw_load_infra_storages_info [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> record {
|
||||||
|
let res = (dispatch_provider_function $server.provider "load_infra_storages_info" $settings $server $error_exit)
|
||||||
|
if $res == null {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { {} }
|
||||||
|
} else {
|
||||||
|
$res
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Get infrastructure item for server (provider-agnostic)
|
||||||
|
export def mw_get_infra_item [
|
||||||
|
server: record
|
||||||
|
settings: record
|
||||||
|
cloud_data: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> record {
|
||||||
|
let res = (dispatch_provider_function $server.provider "get_infra_item" $server $settings $cloud_data $error_exit)
|
||||||
|
if $res == null {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { {} }
|
||||||
|
} else {
|
||||||
|
$res
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Get infrastructure price (provider-agnostic)
|
||||||
|
export def mw_get_infra_price [
|
||||||
|
server: record
|
||||||
|
data: record
|
||||||
|
key: string
|
||||||
|
error_exit: bool
|
||||||
|
price_col?: string
|
||||||
|
]: nothing -> any {
|
||||||
|
let res = (dispatch_provider_function $server.provider "get_infra_price" $server $data $key $error_exit $price_col)
|
||||||
|
if $res == null {
|
||||||
|
return (if $error_exit { (exit) } else { 0.0 })
|
||||||
|
}
|
||||||
|
# Convert result to float (wrapper scripts return strings)
|
||||||
|
let res_type = ($res | describe)
|
||||||
|
if ($res_type | str starts-with "float") or ($res_type | str starts-with "int") {
|
||||||
|
$res
|
||||||
|
} else if ($res_type | str starts-with "string") {
|
||||||
|
# Special handling for "unit" key which returns string like "0.0104 Hrs"
|
||||||
|
if $key == "unit" {
|
||||||
|
$res
|
||||||
|
} else {
|
||||||
|
# Parse string as float (wrapper scripts serialize numbers as strings)
|
||||||
|
# Check if string can be converted to float
|
||||||
|
let float_result = (do --ignore-errors { $res | into float })
|
||||||
|
if ($float_result | is-empty) {
|
||||||
|
0.0
|
||||||
|
} else {
|
||||||
|
$float_result
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
# Return 0.0 for any other non-numeric type
|
||||||
|
0.0
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Get infrastructure storage (provider-agnostic)
|
||||||
|
export def mw_get_infra_storage [
|
||||||
|
server: record
|
||||||
|
settings: record
|
||||||
|
cloud_data: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> record {
|
||||||
|
let res = (dispatch_provider_function $server.provider "get_infra_storage" $server $settings $cloud_data $error_exit)
|
||||||
|
if $res == null {
|
||||||
|
if $error_exit { exit } else { {} }
|
||||||
|
} else {
|
||||||
|
$res
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# === ENHANCED MULTI-PROVIDER OPERATIONS ===
|
||||||
|
|
||||||
|
# Deploy multi-provider infrastructure (simplified)
|
||||||
|
export def mw_deploy_multi_provider_infra [
|
||||||
|
settings: record
|
||||||
|
deployment_plan: record
|
||||||
|
]: nothing -> record {
|
||||||
|
log-section "Starting multi-provider deployment" "middleware"
|
||||||
|
|
||||||
|
# Group servers by provider
|
||||||
|
let provider_groups = ($settings.data.servers | group-by provider)
|
||||||
|
let providers_used = ($provider_groups | columns)
|
||||||
|
|
||||||
|
log-info $"Will deploy to providers: ($providers_used | str join ', ')" "middleware"
|
||||||
|
|
||||||
|
# Return basic deployment info
|
||||||
|
{
|
||||||
|
providers_used: $providers_used
|
||||||
|
total_servers: ($settings.data.servers | length)
|
||||||
|
deployment_plan: $deployment_plan
|
||||||
|
status: "planned"
|
||||||
|
timestamp: (date now)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Get provider status with capabilities
|
||||||
|
export def mw_provider_status [
|
||||||
|
--verbose
|
||||||
|
]: nothing -> table {
|
||||||
|
if not ($env.PROVIDER_REGISTRY_INITIALIZED? | default false) {
|
||||||
|
init-middleware | ignore
|
||||||
|
}
|
||||||
|
|
||||||
|
let providers = (list-providers --available-only)
|
||||||
|
|
||||||
|
$providers | each {|provider|
|
||||||
|
let allowed = (is_provider_allowed $provider.name)
|
||||||
|
let capabilities = (get-provider-capabilities-for $provider.name)
|
||||||
|
|
||||||
|
let basic_info = {
|
||||||
|
name: $provider.name
|
||||||
|
type: ($provider.details? | get -o type | default "extension")
|
||||||
|
available: ($provider.details? | get -o available | default true)
|
||||||
|
loaded: ($provider.details? | get -o loaded | default false)
|
||||||
|
profile_allowed: $allowed
|
||||||
|
status: (if $allowed { "✅ Available" } else { "🛑 Blocked by profile" })
|
||||||
|
}
|
||||||
|
|
||||||
|
if $verbose {
|
||||||
|
$basic_info | merge {
|
||||||
|
capabilities: $provider.capabilities
|
||||||
|
server_management: ($capabilities.server_management? | default false)
|
||||||
|
multi_region: ($capabilities.multi_region? | default false)
|
||||||
|
auto_scaling: ($capabilities.auto_scaling? | default false)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
$basic_info
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Get deployment recommendations for multi-provider setup
|
||||||
|
export def mw_suggest_deployment_strategy [
|
||||||
|
requirements: record
|
||||||
|
]: nothing -> record {
|
||||||
|
log-info "Analyzing requirements for deployment strategy" "middleware"
|
||||||
|
|
||||||
|
let available_providers = (list-providers --available-only)
|
||||||
|
|
||||||
|
mut recommendations = {
|
||||||
|
strategy: "single-provider" # default
|
||||||
|
primary_provider: ""
|
||||||
|
secondary_providers: []
|
||||||
|
rationale: []
|
||||||
|
}
|
||||||
|
|
||||||
|
# Analyze requirements and suggest strategy
|
||||||
|
if ($requirements.regions? | default [] | length) > 1 {
|
||||||
|
$recommendations.strategy = "multi-provider"
|
||||||
|
$recommendations.rationale = ($recommendations.rationale | append "Multi-region requirement detected")
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($requirements.high_availability? | default false) {
|
||||||
|
$recommendations.strategy = "multi-provider"
|
||||||
|
$recommendations.rationale = ($recommendations.rationale | append "High availability requirement")
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($requirements.cost_optimization? | default false) {
|
||||||
|
$recommendations.rationale = ($recommendations.rationale | append "Cost optimization: consider mixing providers")
|
||||||
|
}
|
||||||
|
|
||||||
|
# Select primary provider based on capabilities
|
||||||
|
let suitable_providers = ($available_providers | where {|p|
|
||||||
|
let caps = (get-provider-capabilities-for $p.name)
|
||||||
|
($caps.server_management? | default false) == true
|
||||||
|
})
|
||||||
|
|
||||||
|
if ($suitable_providers | length) > 0 {
|
||||||
|
$recommendations.primary_provider = ($suitable_providers | get 0 | get name)
|
||||||
|
|
||||||
|
if $recommendations.strategy == "multi-provider" {
|
||||||
|
$recommendations.secondary_providers = ($suitable_providers | skip 1 | get name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
log-info $"Recommended strategy: ($recommendations.strategy)" "middleware"
|
||||||
|
$recommendations
|
||||||
|
}
|
||||||
|
|
||||||
|
# Initialize middleware when loaded
|
||||||
|
export-env {
|
||||||
|
# This will be set when middleware functions are first called
|
||||||
|
}
|
||||||
604
providers/prov_lib/middleware.nu-e
Normal file
604
providers/prov_lib/middleware.nu-e
Normal file
@ -0,0 +1,604 @@
|
|||||||
|
# CNPROV middleware generated by 'make_middleware' on 2024-04-08_21:24:42
|
||||||
|
use ../../../core/nulib/lib_provisioning/config/accessor.nu *
|
||||||
|
use ../aws/nulib/aws/env.nu
|
||||||
|
use ../aws/nulib/aws/servers.nu *
|
||||||
|
use ../aws/nulib/aws/cache.nu *
|
||||||
|
use ../aws/nulib/aws/prices.nu *
|
||||||
|
use ../local/nulib/local/env.nu
|
||||||
|
use ../local/nulib/local/servers.nu *
|
||||||
|
use ../upcloud/nulib/upcloud/env.nu
|
||||||
|
use ../upcloud/nulib/upcloud/servers.nu *
|
||||||
|
use ../upcloud/nulib/upcloud/cache.nu *
|
||||||
|
use ../upcloud/nulib/upcloud/prices.nu *
|
||||||
|
def provider_undefined [
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
#use defs/lists.nu providers_list
|
||||||
|
let str_providers_list = (providers_list "selection" | each { |it| $it.name} | str join " ")
|
||||||
|
print ($"(_ansi blue_bold)($server.hostname)(_ansi reset) with provider " +
|
||||||
|
$"(_ansi green_bold)($server.provider)(_ansi reset) ($server.zone) ")
|
||||||
|
let text = ( $"expected to be one of available providers [(_ansi green_italic)($str_providers_list)(_ansi reset)], " +
|
||||||
|
$"got (_ansi green_bold)($server.provider)(_ansi reset)")
|
||||||
|
print $"Error 🛑 provider ($text)"
|
||||||
|
}
|
||||||
|
export def mw_query_servers [
|
||||||
|
settings: record
|
||||||
|
find?: string
|
||||||
|
cols?: string
|
||||||
|
--prov: string
|
||||||
|
--serverpos: int
|
||||||
|
] {
|
||||||
|
let str_find = if $find != null { $find } else { "" }
|
||||||
|
let str_cols = if $cols != null { $cols } else { "" }
|
||||||
|
$settings.data.servers | enumerate | each { |it|
|
||||||
|
#let res = for idx in ..($settings.data.servers | length) {
|
||||||
|
#let srv = ($settings.data.servers | get -o $idx)
|
||||||
|
if $prov == null or $it.item.provider == $prov {
|
||||||
|
if $serverpos == null or $serverpos == $it.index {
|
||||||
|
let res = match $it.item.provider {
|
||||||
|
"aws" => {
|
||||||
|
(aws_query_servers $str_find $str_cols)
|
||||||
|
},
|
||||||
|
"local" => {
|
||||||
|
(local_query_servers $str_find $str_cols)
|
||||||
|
},
|
||||||
|
"upcloud" => {
|
||||||
|
(upcloud_query_servers $str_find $str_cols)
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
provider_undefined $it.item
|
||||||
|
[]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if ($res | length) > 0 {
|
||||||
|
let result = if $str_find != "" {
|
||||||
|
$res | find $str_find
|
||||||
|
} else {
|
||||||
|
$res
|
||||||
|
}
|
||||||
|
if $str_cols != "" {
|
||||||
|
let field_list = ($str_cols | split row ",")
|
||||||
|
($result | select -o $field_list)
|
||||||
|
} else {
|
||||||
|
$result
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
# $list | append $srv
|
||||||
|
} | flatten
|
||||||
|
}
|
||||||
|
export def mw_servers_ips [
|
||||||
|
settings: record
|
||||||
|
data: list
|
||||||
|
prov?: string
|
||||||
|
serverpos?: int
|
||||||
|
]: nothing -> list {
|
||||||
|
mut index = -1
|
||||||
|
mut result = []
|
||||||
|
for srv in $data {
|
||||||
|
$index += 1
|
||||||
|
let settings_server = ($settings.data.servers | where {|it| $it.hostname == $srv.hostname})
|
||||||
|
if ($settings_server | length) == 0 { continue }
|
||||||
|
let provider = ($settings_server | get -o 0 | get -o provider | default "")
|
||||||
|
if $prov != null and $provider != $prov { continue }
|
||||||
|
if $serverpos != null and $serverpos != $index { continue }
|
||||||
|
match $provider {
|
||||||
|
"aws" => {
|
||||||
|
if $srv.ip_addresses? != null {
|
||||||
|
$result = ($result | append ($srv.ip_addresses? |
|
||||||
|
each {|it| { hostname: $srv.hostname, ip: $it.address, access: $it.access, family: $it.family }} |
|
||||||
|
flatten
|
||||||
|
))
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"local" => {
|
||||||
|
if $srv.ip_addresses? != null {
|
||||||
|
$result = ($result | append ($srv.ip_addresses? |
|
||||||
|
each {|it| { hostname: $srv.hostname, ip: $it.address, access: $it.access, family: $it.family }} |
|
||||||
|
flatten
|
||||||
|
))
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"upcloud" => {
|
||||||
|
if $srv.ip_addresses? != null {
|
||||||
|
$result = ($result | append ($srv.ip_addresses? |
|
||||||
|
each {|it| { hostname: $srv.hostname, ip: $it.address, access: $it.access, family: $it.family }} |
|
||||||
|
flatten
|
||||||
|
))
|
||||||
|
}
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
provider_undefined $srv.provider
|
||||||
|
[]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
$result
|
||||||
|
}
|
||||||
|
export def mw_server_info [
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
find?: string
|
||||||
|
cols?: string
|
||||||
|
]: nothing -> record {
|
||||||
|
let str_find = if $find != null { $find } else { "" }
|
||||||
|
let str_cols = if $cols != null { $cols } else { "" }
|
||||||
|
let res = match $server.provider {
|
||||||
|
"aws" => {
|
||||||
|
(aws_server_info $server $check)
|
||||||
|
},
|
||||||
|
"local" => {
|
||||||
|
(local_server_info $server $check)
|
||||||
|
},
|
||||||
|
"upcloud" => {
|
||||||
|
(upcloud_server_info $server $check)
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server.hostname
|
||||||
|
[]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if ($res | describe | str starts-with "record") and $res.hostname? != null {
|
||||||
|
let result = if $str_find != "" {
|
||||||
|
$res | find $str_find
|
||||||
|
} else {
|
||||||
|
$res
|
||||||
|
}
|
||||||
|
let info = if $str_cols != "" {
|
||||||
|
let field_list = ($str_cols | split row ",")
|
||||||
|
($result | select -o $field_list)
|
||||||
|
} else {
|
||||||
|
($result)
|
||||||
|
}
|
||||||
|
let priv = match $server.provider {
|
||||||
|
"aws" => {
|
||||||
|
($info | get -o private_ips | default [] | each {|it| ($it | select Description PrivateIpAddress VpcId SubnetId Groups) })
|
||||||
|
},
|
||||||
|
_ => ($info | get -o priv | default [])
|
||||||
|
}
|
||||||
|
let full_info = if ($priv | length) > 0 {
|
||||||
|
($info | merge { private_ips: $priv })
|
||||||
|
} else {
|
||||||
|
$info
|
||||||
|
}
|
||||||
|
let out = (get-provisioning-out)
|
||||||
|
if ($out | is-empty) {
|
||||||
|
print ($full_info | table -e)
|
||||||
|
}
|
||||||
|
if (not $check) {
|
||||||
|
($full_info | table -e)
|
||||||
|
}
|
||||||
|
$full_info
|
||||||
|
} else {
|
||||||
|
$res
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def mw_servers_info [
|
||||||
|
settings: record
|
||||||
|
find?: string
|
||||||
|
cols?: string
|
||||||
|
--prov: string
|
||||||
|
--serverpos: int
|
||||||
|
--check
|
||||||
|
]: nothing -> list {
|
||||||
|
let str_find = if $find != null { $find } else { "" }
|
||||||
|
let str_cols = if $cols != null { $cols } else { "" }
|
||||||
|
|
||||||
|
$settings.data.servers | enumerate | each { |it|
|
||||||
|
if $prov == null or $it.item.provider == $prov {
|
||||||
|
if $serverpos == null or $serverpos == $it.index {
|
||||||
|
mw_server_info $it.item $check $str_find $str_cols
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def mw_create_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
let zone = $server.zone? | default ""
|
||||||
|
let res = match $server.provider {
|
||||||
|
"aws" => {
|
||||||
|
print (aws_on_prov_server $server)
|
||||||
|
(aws_check_server_requirements $settings $server $check)
|
||||||
|
},
|
||||||
|
"local" => {
|
||||||
|
print (local_on_prov_server $server)
|
||||||
|
(local_check_server_requirements $settings $server $check)
|
||||||
|
},
|
||||||
|
"upcloud" => {
|
||||||
|
print (upcloud_on_prov_server $server)
|
||||||
|
(upcloud_check_server_requirements $settings $server $check)
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { false }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if not $res {
|
||||||
|
(throw-error $"🛑 ($server.provider) check requirements error"
|
||||||
|
$"for server ($server.hostname)"
|
||||||
|
"create_server" --span (metadata $server.provider).span)
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
print ($"Create (_ansi blue_bold)($server.hostname)(_ansi reset) with provider " +
|
||||||
|
$"(_ansi green_bold)($server.provider)(_ansi reset) ($zone) ")
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
export def mw_server_state [
|
||||||
|
server: record
|
||||||
|
new_state: string
|
||||||
|
error_exit: bool
|
||||||
|
wait: bool
|
||||||
|
settings: record
|
||||||
|
]: nothing -> bool {
|
||||||
|
match $server.provider {
|
||||||
|
"aws" => {
|
||||||
|
(aws_server_state $server $new_state $error_exit $wait $settings)
|
||||||
|
},
|
||||||
|
"local" => {
|
||||||
|
(local_server_state $server $new_state $error_exit $wait $settings)
|
||||||
|
},
|
||||||
|
"upcloud" => {
|
||||||
|
(upcloud_server_state $server $new_state $error_exit $wait $settings)
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { return false }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
true
|
||||||
|
}
|
||||||
|
export def mw_server_exists [
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
match $server.provider {
|
||||||
|
"aws" => {
|
||||||
|
(aws_server_exists $server $error_exit)
|
||||||
|
},
|
||||||
|
"local" => {
|
||||||
|
(local_server_exists $server $error_exit)
|
||||||
|
},
|
||||||
|
"upcloud" => {
|
||||||
|
(upcloud_server_exists $server $error_exit)
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { false }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def mw_server_is_running [
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
match $server.provider {
|
||||||
|
"aws" => {
|
||||||
|
(aws_server_is_running $server $error_exit)
|
||||||
|
},
|
||||||
|
"local" => {
|
||||||
|
(local_server_is_running $server $error_exit)
|
||||||
|
},
|
||||||
|
"upcloud" => {
|
||||||
|
(upcloud_server_is_running $server $error_exit)
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { false }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def mw_get_ip [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
ip_type: string
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> string {
|
||||||
|
let use_type = match $ip_type {
|
||||||
|
"$network_public_ip" => "public",
|
||||||
|
"$network_private_ip" => "private",
|
||||||
|
_ => $ip_type
|
||||||
|
}
|
||||||
|
let res = match $server.provider {
|
||||||
|
"aws" => {
|
||||||
|
(aws_server [ "get_ip", $use_type ] --server $server --settings $settings)
|
||||||
|
},
|
||||||
|
"local" => {
|
||||||
|
(local_server [ "get_ip", $use_type ] --server $server --settings $settings)
|
||||||
|
},
|
||||||
|
"upcloud" => {
|
||||||
|
(upcloud_server [ "get_ip", $use_type ] --server $server --settings $settings)
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { "" }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
$"($res)" | str trim
|
||||||
|
}
|
||||||
|
export def mw_post_create_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
match $server.provider {
|
||||||
|
"aws" => {
|
||||||
|
(aws_post_create_server $settings $server $check)
|
||||||
|
},
|
||||||
|
"local" => {
|
||||||
|
(local_post_create_server $settings $server $check)
|
||||||
|
},
|
||||||
|
"upcloud" => {
|
||||||
|
(upcloud_post_create_server $settings $server $check)
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server.provider
|
||||||
|
true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def mw_modify_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
new_values: list
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
match $server.provider {
|
||||||
|
"aws" => {
|
||||||
|
(aws_modify_server $settings $server $new_values $error_exit)
|
||||||
|
},
|
||||||
|
"local" => {
|
||||||
|
(local_modify_server $settings $server $new_values $error_exit)
|
||||||
|
},
|
||||||
|
"upcloud" => {
|
||||||
|
(upcloud_modify_server $settings $server $new_values $error_exit)
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server.provider
|
||||||
|
true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def mw_delete_server_storage [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
let zone = $server.zone? | default ""
|
||||||
|
match $server.provider {
|
||||||
|
"aws" => {
|
||||||
|
print (aws_on_prov_server $server)
|
||||||
|
(aws_delete_server_storage $settings $server $error_exit)
|
||||||
|
},
|
||||||
|
"local" => {
|
||||||
|
print (local_on_prov_server $server)
|
||||||
|
(local_delete_server_storage $settings $server $error_exit)
|
||||||
|
},
|
||||||
|
"upcloud" => {
|
||||||
|
print (upcloud_on_prov_server $server)
|
||||||
|
(upcloud_delete_server_storage $settings $server $error_exit)
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { false }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def mw_delete_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
keep_storage: bool
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
let zone = $server.zone? | default ""
|
||||||
|
match $server.provider {
|
||||||
|
"aws" => {
|
||||||
|
print (aws_on_prov_server $server)
|
||||||
|
(aws_delete_server $settings $server $keep_storage $error_exit)
|
||||||
|
},
|
||||||
|
"local" => {
|
||||||
|
print (local_on_prov_server $server)
|
||||||
|
(local_delete_server $settings $server $keep_storage $error_exit)
|
||||||
|
},
|
||||||
|
"upcloud" => {
|
||||||
|
print (upcloud_on_prov_server $server)
|
||||||
|
(upcloud_delete_server $settings $server $keep_storage $error_exit)
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { false }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def mw_load_infra_servers_info [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> record {
|
||||||
|
match $server.provider {
|
||||||
|
"aws" => {
|
||||||
|
(aws_load_infra_servers_info $settings $server $error_exit)
|
||||||
|
},
|
||||||
|
"upcloud" => {
|
||||||
|
(upcloud_load_infra_servers_info $settings $server $error_exit)
|
||||||
|
},
|
||||||
|
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { {} }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def mw_load_infra_storages_info [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> record {
|
||||||
|
match $server.provider {
|
||||||
|
"aws" => {
|
||||||
|
(aws_load_infra_storages_info $settings $server $error_exit)
|
||||||
|
},
|
||||||
|
"upcloud" => {
|
||||||
|
(upcloud_load_infra_storages_info $settings $server $error_exit)
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { {} }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def mw_get_infra_storage [
|
||||||
|
server: record
|
||||||
|
settings: record
|
||||||
|
cloud_data: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> list {
|
||||||
|
match $server.provider {
|
||||||
|
"aws" => {
|
||||||
|
(aws_get_item_for_storage $server $settings $cloud_data)
|
||||||
|
},
|
||||||
|
"upcloud" => {
|
||||||
|
(upcloud_get_item_for_storage $server $settings $cloud_data)
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { [] }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def mw_get_infra_item [
|
||||||
|
server: record
|
||||||
|
settings: record
|
||||||
|
cloud_data: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> record {
|
||||||
|
match $server.provider {
|
||||||
|
"aws" => {
|
||||||
|
(aws_get_item_for_server $server $settings $cloud_data)
|
||||||
|
},
|
||||||
|
"upcloud" => {
|
||||||
|
(upcloud_get_item_for_server $server $settings $cloud_data)
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { return {} }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def mw_get_infra_price [
|
||||||
|
server: record
|
||||||
|
data: record
|
||||||
|
key: string
|
||||||
|
error_exit: bool
|
||||||
|
price_col?: string
|
||||||
|
]: nothing -> float {
|
||||||
|
if ($data | get -o item | is-empty) { return {} }
|
||||||
|
match $server.provider {
|
||||||
|
"aws" => {
|
||||||
|
(aws_get_price $data $key $price_col)
|
||||||
|
},
|
||||||
|
"upcloud" => {
|
||||||
|
(upcloud_get_price $data $key $price_col)
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { return 0 }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def mw_start_cache_info [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
]: nothing -> nothing {
|
||||||
|
match $server.provider {
|
||||||
|
"aws" => {
|
||||||
|
(aws_start_cache_info $settings $server)
|
||||||
|
},
|
||||||
|
"upcloud" => {
|
||||||
|
(upcloud_start_cache_info $settings $server)
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def mw_create_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> nothing {
|
||||||
|
match $server.provider {
|
||||||
|
"aws" => {
|
||||||
|
(aws_create_cache $settings $server $error_exit)
|
||||||
|
},
|
||||||
|
"upcloud" => {
|
||||||
|
(upcloud_create_cache $settings $server $error_exit)
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { return 0 }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def mw_read_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> nothing {
|
||||||
|
match $server.provider {
|
||||||
|
"aws" => {
|
||||||
|
(aws_read_cache $settings $server $error_exit)
|
||||||
|
},
|
||||||
|
"upcloud" => {
|
||||||
|
(upcloud_read_cache $settings $server $error_exit)
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { return }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def mw_clean_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> nothing {
|
||||||
|
match $server.provider {
|
||||||
|
"aws" => {
|
||||||
|
(aws_clean_cache $settings $server $error_exit)
|
||||||
|
},
|
||||||
|
"upcloud" => {
|
||||||
|
(upcloud_clean_cache $settings $server $error_exit)
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { return }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
export def mw_ip_from_cache [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> nothing {
|
||||||
|
match $server.provider {
|
||||||
|
"aws" => {
|
||||||
|
(aws_ip_from_cache $settings $server $error_exit)
|
||||||
|
},
|
||||||
|
"upcloud" => {
|
||||||
|
(upcloud_ip_from_cache $settings $server $error_exit)
|
||||||
|
},
|
||||||
|
"local" => {
|
||||||
|
($server | get -o network_public_ip | default "")
|
||||||
|
#(local_ip_from_cache $settings $server $error_exit)
|
||||||
|
},
|
||||||
|
_ => {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { return }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
460
providers/prov_lib/middleware_provider_agnostic.nu
Normal file
460
providers/prov_lib/middleware_provider_agnostic.nu
Normal file
@ -0,0 +1,460 @@
|
|||||||
|
# Provider-Agnostic Middleware
|
||||||
|
# Uses dynamic provider loading and interface-based dispatch
|
||||||
|
# Supports multi-provider infrastructure deployments
|
||||||
|
|
||||||
|
use ../../../core/nulib/lib_provisioning/config/accessor.nu *
|
||||||
|
use ../../../core/nulib/lib_provisioning/extensions/profiles.nu *
|
||||||
|
use ../../../core/nulib/lib_provisioning/providers/registry.nu *
|
||||||
|
use ../../../core/nulib/lib_provisioning/providers/loader.nu *
|
||||||
|
use ../../../core/nulib/lib_provisioning/providers/interface.nu *
|
||||||
|
use ../../../core/nulib/lib_provisioning/utils/logging.nu *
|
||||||
|
|
||||||
|
# Initialize middleware
|
||||||
|
export def init-middleware []: nothing -> nothing {
|
||||||
|
init-provider-registry
|
||||||
|
log-info "Provider-agnostic middleware initialized" "middleware"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Check if provider is allowed by profile (safer version)
|
||||||
|
def is_provider_allowed [provider_name: string]: nothing -> bool {
|
||||||
|
try {
|
||||||
|
let profile = (load-profile)
|
||||||
|
|
||||||
|
# If not restricted, allow everything
|
||||||
|
if not $profile.restricted {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
# Check if provider is explicitly blocked
|
||||||
|
if ($profile.blocked.providers | where {|p| $p == $provider_name} | length) > 0 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
# If allowed list is empty, allow all (except blocked)
|
||||||
|
if ($profile.allowed.providers | length) == 0 {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
# Check if provider is in allowed list
|
||||||
|
($profile.allowed.providers | where {|p| $p == $provider_name} | length) > 0
|
||||||
|
|
||||||
|
} catch {
|
||||||
|
# If profile loading fails, default to allow
|
||||||
|
true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
def provider_undefined [
|
||||||
|
server: record
|
||||||
|
] {
|
||||||
|
let available_providers = (list-providers --available-only | get name | str join ", ")
|
||||||
|
log-error $"Provider ($server.provider) not found or not available" "middleware"
|
||||||
|
print $"(_ansi blue_bold)($server.hostname)(_ansi reset) with provider " +
|
||||||
|
$"(_ansi green_bold)($server.provider)(_ansi reset) ($server.zone?) "
|
||||||
|
print $"Error 🛑 provider expected to be one of available providers [(_ansi green_italic)($available_providers)(_ansi reset)], " +
|
||||||
|
$"got (_ansi green_bold)($server.provider)(_ansi reset)"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Dynamic provider dispatch helper
|
||||||
|
def dispatch_provider_function [
|
||||||
|
provider_name: string
|
||||||
|
function_name: string
|
||||||
|
...args
|
||||||
|
]: nothing -> any {
|
||||||
|
# Check if provider is allowed
|
||||||
|
if not (is_provider_allowed $provider_name) {
|
||||||
|
log-error $"Provider ($provider_name) blocked by profile" "middleware"
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
|
||||||
|
# Load provider if not already loaded
|
||||||
|
let provider = (get-provider $provider_name)
|
||||||
|
if ($provider | is-empty) {
|
||||||
|
log-error $"Failed to load provider ($provider_name)" "middleware"
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
|
||||||
|
# Call provider function
|
||||||
|
call-provider-function $provider_name $function_name ...$args
|
||||||
|
}
|
||||||
|
|
||||||
|
# === CORE MIDDLEWARE FUNCTIONS ===
|
||||||
|
|
||||||
|
# Query servers (supports multi-provider)
|
||||||
|
export def mw_query_servers [
|
||||||
|
settings: record
|
||||||
|
find?: string
|
||||||
|
cols?: string
|
||||||
|
--prov: string
|
||||||
|
--serverpos: int
|
||||||
|
]: nothing -> list {
|
||||||
|
if not ($env.PROVIDER_REGISTRY_INITIALIZED? | default false) {
|
||||||
|
init-middleware | ignore
|
||||||
|
}
|
||||||
|
|
||||||
|
let str_find = if $find != null { $find } else { "" }
|
||||||
|
let str_cols = if $cols != null { $cols } else { "" }
|
||||||
|
|
||||||
|
$settings.data.servers | enumerate | each { |it|
|
||||||
|
if $prov == null or $it.item.provider == $prov {
|
||||||
|
if $serverpos == null or $serverpos == $it.index {
|
||||||
|
let provider_name = $it.item.provider
|
||||||
|
|
||||||
|
# Dynamic provider dispatch
|
||||||
|
let res = try {
|
||||||
|
dispatch_provider_function $provider_name "query_servers" $str_find $str_cols
|
||||||
|
} catch {
|
||||||
|
provider_undefined $it.item
|
||||||
|
[]
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($res != null) and ($res | length) > 0 {
|
||||||
|
let result = if $str_find != "" {
|
||||||
|
$res | find $str_find
|
||||||
|
} else {
|
||||||
|
$res
|
||||||
|
}
|
||||||
|
if $str_cols != "" {
|
||||||
|
let field_list = ($str_cols | split row ",")
|
||||||
|
($result | select -o $field_list)
|
||||||
|
} else {
|
||||||
|
$result
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
[]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} | flatten
|
||||||
|
}
|
||||||
|
|
||||||
|
# Server information (provider-agnostic)
|
||||||
|
export def mw_server_info [
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
find?: string
|
||||||
|
cols?: string
|
||||||
|
]: nothing -> record {
|
||||||
|
let str_find = if $find != null { $find } else { "" }
|
||||||
|
let str_cols = if $cols != null { $cols } else { "" }
|
||||||
|
|
||||||
|
let res = try {
|
||||||
|
dispatch_provider_function $server.provider "server_info" $server $check $str_find $str_cols
|
||||||
|
} catch {
|
||||||
|
provider_undefined $server
|
||||||
|
{}
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($res | describe | str starts-with "record") and $res.hostname? != null {
|
||||||
|
let result = if $str_find != "" {
|
||||||
|
$res | find $str_find
|
||||||
|
} else {
|
||||||
|
$res
|
||||||
|
}
|
||||||
|
let info = if $str_cols != "" {
|
||||||
|
let field_list = ($str_cols | split row ",")
|
||||||
|
($result | select -o $field_list)
|
||||||
|
} else {
|
||||||
|
$result
|
||||||
|
}
|
||||||
|
|
||||||
|
# Handle provider-specific private IP formats
|
||||||
|
let priv = if $server.provider == "aws" {
|
||||||
|
($info | get -o private_ips | default [] | each {|it| ($it | select Description PrivateIpAddress VpcId SubnetId Groups) })
|
||||||
|
} else {
|
||||||
|
($info | get -o priv | default [])
|
||||||
|
}
|
||||||
|
|
||||||
|
let full_info = if ($priv | length) > 0 {
|
||||||
|
($info | merge { private_ips: $priv })
|
||||||
|
} else {
|
||||||
|
$info
|
||||||
|
}
|
||||||
|
|
||||||
|
let out = (get-provisioning-out)
|
||||||
|
if ($out | is-empty) {
|
||||||
|
print ($full_info | table -e)
|
||||||
|
}
|
||||||
|
if (not $check) {
|
||||||
|
($full_info | table -e)
|
||||||
|
}
|
||||||
|
$full_info
|
||||||
|
} else {
|
||||||
|
$res
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Server creation (provider-agnostic with profile checking)
|
||||||
|
export def mw_create_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
check: bool
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
# Check if provider is allowed by profile
|
||||||
|
if not (is_provider_allowed $server.provider) {
|
||||||
|
log-error $"Provider ($server.provider) blocked by current profile" "middleware"
|
||||||
|
if $error_exit { exit } else { return false }
|
||||||
|
}
|
||||||
|
|
||||||
|
let zone = $server.zone? | default ""
|
||||||
|
|
||||||
|
# Dynamic provider dispatch for requirements check
|
||||||
|
let res = try {
|
||||||
|
dispatch_provider_function $server.provider "check_server_requirements" $settings $server $check
|
||||||
|
} catch {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { false }
|
||||||
|
}
|
||||||
|
|
||||||
|
if not $res {
|
||||||
|
log-error $"($server.provider) check requirements error for server ($server.hostname)" "middleware"
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
# Show provider-specific server creation message
|
||||||
|
try {
|
||||||
|
let msg = (dispatch_provider_function $server.provider "on_prov_server" $server)
|
||||||
|
if $msg != null { print $msg }
|
||||||
|
} catch { }
|
||||||
|
|
||||||
|
print ($"Create (_ansi blue_bold)($server.hostname)(_ansi reset) with provider " +
|
||||||
|
$"(_ansi green_bold)($server.provider)(_ansi reset) ($zone) ")
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
# Server state management (provider-agnostic)
|
||||||
|
export def mw_server_state [
|
||||||
|
server: record
|
||||||
|
new_state: string
|
||||||
|
error_exit: bool
|
||||||
|
wait: bool
|
||||||
|
settings: record
|
||||||
|
]: nothing -> bool {
|
||||||
|
# Check if provider is allowed by profile
|
||||||
|
if not (is_provider_allowed $server.provider) {
|
||||||
|
log-error $"Provider ($server.provider) blocked by current profile" "middleware"
|
||||||
|
if $error_exit { exit } else { return false }
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
let result = (dispatch_provider_function $server.provider "server_state" $server $new_state $error_exit $wait $settings)
|
||||||
|
$result != null
|
||||||
|
} catch {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { return false }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Server existence check (provider-agnostic)
|
||||||
|
export def mw_server_exists [
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
try {
|
||||||
|
let result = (dispatch_provider_function $server.provider "server_exists" $server $error_exit)
|
||||||
|
$result != null
|
||||||
|
} catch {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { false }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Server running status (provider-agnostic)
|
||||||
|
export def mw_server_is_running [
|
||||||
|
server: record
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
try {
|
||||||
|
let result = (dispatch_provider_function $server.provider "server_is_running" $server $error_exit)
|
||||||
|
$result != null
|
||||||
|
} catch {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { false }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Get IP (provider-agnostic)
|
||||||
|
export def mw_get_ip [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
ip_type: string
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> string {
|
||||||
|
try {
|
||||||
|
let result = (dispatch_provider_function $server.provider "get_ip" $settings $server $ip_type $error_exit)
|
||||||
|
if $result != null {
|
||||||
|
$result | str trim
|
||||||
|
} else {
|
||||||
|
""
|
||||||
|
}
|
||||||
|
} catch {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { "" }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Multi-provider infrastructure operations
|
||||||
|
export def mw_servers_ips [
|
||||||
|
settings: record
|
||||||
|
data: list
|
||||||
|
prov?: string
|
||||||
|
serverpos?: int
|
||||||
|
]: nothing -> list {
|
||||||
|
mut result = []
|
||||||
|
|
||||||
|
for srv in ($data | enumerate) {
|
||||||
|
let settings_server = ($settings.data.servers | where {|it| $it.hostname == $srv.item.hostname})
|
||||||
|
if ($settings_server | length) == 0 { continue }
|
||||||
|
|
||||||
|
let provider = ($settings_server | get -o 0 | get -o provider | default "")
|
||||||
|
if $prov != null and $provider != $prov { continue }
|
||||||
|
if $serverpos != null and $serverpos != $srv.index { continue }
|
||||||
|
|
||||||
|
try {
|
||||||
|
let provider_result = (dispatch_provider_function $provider "servers_ips" $settings [$srv.item] $prov $serverpos)
|
||||||
|
if $provider_result != null {
|
||||||
|
$result = ($result | append $provider_result)
|
||||||
|
}
|
||||||
|
} catch {
|
||||||
|
provider_undefined { provider: $provider, hostname: $srv.item.hostname }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
$result
|
||||||
|
}
|
||||||
|
|
||||||
|
# Multi-provider server deletion
|
||||||
|
export def mw_delete_server [
|
||||||
|
settings: record
|
||||||
|
server: record
|
||||||
|
keep_storage: bool
|
||||||
|
error_exit: bool
|
||||||
|
]: nothing -> bool {
|
||||||
|
let zone = $server.zone? | default ""
|
||||||
|
|
||||||
|
try {
|
||||||
|
let msg = (dispatch_provider_function $server.provider "on_prov_server" $server)
|
||||||
|
if $msg != null { print $msg }
|
||||||
|
|
||||||
|
let result = (dispatch_provider_function $server.provider "delete_server" $settings $server $keep_storage $error_exit)
|
||||||
|
$result != null
|
||||||
|
} catch {
|
||||||
|
provider_undefined $server
|
||||||
|
if $error_exit { exit } else { false }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# === ENHANCED MULTI-PROVIDER OPERATIONS ===
|
||||||
|
|
||||||
|
# Deploy multi-provider infrastructure (simplified)
|
||||||
|
export def mw_deploy_multi_provider_infra [
|
||||||
|
settings: record
|
||||||
|
deployment_plan: record
|
||||||
|
]: nothing -> record {
|
||||||
|
log-section "Starting multi-provider deployment" "middleware"
|
||||||
|
|
||||||
|
# Group servers by provider
|
||||||
|
let provider_groups = ($settings.data.servers | group-by provider)
|
||||||
|
let providers_used = ($provider_groups | columns)
|
||||||
|
|
||||||
|
log-info $"Will deploy to providers: ($providers_used | str join ', ')" "middleware"
|
||||||
|
|
||||||
|
# Return basic deployment info
|
||||||
|
{
|
||||||
|
providers_used: $providers_used
|
||||||
|
total_servers: ($settings.data.servers | length)
|
||||||
|
deployment_plan: $deployment_plan
|
||||||
|
status: "planned"
|
||||||
|
timestamp: (date now)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Get provider status with capabilities
|
||||||
|
export def mw_provider_status [
|
||||||
|
--verbose
|
||||||
|
]: nothing -> table {
|
||||||
|
if not ($env.PROVIDER_REGISTRY_INITIALIZED? | default false) {
|
||||||
|
init-middleware | ignore
|
||||||
|
}
|
||||||
|
|
||||||
|
let providers = (list-providers --available-only)
|
||||||
|
|
||||||
|
$providers | each {|provider|
|
||||||
|
let allowed = (is_provider_allowed $provider.name)
|
||||||
|
let capabilities = (get-provider-capabilities-for $provider.name)
|
||||||
|
|
||||||
|
let basic_info = {
|
||||||
|
name: $provider.name
|
||||||
|
type: $provider.type
|
||||||
|
available: $provider.available
|
||||||
|
loaded: $provider.loaded
|
||||||
|
profile_allowed: $allowed
|
||||||
|
status: (if $allowed { "✅ Available" } else { "🛑 Blocked by profile" })
|
||||||
|
}
|
||||||
|
|
||||||
|
if $verbose {
|
||||||
|
$basic_info | merge {
|
||||||
|
capabilities: $provider.capabilities
|
||||||
|
server_management: ($capabilities.server_management? | default false)
|
||||||
|
multi_region: ($capabilities.multi_region? | default false)
|
||||||
|
auto_scaling: ($capabilities.auto_scaling? | default false)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
$basic_info
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Get deployment recommendations for multi-provider setup
|
||||||
|
export def mw_suggest_deployment_strategy [
|
||||||
|
requirements: record
|
||||||
|
]: nothing -> record {
|
||||||
|
log-info "Analyzing requirements for deployment strategy" "middleware"
|
||||||
|
|
||||||
|
let available_providers = (list-providers --available-only)
|
||||||
|
|
||||||
|
mut recommendations = {
|
||||||
|
strategy: "single-provider" # default
|
||||||
|
primary_provider: ""
|
||||||
|
secondary_providers: []
|
||||||
|
rationale: []
|
||||||
|
}
|
||||||
|
|
||||||
|
# Analyze requirements and suggest strategy
|
||||||
|
if ($requirements.regions? | default [] | length) > 1 {
|
||||||
|
$recommendations.strategy = "multi-provider"
|
||||||
|
$recommendations.rationale = ($recommendations.rationale | append "Multi-region requirement detected")
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($requirements.high_availability? | default false) {
|
||||||
|
$recommendations.strategy = "multi-provider"
|
||||||
|
$recommendations.rationale = ($recommendations.rationale | append "High availability requirement")
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($requirements.cost_optimization? | default false) {
|
||||||
|
$recommendations.rationale = ($recommendations.rationale | append "Cost optimization: consider mixing providers")
|
||||||
|
}
|
||||||
|
|
||||||
|
# Select primary provider based on capabilities
|
||||||
|
let suitable_providers = ($available_providers | where {|p|
|
||||||
|
let caps = (get-provider-capabilities-for $p.name)
|
||||||
|
$caps.server_management == true
|
||||||
|
})
|
||||||
|
|
||||||
|
if ($suitable_providers | length) > 0 {
|
||||||
|
$recommendations.primary_provider = ($suitable_providers | get 0 | get name)
|
||||||
|
|
||||||
|
if $recommendations.strategy == "multi-provider" {
|
||||||
|
$recommendations.secondary_providers = ($suitable_providers | skip 1 | get name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
log-info $"Recommended strategy: ($recommendations.strategy)" "middleware"
|
||||||
|
$recommendations
|
||||||
|
}
|
||||||
|
|
||||||
|
# Initialize middleware when loaded
|
||||||
|
export-env {
|
||||||
|
# This will be set when middleware functions are first called
|
||||||
|
}
|
||||||
6
providers/prov_lib/mod.nu
Normal file
6
providers/prov_lib/mod.nu
Normal file
@ -0,0 +1,6 @@
|
|||||||
|
use upcloud/servers.nu *
|
||||||
|
use aws/servers.nu *
|
||||||
|
use local/servers.nu *
|
||||||
|
|
||||||
|
export use middleware.nu *
|
||||||
|
|
||||||
6
providers/prov_lib/mod.nu-e
Normal file
6
providers/prov_lib/mod.nu-e
Normal file
@ -0,0 +1,6 @@
|
|||||||
|
use upcloud/servers.nu *
|
||||||
|
use aws/servers.nu *
|
||||||
|
use local/servers.nu *
|
||||||
|
|
||||||
|
export use middleware.nu *
|
||||||
|
|
||||||
52
providers/upcloud/README.md
Normal file
52
providers/upcloud/README.md
Normal file
@ -0,0 +1,52 @@
|
|||||||
|
# UpCloud Declarative Provision via scripts & templates
|
||||||
|
|
||||||
|
lib-tasks/kubernetes
|
||||||
|
Part of [Cloud Native zone Provision](/CloudNativeZone/cnz-provision)
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
Install [Python](https://es.wikipedia.org/wiki/Python)
|
||||||
|
|
||||||
|
For [Ubuntu](https://ubuntu.com/)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo apt install wget build-essential libncursesw5-dev libssl-dev \
|
||||||
|
libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev libffi-dev zlib1g-dev
|
||||||
|
|
||||||
|
sudo add-apt-repository ppa:deadsnakes/ppa
|
||||||
|
|
||||||
|
sudo apt install python3.13
|
||||||
|
sudo apt-get -y install python3-pip
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
Install [Jinja2 engine](https://jinja.palletsprojects.com/en/3.1.x/)
|
||||||
|
|
||||||
|
```python
|
||||||
|
pip3 install Jinja2
|
||||||
|
```
|
||||||
|
|
||||||
|
Install [Python YAML](https://pypi.org/project/PyYAML/)
|
||||||
|
|
||||||
|
```python
|
||||||
|
pip3 install PyYAML
|
||||||
|
```
|
||||||
|
|
||||||
|
[Install YQ](https://github.com/mikefarah/yq/#install)
|
||||||
|
|
||||||
|
[Install JQ](https://jqlang.github.io/jq/download/)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
apt install jq
|
||||||
|
```
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
[YAML org](https://yaml.org/)
|
||||||
|
|
||||||
|
[YQ](https://github.com/mikefarah/yq)
|
||||||
|
|
||||||
|
[YQ Documentation](https://mikefarah.gitbook.io/yq/)
|
||||||
|
|
||||||
|
[Jinja2 Tempalte engine](https://jinja.palletsprojects.com/en/3.1.x/)
|
||||||
|
|
||||||
4
providers/upcloud/bin/get_plans.sh
Executable file
4
providers/upcloud/bin/get_plans.sh
Executable file
@ -0,0 +1,4 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
[ -z "$1" ] && echo "no prefix plans found !!
|
||||||
|
All plans can be display with: upctl server plans" && exit 1
|
||||||
|
upctl server plans | grep $1 | awk '{ print $1}' | sed 's/^/\| "/g' | sed 's/$/"/g' | tr -d "\n"
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
x
Reference in New Issue
Block a user