jesus/TypeDialog
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
TypeDialog/CHANGELOG.md
Jesús Pérez 9ca1bfb8cf
chore: new projet folders structure
2025-12-24 03:11:32 +00:00

17 KiB
Raw Permalink Blame History

Changes

[Unreleased]

Added - Professional Quality Improvements

Community Health Files

  • Created CONTRIBUTING.md with contribution guidelines, development workflow, and coding standards
  • Created SECURITY.md with vulnerability reporting process and security best practices
  • Created CODE_OF_CONDUCT.md with Contributor Covenant 2.1

Security & Compliance

  • Created deny.toml with cargo-deny configuration for license/security policies
  • Configured Renovate for automated dependency updates with automerge for minor/patch
  • Added pre-commit hooks (fmt, clippy, shellcheck, markdownlint, trailing-whitespace, check-yaml/toml/json)

Testing Infrastructure

  • Configured cargo-llvm-cov for code coverage tracking
  • Integrated Codecov for coverage reporting in CI
  • Added coverage badge to README
  • Created benchmark tests with criterion for performance baselines
  • Added property-based testing with proptest for validation functions
  • Justfile recipes: just dev::coverage, just dev::bench, just dev::coverage-open

CI/CD Enhancements

  • Multi-OS CI matrix: Linux, macOS, Windows (was Linux only)
  • Cargo cache with rust-cache action (faster CI builds)
  • Code coverage job uploading to Codecov
  • Benchmark job running on pull requests
  • cargo-deny checks in CI pipeline

Release Automation

  • Full automated release workflow: tag → build multi-OS → publish GitHub + crates.io
  • Builds for 5 targets: Linux x86_64/aarch64, macOS x86_64/aarch64, Windows x86_64
  • Automatic tarball + SHA256 generation and upload
  • Automatic crates.io publishing in dependency order
  • cargo-binstall metadata for easy installation: cargo binstall typedialog

Impact

  • Code coverage now tracked (was unknown)
  • CI tests on 3 operating systems (was 1)
  • Release process fully automated (was manual)
  • 7 pre-commit hooks validate code quality
  • Benchmark baselines established for performance tracking
  • Property tests validate edge cases
  • Security scanning with cargo-deny
  • Automated dependency updates with Renovate

Added - Multi-Language Linting System

Linting Infrastructure

  • Created .cargo/config.toml with strict rustflags (warnings, future-incompatible, unused, etc.)
  • Extended justfiles/dev.just with language-specific linters:
    • just dev::lint-rust - Clippy linting
    • just dev::lint-bash - Shellcheck for 142 bash scripts
    • just dev::lint-nickel - Nickel typecheck for 63 files
    • just dev::lint-nushell - Nushell validation for 9 scripts
    • just dev::lint-markdown - Markdownlint for docs/ folder only
    • just dev::lint-all - Execute all linters
  • Extended justfiles/ci.just with CI-compatible linter recipes
  • Updated just check-all to include multi-language linting

GitHub Actions CI/CD

  • Created .github/workflows/ci.yml with automated linting matrix
  • Parallel linting jobs per language (rust, bash, nickel, nushell, markdown)
  • Integrated with test and build stages

Linter Tools

  • Added shellcheck for bash script analysis
  • Added markdownlint-cli2 for docs/ validation
  • Leveraged existing nickel CLI for typecheck
  • Leveraged nushell built-in --check flag

Configuration

  • Created .markdownlintrc with basic rules (MD013, MD033, MD041 disabled)
  • Updated scripts/check_deps.sh to verify all linter dependencies
  • Updated docs/build.md with shellcheck and markdownlint installation

Impact

  • 142 bash scripts now validated with shellcheck
  • 63 Nickel files now typechecked
  • 9 Nushell scripts validated
  • docs/ markdown files (subset of 8,530 total) validated
  • Rust code enforces stricter compilation standards via rustflags

Added - TypeDialog Agent System

Multi-Provider LLM Support

  • Complete LLM provider abstraction supporting Claude, OpenAI, Gemini, and Ollama
  • Unified trait-based architecture for consistent API across providers
  • Streaming support with two protocols: SSE (Claude/OpenAI) and JSON (Gemini/Ollama)
  • Token usage tracking across all providers (except OpenAI streaming limitation)
  • Local execution with Ollama for privacy-sensitive workloads

Agent Execution System

  • New .agent.mdx file format for defining AI agents
  • YAML frontmatter configuration with @agent, @input, @import, @shell, @validate directives
  • CLI tool typedialog-ag for executing agent files
  • Server component typedialog-ag-server for HTTP API access

Template System

  • Tera template engine (Jinja2-compatible) for variable substitution and conditionals
  • Template variables with {{ var }} syntax
  • Conditional logic with {% if %}...{% endif %}
  • File imports with glob pattern support: @import "src/**/*.rs" as code
  • Shell command integration: @shell "git status" as status
  • Environment variable loading

Output Validation

  • Configurable validation rules for agent outputs
  • Format validation (markdown, JSON, YAML)
  • Content requirements (must_contain checks)
  • Length constraints (min_length, max_length)

Provider-Specific Features

  • Claude (Anthropic): 3 models (Haiku, Sonnet, Opus), excellent code/reasoning, SSE streaming
  • OpenAI (GPT): gpt-4o, gpt-4o-mini, o1, o3 models, broad capabilities, SSE streaming
  • Gemini (Google): Creative tasks, 2M context window, free tier, JSON streaming
  • Ollama (Local): Complete privacy, offline operation, no API costs, supports llama2/mistral/codellama/etc.

Added - Organization & Documentation

Project Structure

  • Created demos/agent/ with 4 provider comparison demos
  • Created tests/agent/ with validation tests
  • Created examples/12-agent-execution/ with 8 practical CLI examples
  • Organized docs into backend-specific folders: docs/cli/, docs/tui/, docs/web/, docs/ai/, docs/agent/, docs/encryption/

User Documentation

  • docs/agent/README.md - Agent system overview and navigation
  • docs/agent/getting_started.md - Installation and first agent guide
  • docs/agent/llm_providers.md - Complete provider comparison (400+ lines)
  • docs/cli/README.md - CLI backend documentation
  • docs/tui/README.md - TUI backend documentation
  • docs/web/README.md - Web backend documentation
  • docs/ai/README.md - AI backend documentation
  • docs/encryption/README.md - Encryption documentation

Agent Examples

  • basic-greeting.agent.mdx - Simple introduction (Claude Haiku)
  • code-review.agent.mdx - Security & performance analysis (Claude Sonnet)
  • documentation.agent.mdx - Generate docs from code (GPT-4o)
  • task-planner.agent.mdx - Project planning (Claude Sonnet)
  • local-privacy.agent.mdx - Privacy-first analysis (Ollama/llama2)
  • creative-writer.agent.mdx - Creative content (Gemini 2.0)
  • refactoring-assistant.agent.mdx - Code refactoring (GPT-4o-mini)
  • architect.agent.mdx - High-level design (Claude Opus)

Cross-References

  • Extensive linking between examples, tests, demos, and documentation
  • Navigation structure: examples → demos → tests → core → technical docs

Added - Configuration & Provisioning

Configuration Structure

  • Renamed config/agent/ → config/ag/ for consistency with typedialog-ag binary name
  • Added configuration profiles for all 6 backends: cli/, tui/, web/, ai/, ag/, prov-gen/
  • Each backend has 3 environments: default.toml, dev.toml, production.toml
  • Total: 18 configuration files (6 backends × 3 environments)

config/ag/ - Agent configuration (LLM execution)

  • default.toml - Claude Haiku default, streaming enabled, Tera templates
  • dev.toml - Ollama (local) for development, debug logging, temperature 0.7
  • production.toml - Claude Sonnet, lower temperature (0.3), rate limiting enabled

config/prov-gen/ - Provisioning generator configuration (IaC)

  • default.toml - AWS + Hetzner providers, Nickel validation, AI disabled
  • dev.toml - Hetzner + LXD (cheaper), Ollama for AI, verbose logging
  • production.toml - AWS + GCP, strict validation, encryption required, tests required

Provisioning Generator Templates

  • Created provider-gcp-section.toml (256 lines, 15 elements)

    • Google Cloud Platform: 24 regions, 16 machine types (e2/n1/n2/c2)
    • Project ID + Service Account authentication
    • Disk types: Standard, Balanced SSD, Premium SSD, Ultra SSD
    • VPC networking, preemptible instances, deletion protection

  • Created provider-azure-section.toml (296 lines, 17 elements)

    • Microsoft Azure: 30 regions globally
    • Service Principal authentication (Tenant ID, Client ID, Secret)
    • 12 VM sizes (B-series burstable, D/E-series, F-series compute)
    • Managed disks, Virtual Networks, accelerated networking

  • Created provider-lxd-section.toml (207 lines, 16 elements)

    • LXD containers and virtual machines (local/remote)
    • 10 base images (Ubuntu, Debian, Alpine, Rocky, Arch)
    • CPU/Memory/Disk limits, 5 storage pools (dir, zfs, btrfs, lvm)
    • Network modes: lxdbr0 (NAT), host, macvlan
    • Nesting support (Docker inside LXD), cloud-init integration

Total provider templates: 6 providers (AWS, GCP, Azure, Hetzner, UpCloud, LXD) - 1,191 lines

Dependency Management

  • Created scripts/check_deps.sh - Automated dependency verification script
  • Checks critical dependencies: just, nickel, cargo, prov-ecosystem, secretumvault
  • Exit status 0 if all found, 1 if any missing
  • Provides installation instructions for missing dependencies

Provisioning Generator Command Enhancements

  • Added -f, --format <FORMAT> flag to typedialog-prov-gen templates command
    • Supported formats: json, yaml, toml, text (default)
    • Enables machine-readable template listing for automation and integration
    • JSON/YAML for CI/CD pipelines, TOML for configuration files
  • Template path display: Shows resolved absolute path before listing
    • Output: "From: /absolute/path/to/templates"
    • Helps users verify which template directory is loaded
  • TemplateCategory structure now serializable across all formats
    • Consistent structure: name, description, templates[]
    • TOML output uses [[categories]] array-of-tables format

Changed - Documentation Updates

docs/build.md - Critical Dependencies Section (320+ lines added)

  • Internal Dependencies: prov-ecosystem (for typedialog-prov-gen), secretumvault (for encryption)

    • Installation: sibling directory or git submodule
    • Path configuration: Cargo.toml paths, environment variables
    • Verification commands

  • External Dependencies: just (command runner), Nickel (configuration language)

    • Multi-platform installation: Homebrew, cargo, from source
    • PATH configuration
    • Verification tests

  • Dependency Quick Check: Bash script to verify all dependencies

  • Troubleshooting: 3 common issues with solutions each

    • prov-ecosystem/secretumvault not found (3 solutions)
    • Nickel binary not found (3 solutions)
    • just command not found (3 solutions)

docs/README.md - Navigation Updates

  • Added "Critical Dependencies" to Building section
  • Updated "Building & Distribution" with dependencies reference

config/README.md - Complete Backend Documentation (550+ lines)

  • Documented all 6 backends: CLI, TUI, Web, AI, Agent (ag), Prov-gen
  • Each backend has: configuration table, usage examples, features list
  • AI Backend: RAG pipeline, embeddings (OpenAI/Ollama), vector store, knowledge graphs
  • Agent Backend: Multi-provider LLM, Tera templates, streaming, variable substitution
  • Prov-gen: Multi-cloud IaC, 7-layer validation, Nickel integration, AI-assisted generation
  • Added configuration details for AI, Agent, and Prov-gen backends
  • Updated development and production configuration examples
  • Distribution structure updated to show 6 backends

README.md - Main Project Documentation

  • Updated Features: "6 Backends" (was "3 Backends")

  • Added AI Backend section (lines 95-113)

    • RAG system with semantic search
    • Multi-provider embeddings (OpenAI, Ollama)
    • Vector store (in-memory, Redis)
    • Knowledge graph generation

  • Added Provisioning Generator section (lines 182-223)

    • Infrastructure as Code generation
    • Quick example with 3 command variants
    • 6 features: Multi-cloud, 7-layer validation, Nickel, AI-assist, templates, profiles
    • Supported providers table: AWS, GCP, Azure, Hetzner, UpCloud, LXD

  • Updated Configuration structure: 6 backends listed

  • Updated Project Structure: All 9 crates documented

  • Updated Key Technologies: Organized into Core, Backends, AI & Agent, Infrastructure

  • Updated Examples: Added 11-prov-gen/ and 12-agent-execution/ categories

docs/prov-gen/README.md

  • Updated Multi-Provider Support: AWS, GCP, Azure, Hetzner, UpCloud, LXD (was AWS, GCP, Azure, Hetzner, LXD, bare metal)
  • Changed "generates complete provisioning" → "generates provisioning" (modesty)

Changed - Naming Consistency

Package Renaming

  • typeagent-core → typedialog-ag-core
  • typeagent-cli → typedialog-ag
  • typeagent-server → typedialog-ag-server
  • config/agent/ → config/ag/

Rationale: Align with project-wide typedialog-* naming convention and binary names

Impact

  • Updated 50+ files (imports, docs, scripts, configs)
  • Updated workspace Cargo.toml dependencies
  • Updated all crate manifests
  • Updated justfile recipes
  • Updated configuration directory structure
  • 53 tests passing, 0 clippy warnings after refactor

Changed - Build & Distribution

Justfile Updates

  • Added just build::agent - Build Agent CLI
  • Added just build::agent-server - Build Agent server
  • Added just build::prov-gen - Build Provisioning Generator
  • Added just test::agent - Test Agent system (53 tests)
  • Added just test::prov-gen - Test Provisioning Generator (11 tests)
  • Updated just build::all to include Agent and Prov-gen builds
  • Fixed just test::agent - Removed invalid lib test for binary-only typedialog-ag crate

Script Updates

  • scripts/create_distro.sh - Include all binaries: typedialog-ag, typedialog-ag-server, typedialog-prov-gen
  • scripts/check_deps.sh - New dependency verification script (67 lines)
  • installers/bootstrap/install.sh - Install Agent and Prov-gen binaries
  • installers/bootstrap/install.ps1 - Install Agent and Prov-gen binaries (Windows)
  • .woodpecker/Dockerfile.cross - Cross-compile Agent and Prov-gen binaries

Build Features

  • New feature flag FEATURES_AGENT := "markup,nickel,cache"
  • Agent and Prov-gen builds integrated into full CI/CD pipeline
  • All builds verified: just build::prov-gen ✓, just build::agent ✓
  • All tests passing: just test::prov-gen ✓ (11 tests), just test::agent ✓ (53 tests, 5 ignored)

[Previous]

Added

RepeatingGroup Duplicate Detection

  • Implemented duplicate item validation across all backends (CLI, TUI, Web)
  • Added unique = true attribute support to prevent duplicate items
  • Added item counter display updates in Web backend
  • New is_duplicate() function in CLI and TUI backends

Web Backend Improvements

  • Refactored JavaScript: Replaced global constants with closure-based state management
  • Added render_global_repeating_group_script() for generic repeating group handling
  • Configuration stored in data-* attributes: data-min-items, data-max-items, data-has-unique, data-unique-key, data-field-names
  • Live item counter updates after add/delete operations
  • Single document-level event delegation for all repeating group interactions

Documentation

  • Updated docs/field_types.md with unique validation details
  • Updated examples/README.md with RepeatingGroup testing commands
  • Added docs/field_types.md (new file)
  • Updated example forms with unique = true attribute

Fixed

  • CLI Backend: Fixed validation of duplicate items in add/edit operations
  • TUI Backend: Fixed validation of duplicate items with error overlay feedback
  • Web Backend: Fixed has_unique flag reading from field definition (was incorrectly scanning fragment fields)
  • Web Backend: Fixed item counter not updating when adding/deleting items

Changed

Architecture

  • Each repeating group now maintains independent state via closure
  • Removed ~600 lines of dead code in Web JavaScript
  • Event handling now context-aware (finds correct repeating group controller)

Examples

  • examples/05-fragments/array-trackers.toml: Added unique = true to UDP and HTTP tracker arrays
  • examples/07-nickel-generation/arrays-form.toml: Added unique = true to all RepeatingGroup fields

Technical Notes

  • Duplicate detection compares ALL field values when unique = true
  • Works consistently across CLI, TUI, and Web backends
  • Backwards compatible: repeating groups without unique = true unaffected
  • max_items limit already enforced in all backends (no changes needed)

Testing

  • All 174 unit tests passing
  • No clippy warnings
  • Build verified with --all-features flag
  • Manual testing in Web backend: duplicate detection working correctly
  • Item counter updates verified on add/delete operations