chore: Modify fragment loader to support multiple paths and TYPEDIALOG_FRAGMENT_PATH

2025-12-26 22:35:06 +00:00 · 2025-12-26 22:35:06 +00:00 · 960e36aaf5
commit 960e36aaf5
parent 24d715523f
3 changed files with 152 additions and 56 deletions
--- a/README.md
+++ b/README.md
@ -7,14 +7,15 @@
 # TypeDialog

 > ▲ Create Type-Safe Interactive Dialogs.
-
+>
 > Prompts, forms, schemas definition, use backends (CLI, TUI, Web, AI).
-
+>
 > Extended with LLM agents, IaC generation, and [Nickel](https://nickel-lang.org) validation.

 ## Features Implemented

 ### Core
+
 - **6 Backends**: CLI (inquire), TUI (ratatui), Web (axum), AI (RAG/embeddings), Agent (LLM execution), Prov-gen (IaC generation)
 - **8 Prompt Types**: text, confirm, select, multi-select, password, date, editor, custom
 - **Declarative Forms**: TOML-based definitions with fragments & composition
@ -22,6 +23,7 @@
 - **Zero Runtime Dependencies**: Core library works standalone

 ### Advanced
+
 - **Type-Safe Schemas**: Bidirectional [Nickel](https://nickel-lang.org) integration for validation & serialization
 - **Dynamic Logic**: Conditional fields, smart defaults, real-time validation, repeating groups
 - **i18n**: Fluent (.ftl) translations with automatic locale detection
@ -29,6 +31,7 @@
 - **Contracts**: Pre/post-conditions and business rule enforcement

 ### Infrastructure
+
 - **3,818 Tests**: Comprehensive coverage (503% growth during development)
 - **CI/CD**: GitHub Actions + Woodpecker pipelines with automated testing
 - **Docker**: Multi-stage builds with optimization for deployment
@ -44,6 +47,7 @@
 See [docs/installation.md](docs/installation.md) for detailed setup.

 Requirements:
+
 - **Rust 1.70+** - [Install](https://rustup.rs/)
 - **just** - `cargo install just` (or `brew install just`)

@ -62,11 +66,15 @@ just test::all

 # Run example
 cargo run --example form
+
+# Run with defaults pre-loaded
+typedialog form config.toml --defaults defaults.json
 ```

 ## Backends at a Glance

 ### CLI Backend (inquire)
+
 Interactive terminal prompts for scripting and automation.

 ```bash
@ -78,12 +86,16 @@ typedialog select "Choose role" Admin User Guest

 # Output as JSON
 typedialog text "Email" --format json
+
+# Pre-populate form with defaults
+typedialog form schema.toml --defaults config.json --format json
 ```

 **Use for:** Scripts, CI/CD pipelines, server tools, piping between tools
 **See:** [`examples/04-backends/cli/`](examples/04-backends/cli/)

 ### TUI Backend (ratatui)
+
 Full terminal UI with keyboard navigation and mouse support.

 ```bash
@ -94,6 +106,7 @@ cargo run -p typedialog-tui --example form_with_autocompletion
 **See:** [`examples/04-backends/tui/`](examples/04-backends/tui/)

 ### Web Backend (axum)
+
 HTTP server with browser-based forms.

 ```bash
@ -105,6 +118,7 @@ cargo run -p typedialog-web -- --config config/web/dev.toml
 **See:** [`examples/04-backends/web/`](examples/04-backends/web/)

 ### AI Backend (typedialog-ai)
+
 Retrieval-Augmented Generation (RAG) system with semantic search and embeddings.

 ```bash
@ -117,6 +131,7 @@ typedialog-ai --config config/ai/production.toml --build-graph ./docs

 **Use for:** Documentation search, context-aware assistance, knowledge retrieval, semantic search
 **Features:**
+
 - Multi-provider embeddings (OpenAI, Ollama)
 - Vector store (in-memory, Redis)
 - Knowledge graph generation
@ -135,6 +150,7 @@ Execute AI agents defined as `.agent.mdx` files with template variables, file im
 ### Quick Example

 Create `hello.agent.mdx`:
+
 ```yaml
 ---
@agent {
@ -149,6 +165,7 @@ Say hello to {{name}} in a warm and friendly way!
 ```

 Run it:
+
 ```bash
 typedialog-ag hello.agent.mdx
 # Prompts: name (String): Alice
@ -157,12 +174,12 @@ typedialog-ag hello.agent.mdx

 ### Supported LLM Providers

-| Provider | Models | Best For | Privacy |
-|----------|--------|----------|---------|
-| **Claude** | Haiku, Sonnet, Opus | Code, reasoning, analysis | Cloud |
-| **OpenAI** | GPT-4o, GPT-4o-mini, o1, o3 | Code, general tasks | Cloud |
-| **Gemini** | 2.0 Flash, 1.5 Pro | Creative, multi-modal | Cloud (free tier) |
-| **Ollama** | llama2, mistral, codellama, etc. | Privacy, offline, free | Local |
+| Provider   | Models                           | Best For                  | Privacy           |
+| ---------- | -------------------------------- | ------------------------- | ----------------- |
+| **Claude** | Haiku, Sonnet, Opus              | Code, reasoning, analysis | Cloud             |
+| **OpenAI** | GPT-4o, GPT-4o-mini, o1, o3      | Code, general tasks       | Cloud             |
+| **Gemini** | 2.0 Flash, 1.5 Pro               | Creative, multi-modal     | Cloud (free tier) |
+| **Ollama** | llama2, mistral, codellama, etc. | Privacy, offline, free    | Local             |

 ### Features

@ -186,6 +203,7 @@ typedialog-ag examples/12-agent-execution/local-privacy.agent.mdx
 ```

 **Learn more:**
+
 - [Agent Documentation](docs/agent/) - Complete guide
 - [Getting Started](docs/agent/getting_started.md) - Installation and first agent
 - [LLM Providers](docs/agent/llm_providers.md) - Provider comparison
@ -197,7 +215,7 @@ typedialog-ag examples/12-agent-execution/local-privacy.agent.mdx

 Generate infrastructure configurations for AWS, GCP, Azure, Hetzner, UpCloud, and LXD from interactive forms or declarative specifications.

-### Quick Example
+### Provisioning Example

 ```bash
 # Generate infrastructure with interactive prompts
@ -210,7 +228,7 @@ typedialog-prov-gen --name myproject --providers aws,hetzner --ai-assist
 typedialog-prov-gen --name myproject --dry-run
 ```

-### Features
+### Provisioning Features

 - **Multi-Cloud Support**: AWS, GCP, Azure, Hetzner, UpCloud, LXD
 - **7-Layer Validation**: Forms → Constraints → Values → Validators → Schemas → Defaults → JSON
@ -221,16 +239,17 @@ typedialog-prov-gen --name myproject --dry-run

 ### Supported Providers

-| Provider | Type | Best For |
-|----------|------|----------|
-| **AWS** | Cloud | Enterprise, scalability, full service catalog |
-| **GCP** | Cloud | Data analytics, ML workloads |
-| **Azure** | Cloud | Enterprise integration, hybrid cloud, Microsoft ecosystem |
-| **Hetzner** | Cloud/Dedicated | Cost-effective European hosting |
-| **UpCloud** | Cloud | High-performance SSD, flexible pricing |
-| **LXD** | Local/Private | Development, on-premise, containers |
+| Provider    | Type            | Best For                                                    |
+| ----------- | --------------- | ----------------------------------------------------------- |
+| **AWS**     | Cloud           | Enterprise, scalability, full service catalog               |
+| **GCP**     | Cloud           | Data analytics, ML workloads                                |
+| **Azure**   | Cloud           | Enterprise integration, hybrid cloud, Microsoft ecosystem   |
+| **Hetzner** | Cloud/Dedicated | Cost-effective European hosting                             |
+| **UpCloud** | Cloud           | High-performance SSD, flexible pricing                      |
+| **LXD**     | Local/Private   | Development, on-premise, containers                         |

 **Learn more:**
+
 - [Prov-gen Documentation](docs/prov-gen/) - Complete guide
 - [Examples](examples/11-prov-gen/) - Multi-cloud configurations
 - [Templates](crates/typedialog-prov-gen/templates/) - Provider fragments
@ -253,12 +272,14 @@ typedialog form schema.toml --backend tui
 ```

 **Benefits:**
+
 - Schema validation before/after collection
 - Type contracts enforced throughout pipeline
 - Documentation embedded in schemas
 - Deterministic configuration generation

 **Learn more:**
+
 - [Nickel Integration Guide](docs/configuration.md#nickel-integration)
 - [Examples: 06-integrations/nickel/](examples/06-integrations/nickel/)
 - [Nickel Lang Documentation](https://nickel-lang.org/)
@ -267,29 +288,29 @@ typedialog form schema.toml --backend tui

 Complete documentation in [`docs/`](docs/):

-| Document | Purpose |
-|----------|---------|
-| [**installation.md**](docs/installation.md) | Prerequisites & setup |
-| [**development.md**](docs/development.md) | Development workflows |
-| [**build.md**](docs/build.md) | Building & cross-compilation |
-| [**release.md**](docs/release.md) | Release process |
-| [**configuration.md**](docs/configuration.md) | Backend & Nickel configuration |
+| Document                                          | Purpose                        |
+| ------------------------------------------------- | ------------------------------ |
+| [**installation.md**](docs/installation.md)       | Prerequisites & setup          |
+| [**development.md**](docs/development.md)         | Development workflows          |
+| [**build.md**](docs/build.md)                     | Building & cross-compilation   |
+| [**release.md**](docs/release.md)                 | Release process                |
+| [**configuration.md**](docs/configuration.md)     | Backend & Nickel configuration |

-## Examples
+## Project Examples

 Complete working examples in [`examples/`](examples/):

-| Category | Path | Contents |
-|----------|------|----------|
-| **Getting Started** | [01-basic](examples/01-basic/) | Form syntax, sections, validation |
-| **Advanced Features** | [02-advanced](examples/02-advanced/) | Conditional logic, dynamic fields |
-| **Styling** | [03-styling](examples/03-styling/) | Themes, borders, visual design |
-| **Backends** | [04-backends](examples/04-backends/) | CLI, TUI, Web implementations |
-| **Composition** | [05-fragments](examples/05-fragments/) | Reusable components |
-| **Integrations** | [06-integrations](examples/06-integrations/) | [Nickel](examples/06-integrations/nickel/), [i18n](examples/06-integrations/i18n/) |
-| **Production** | [09-templates](examples/09-templates/) | Real-world use cases |
-| **Provisioning** | [11-prov-gen](examples/11-prov-gen/) | Infrastructure generation, multi-cloud |
-| **Agent Execution** | [12-agent-execution](examples/12-agent-execution/) | LLM agents, AI workflows |
+| Category              | Path                                               | Contents                                                                           |
+| --------------------- | -------------------------------------------------- | ---------------------------------------------------------------------------------- |
+| **Getting Started**   | [01-basic](examples/01-basic/)                     | Form syntax, sections, validation                                                  |
+| **Advanced Features** | [02-advanced](examples/02-advanced/)               | Conditional logic, dynamic fields                                                  |
+| **Styling**           | [03-styling](examples/03-styling/)                 | Themes, borders, visual design                                                     |
+| **Backends**          | [04-backends](examples/04-backends/)               | CLI, TUI, Web implementations                                                      |
+| **Composition**       | [05-fragments](examples/05-fragments/)             | Reusable components                                                                |
+| **Integrations**      | [06-integrations](examples/06-integrations/)       | [Nickel](examples/06-integrations/nickel/), [i18n](examples/06-integrations/i18n/) |
+| **Production**        | [09-templates](examples/09-templates/)             | Real-world use cases                                                               |
+| **Provisioning**      | [11-prov-gen](examples/11-prov-gen/)               | Infrastructure generation, multi-cloud                                             |
+| **Agent Execution**   | [12-agent-execution](examples/12-agent-execution/) | LLM agents, AI workflows                                                           |

 **Quick start:** [examples/README.md](examples/README.md)

@ -352,7 +373,7 @@ See [docs/release.md](docs/release.md) for release workflow.

 Pre-configured settings for each backend and environment:

-```
+```text
 config/
 ├── cli/         # default, dev, production
 ├── tui/         # default, dev, production
@ -366,7 +387,7 @@ See [docs/configuration.md](docs/configuration.md) and [config/README.md](config

 ## Project Structure

-```
+```text
 typedialog/
 ├── crates/
 │   ├── typedialog-core/       # Core library (forms, validation)
@ -390,6 +411,7 @@ typedialog/
 ## Key Technologies

 **Core:**
+
 - **Rust** - Type-safe systems language
 - **Nickel** - Type-safe configuration language (schema integration)
 - **TOML** - Form and configuration language
@ -397,11 +419,13 @@ typedialog/
 - **just** - Command orchestration

 **Backends:**
+
 - **inquire** - Interactive prompt library (CLI backend)
 - **Ratatui** - Terminal UI framework (TUI backend)
 - **Axum** - Web framework (Web backend)

 **AI & Agent:**
+
 - **Tera** - Template engine (Jinja2-compatible) for agent files
 - **Claude API** - Anthropic's language models
 - **OpenAI API** - GPT models
@ -409,6 +433,7 @@ typedialog/
 - **Ollama** - Local LLM runtime

 **Infrastructure:**
+
 - **Nickel contracts** - Type-safe IaC validation
 - **AWS/GCP/Hetzner/UpCloud APIs** - Multi-cloud provisioning

@ -441,14 +466,17 @@ just distro::package-release    # Prepare release
 ## System Requirements

 ### Minimum
+
 - Rust 1.70+
 - 4GB RAM
 - 2GB disk space

 ### For Cross-Compilation
+
 - Docker (or cargo-cross)

 ### Optional Tools
+
 - **Nickel CLI** - For developing type-safe Nickel schemas (used with `06-integrations/nickel/` examples)
 - **cargo-watch** - For hot-reload during development
 - **cargo-cross** - For cross-compilation to other platforms
--- a/crates/typedialog-core/src/form_parser/parser.rs
+++ b/crates/typedialog-core/src/form_parser/parser.rs
@ -3,10 +3,64 @@
 //! Handles parsing form definitions from TOML format and loading from files.

 use crate::error::Result;
-use std::path::Path;
+use std::path::{Path, PathBuf};

 use super::types::FormDefinition;

+/// Get fragment search paths from environment variable
+///
+/// Reads TYPEDIALOG_FRAGMENT_PATH environment variable (colon-separated on Unix, semicolon on Windows)
+/// and returns a vector of PathBuf. Returns empty vector if not set.
+fn get_fragment_search_paths() -> Vec<PathBuf> {
+    std::env::var("TYPEDIALOG_FRAGMENT_PATH")
+        .ok()
+        .map(|paths| {
+            #[cfg(unix)]
+            let separator = ':';
+            #[cfg(windows)]
+            let separator = ';';
+
+            paths
+                .split(separator)
+                .filter(|s| !s.is_empty())
+                .map(PathBuf::from)
+                .collect()
+        })
+        .unwrap_or_default()
+}
+
+/// Resolve fragment path by searching in multiple directories
+///
+/// Search order:
+/// 1. If absolute path, use it directly
+/// 2. Try base_dir first (backward compatibility)
+/// 3. Try each directory in TYPEDIALOG_FRAGMENT_PATH
+///
+/// Returns the first existing path, or the base_dir-joined path if none exist (for error reporting)
+fn resolve_fragment_path(path: &str, base_dir: &Path) -> PathBuf {
+    // If absolute, use directly
+    if Path::new(path).is_absolute() {
+        return Path::new(path).to_path_buf();
+    }
+
+    // Try base_dir first (backward compatibility)
+    let base_path = base_dir.join(path);
+    if base_path.exists() {
+        return base_path;
+    }
+
+    // Try each search path
+    for search_dir in get_fragment_search_paths() {
+        let candidate = search_dir.join(path);
+        if candidate.exists() {
+            return candidate;
+        }
+    }
+
+    // Return base_path for error reporting (original behavior)
+    base_path
+}
+
 /// Resolve constraint interpolations in TOML content
 /// Replaces "${constraint.path.to.value}" (with quotes) with actual values from constraints.toml
 /// The quotes are removed as part of the replacement, so the value becomes a bare number
@ -103,16 +157,13 @@ pub fn load_from_file(path: impl AsRef<Path>) -> Result<FormDefinition> {

 /// Load unified elements from a TOML file with proper path resolution
 ///
-/// Automatically migrates legacy items/fields to the unified elements format
+/// Automatically migrates legacy items/fields to the unified elements format.
+/// Searches for fragments in base_dir first, then TYPEDIALOG_FRAGMENT_PATH directories.
 pub(super) fn load_elements_from_file(
    path: &str,
    base_dir: &Path,
 ) -> Result<Vec<super::types::FormElement>> {
-    let resolved_path = if Path::new(path).is_absolute() {
-        Path::new(path).to_path_buf()
-    } else {
-        base_dir.join(path)
-    };
+    let resolved_path = resolve_fragment_path(path, base_dir);
    let content = std::fs::read_to_string(&resolved_path)?;

    // Resolve constraint interpolations before parsing
@ -124,15 +175,12 @@ pub(super) fn load_elements_from_file(

 /// Load items from a TOML file with proper path resolution
 /// (For backward compatibility - prefer load_elements_from_file for new code)
+/// Searches for fragments in base_dir first, then TYPEDIALOG_FRAGMENT_PATH directories.
 pub(super) fn load_items_from_file(
    path: &str,
    base_dir: &Path,
 ) -> Result<Vec<super::types::DisplayItem>> {
-    let resolved_path = if Path::new(path).is_absolute() {
-        Path::new(path).to_path_buf()
-    } else {
-        base_dir.join(path)
-    };
+    let resolved_path = resolve_fragment_path(path, base_dir);
    let content = std::fs::read_to_string(&resolved_path)?;

    // Resolve constraint interpolations before parsing
@ -143,15 +191,12 @@ pub(super) fn load_items_from_file(

 /// Load fields from a TOML file with proper path resolution
 /// (For backward compatibility - prefer load_elements_from_file for new code)
+/// Searches for fragments in base_dir first, then TYPEDIALOG_FRAGMENT_PATH directories.
 pub(super) fn load_fields_from_file(
    path: &str,
    base_dir: &Path,
 ) -> Result<Vec<super::types::FieldDefinition>> {
-    let resolved_path = if Path::new(path).is_absolute() {
-        Path::new(path).to_path_buf()
-    } else {
-        base_dir.join(path)
-    };
+    let resolved_path = resolve_fragment_path(path, base_dir);
    let content = std::fs::read_to_string(&resolved_path)?;

    // Resolve constraint interpolations before parsing
--- a/docs/configuration.md
+++ b/docs/configuration.md
@ -245,6 +245,29 @@ typedialog-tui --config config/tui/default.toml form.toml
 export TYPEDIALOG_WEB_PORT=3000
 export TYPEDIALOG_WEB_CORS_ORIGINS="localhost,example.com"
 typedialog-web --config config/web/default.toml
+
+# Fragment Search Paths (All Backends)
+export TYPEDIALOG_FRAGMENT_PATH="/path/to/fragments:/another/path/fragments"
+# On Windows use semicolon separator:
+# set TYPEDIALOG_FRAGMENT_PATH=C:\fragments;D:\other\fragments
+typedialog form.toml
+```
+
+**TYPEDIALOG_FRAGMENT_PATH:**
+
+Colon-separated list (semicolon on Windows) of directories to search for fragment files.
+Fragments are reusable form components loaded via `includes` directive in group definitions.
+
+Search order:
+
+1. Form's directory (backward compatible)
+2. Directories in TYPEDIALOG_FRAGMENT_PATH (left to right)
+
+Example:
+
+```bash
+export TYPEDIALOG_FRAGMENT_PATH="/usr/local/share/typedialog/fragments:$HOME/.typedialog/fragments"
+typedialog form.toml
 ```

 ## CLI Backend Configuration Details