website-htmx-rustelo-code/scripts/content/README.md

# Content Management Scripts

This directory contains all content management scripts for the Rustelo project. These tools handle multilingual markdown content, localization, validation, and content generation workflows.

## 📋 Requirements

**Required Tools:**
- [Nushell](https://www.nushell.sh/) - All `.nu` scripts require Nushell to be installed
- [Just](https://github.com/casey/just) - Task runner (recommended for easy command execution)
- For complete setup, see [Rustelo Requirements - Tools](https://github.com/your-repo/rustelo#tools)

**Additional Dependencies:**
- Rust toolchain with Cargo
- `content-static` feature enabled in the server crate
- `jq` for JSON processing (optional, used for validation)

## 🌍 Content Management Tools by Context

### 📝 Content Creation & Generation

#### `generate-content.nu` - Content Generator from Templates
**Purpose:** Generate new blog posts and recipes from templates with proper localization  
**Task:** Create structured content files with frontmatter and localized templates  
**Context:** Adding new content, rapid content creation, maintaining consistency  
**Command:** `nu scripts/content/generate-content.nu [COMMAND] [OPTIONS]`  

**Commands:**
- `blog-post` - Generate new blog post with frontmatter
- `recipe` - Generate new technical recipe/prescription  
- `templates` - Create/update content templates
- `index` - Update all content indices

**Arguments:**
- `--title TITLE` - Content title (required)
- `--category CATEGORY` - Content category
- `--author AUTHOR` - Content author
- `--tags TAG1,TAG2` - Comma-separated tags
- `--difficulty LEVEL` - Recipe difficulty (Beginner|Intermediate|Advanced)
- `--lang LANGUAGE` - Target language (en|es|all) [default: all]
- `--description DESC` - Short description
- `--published BOOL` - Published status [default: true]

**Examples:**
```bash
nu scripts/content/generate-content.nu blog-post --title "My New Post" --category "Technology"
nu scripts/content/generate-content.nu recipe --title "Docker Setup" --category "DevOps" --difficulty "Intermediate"
nu scripts/content/generate-content.nu templates
```

### 🔧 Content Management & Operations

#### `content-manager.nu` - Unified Content Management Hub
**Purpose:** Central tool for content index generation, validation, and statistics  
**Task:** Manage content indices, validate consistency, show content statistics  
**Context:** Content maintenance, index updates, content auditing  
**Command:** `nu scripts/content/content-manager.nu [COMMAND] [OPTIONS]`  

**Commands:**
- `generate-indices` - Generate JSON indices from markdown frontmatter
- `validate-ids` - Validate ID consistency across languages
- `validate-content` - Validate content structure and fields
- `validate-consistency` - Validate content consistency across languages
- `show-stats` - Show comprehensive content statistics

**Features:**
- **Dynamic Content Discovery** - Reads content types from `content-kinds.toml`
- **Multi-language Support** - Processes English and Spanish content
- **Frontmatter Processing** - Extracts metadata from markdown files
- **Index Generation** - Creates structured JSON indices for web serving
- **Error Reporting** - Detailed validation reports with actionable feedback

**Examples:**
```bash
nu scripts/content/content-manager.nu generate-indices
nu scripts/content/content-manager.nu validate-content
nu scripts/content/content-manager.nu show-stats
```

### 🔍 Content Validation Tools

#### `validate-content.nu` - Content Structure Validator
**Purpose:** Comprehensive validation of content structure and integrity  
**Task:** Validate JSON files, required fields, and cross-references  
**Context:** Content quality assurance, pre-deployment validation, CI/CD pipelines  
**Command:** `nu scripts/content/validate-content.nu [OPTIONS]`  

**Validation Checks:**
- **Directory Structure** - Consistent language directories
- **JSON Validity** - Valid index.json and meta.json files
- **Required Fields** - Mandatory frontmatter fields (title, etc.)
- **Content Length** - Warns about very short content
- **Cross-references** - Index entries match actual files

**Features:**
- Validates all content types discovered from `content-kinds.toml`
- Processes English and Spanish content
- Detailed error reporting with fix suggestions
- Statistics on total files validated

#### `validate-content-consistency.nu` - Multi-language Consistency Validator
**Purpose:** Ensure content consistency across all languages  
**Task:** Validate language parity, metadata consistency, translation completeness  
**Context:** Multi-language content maintenance, translation quality assurance  
**Command:** `nu scripts/content/validate-content-consistency.nu [OPTIONS]`  

**Consistency Checks:**
- **Language Parity** - Same content IDs exist in all languages
- **Metadata Consistency** - Similar frontmatter structure across languages
- **Translation Completeness** - All content properly translated
- **Index Validation** - Consistent index.json files across languages

#### `validate-id-consistency.nu` - ID Consistency Validator
**Purpose:** Validate ID consistency across languages and files  
**Task:** Ensure filename consistency, frontmatter ID fields, and index accuracy  
**Context:** Content integrity checks, preventing broken references  
**Command:** `nu scripts/content/validate-id-consistency.nu [OPTIONS]`  

**ID Validation:**
- **Filename Consistency** - Same files exist across all languages
- **Frontmatter IDs** - ID fields match filenames (or are absent)
- **Index Accuracy** - Index.json entries correspond to actual files
- **Duplicate Detection** - No duplicate IDs in index files

### 🌍 Translation & Localization

#### `sync-translations.nu` - Translation Synchronization Manager
**Purpose:** Manage translation keys and localization files  
**Task:** Extract keys, sync translations, validate completeness  
**Context:** Internationalization, translation workflow, localization maintenance  
**Command:** `nu scripts/content/sync-translations.nu [COMMAND] [OPTIONS]`  

**Commands:**
- `extract-keys` - Extract translation keys from content files
- `sync-keys` - Synchronize translation keys across languages
- `validate-translations` - Validate translation completeness
- `generate-missing` - Generate missing translation entries
- `show-stats` - Show translation statistics
- `create-template` - Create translation template for new language

**Arguments:**
- `--lang LANGUAGE` - Target specific language [default: all]
- `--source LANG` - Source language for template [default: en]
- `--output DIR` - Output directory [default: content/locales]

**Features:**
- **Key Extraction** - Finds translation keys in content using multiple patterns
- **Template Generation** - Creates translation templates for new languages
- **Progress Tracking** - Shows completion rates and missing translations
- **Multiple Formats** - Supports both Fluent (.ftl) and JSON formats

**Examples:**
```bash
nu scripts/content/sync-translations.nu extract-keys
nu scripts/content/sync-translations.nu sync-keys --lang es
nu scripts/content/sync-translations.nu create-template --lang fr --source en
```

### 🚀 Quick Start Commands

```bash
# Generate new blog post in both languages
nu scripts/content/generate-content.nu blog-post --title "My New Post" --category "Tech"

# Generate recipe with specific difficulty
nu scripts/content/generate-content.nu recipe --title "Docker Setup" --difficulty "Intermediate"

# Update all content indices
nu scripts/content/content-manager.nu generate-indices

# Validate all content
nu scripts/content/validate-content.nu

# Check content consistency across languages
nu scripts/content/validate-content-consistency.nu

# Validate ID consistency
nu scripts/content/validate-id-consistency.nu

# Show content statistics
nu scripts/content/content-manager.nu show-stats

# Extract and sync translation keys
nu scripts/content/sync-translations.nu extract-keys
nu scripts/content/sync-translations.nu sync-keys
```

## 📁 File Organization

```
scripts/content/
├── README.md                           # This documentation
├── content-manager.nu                  # 🔧 Central content management
├── generate-content.nu                 # 📝 Content generation from templates
├── validate-content.nu                 # 🔍 Content structure validation
├── validate-content-consistency.nu     # 🔍 Multi-language consistency
├── validate-id-consistency.nu          # 🔍 ID consistency validation
├── sync-translations.nu                # 🌍 Translation synchronization
└── templates/                          # 📋 Content templates
    ├── content-post.json               # Blog post template structure
    ├── content-post.md                 # Blog post markdown template
    ├── recipe.json                     # Recipe template structure
    └── recipe.md                       # Recipe markdown template
```

## 🌐 Supported Languages

Currently supported languages:
- **English (`en`)** - Primary language
- **Spanish (`es`)** - Secondary language

### Adding New Languages

To add support for a new language (e.g., French):

1. **Create content directories:**
   ```bash
   mkdir -p content/blog/fr content/recipes/fr
   ```

2. **Add language to configuration:**
   Update the `languages` array in each script or use environment configuration.

3. **Create localization files:**
   ```bash
   nu scripts/content/sync-translations.nu create-template --lang fr --source en
   ```

4. **Generate initial indices:**
   ```bash
   nu scripts/content/content-manager.nu generate-indices
   ```

## 📊 Content Structure

### Source Files (Markdown)
```
content/
├── blog/
│   ├── en/
│   │   ├── index.json          # English blog index
│   │   ├── post1.md            # English blog posts
│   │   └── post2.md
│   └── es/
│       ├── index.json          # Spanish blog index  
│       ├── articulo1.md        # Spanish blog posts
│       └── articulo2.md
├── recipes/
│   ├── en/
│   │   ├── index.json          # English recipe index
│   │   ├── recipe1.md          # English recipes
│   │   └── recipe2.md
│   └── es/
│       ├── index.json          # Spanish recipe index
│       ├── receta1.md          # Spanish recipes  
│       └── receta2.md
├── content-kinds.toml          # Content type definitions
└── locales/                    # Translation files
    ├── en.json
    ├── es.json
    └── extracted-keys.json
```

### Generated Files (HTML)
```
public/
├── blog/
│   ├── en/
│   │   ├── index.json          # Copied from source
│   │   ├── post1.html          # Generated HTML
│   │   └── post2.html
│   ├── es/
│   │   ├── index.json          # Copied from source
│   │   ├── articulo1.html      # Generated HTML
│   │   └── articulo2.html
│   └── index.json              # Symlink to en/index.json
├── recipes/
│   └── [similar structure]
└── content-manifest.json       # Build metadata and statistics
```

## ⚙️ Configuration

### Environment Variables
- **`SITE_CONTENT_PATH`** - Content root directory [default: `site/content`]
- Used by all scripts for consistent content location

### Content Configuration
- **`content/content-kinds.toml`** - Defines enabled content types
- **`content/locales/`** - Translation files and extracted keys
- **`scripts/content/templates/`** - Content generation templates

## 🔄 Integration with Build System

### Justfile Integration
The content scripts integrate with the project's task runner:

```bash
# Content management commands (updated for Nushell)
just content-generate-indices    # Generate all content indices
just content-validate           # Validate content structure
just content-consistency        # Check multi-language consistency
just content-build             # Build all localized content to HTML

# Content generation
just content-new-post           # Interactive blog post creation
just content-new-recipe         # Interactive recipe creation
```

### Build Pipeline Integration
Content scripts are used in the complete build pipeline:

1. **Content Generation** - Create new content from templates
2. **Validation** - Ensure content integrity and consistency
3. **Index Generation** - Update JSON indices for web serving
4. **Translation Sync** - Manage localization keys
5. **HTML Conversion** - Convert markdown to HTML (via build scripts)

## 🐛 Troubleshooting

### Common Issues

**Content Validation Errors:**
- Ensure `content-static` feature is available in Cargo.toml
- Check that source markdown files exist in correct directories
- Verify frontmatter syntax is valid YAML

**Index Generation Issues:**
- Run `nu scripts/content/content-manager.nu generate-indices` to rebuild
- Check that `content-kinds.toml` properly defines content types
- Ensure markdown files have required frontmatter fields

**Translation Synchronization:**
- Extract keys first: `nu scripts/content/sync-translations.nu extract-keys`
- Check that translation patterns match your content style
- Verify locales directory exists and is writable

**Multi-language Consistency:**
- Use consistent filenames across all languages
- Ensure frontmatter metadata matches across translations
- Run consistency validation after adding new content

### Validation Workflow

```bash
# Complete content validation workflow
nu scripts/content/validate-content.nu                    # Basic structure
nu scripts/content/validate-id-consistency.nu             # ID consistency
nu scripts/content/validate-content-consistency.nu        # Multi-language
nu scripts/content/content-manager.nu show-stats          # Overview
```

## 🔧 Original Bash Scripts

Original bash scripts are preserved in `scripts/sh/content/` for reference during the transition period. The Nushell versions provide:

- **Enhanced Error Handling** - Structured error reporting with actionable feedback
- **Better Data Processing** - Native JSON/YAML handling without external tools
- **Improved Performance** - Faster processing of large content sets
- **Cross-Platform Compatibility** - Consistent behavior across operating systems
- **Structured Output** - Rich, colored output with progress indicators

## 💡 Best Practices

1. **Always validate** content before building for production
2. **Use templates** for consistent content structure
3. **Maintain ID consistency** across all languages
4. **Regular translation sync** to keep localization up to date
5. **Check content statistics** to monitor content growth
6. **Update indices** after adding or modifying content

The content management system provides a complete workflow for maintaining high-quality, multi-language content with proper validation, consistency checks, and localization support.