kogral/CHANGELOG.md

# Changelog

All notable changes to the Knowledge Base project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Added

#### SurrealDB 3.0 — Config-Driven Engine Abstraction (`surrealdb-backend` feature)

- **`SurrealEngineConfig`** tagged enum (`mem`, `surreal_kv`, `rocks_db`, `ws`) selects the
  engine at runtime via `surrealdb::engine::any::connect(url)` URL dispatch — no recompilation
  required to change backends.
- **`SurrealDbBackendConfig`** dual hot/cold layout: `graph` engine (default: SurrealKV B-tree)
  for nodes/edges/metadata, `hot` engine (default: RocksDB LSM) for embeddings and session logs.
- **`SurrealDbStorage::from_config()`** constructor replaces ad-hoc field initialization;
  `Surreal<Any>` unified type — no generics, no `Arc<RwLock<>>`.
- All SurrealDB CRUD routes through `serde_json::Value` (`to_json` / `from_json` helpers) to
  satisfy the `SurrealValue` trait bound without coupling domain models to the database library.
- `SurrealDbStorage::save_embedding()` and `log_session()` for hot-engine direct access.
- Removed `Database(#[from] surrealdb::Error)` variant from `KbError` — all SurrealDB errors
  propagate as `KbError::Storage(e.to_string())`.
- `surrealdb-backend` Cargo feature activates `kv-surrealkv`, `kv-rocksdb`, `protocol-ws`,
  `rustls` engine features on top of the workspace `kv-mem` base.

#### Storage Factory (`storage::factory`)

- **`storage::factory::build(config, base_path)`** — config-driven engine dispatch: SurrealDB
  when `secondary.enabled`, `MemoryStorage` when `primary == memory`, `FilesystemStorage`
  otherwise. Zero recompilation to switch backends.
- **`storage::factory::build_eventing(config, base_path, default_graph)`** — builds the base
  backend then wraps it with `EventingStorage` when `config.nats` is `Some`; falls through to
  `build` when NATS is not configured (`nats-events` feature required).
- **`impl Storage for Box<dyn Storage>`** — blanket delegation impl enabling
  `EventingStorage<Box<dyn Storage>>` without knowing the concrete backend type at compile time.

#### NATS JetStream Event Publishing (`nats-events` feature)

- **`KogralEvent`** typed enum (`NodeSaved`, `NodeDeleted`, `GraphSaved`) with computed NATS
  subject (`kogral.{graph}.node.saved`, etc.) and `serde` JSON serialization.
- **`EventPublisher`** wraps `platform_nats::EventStream` and publishes `KogralEvent` as JSON
  to JetStream after each mutation.
- **`EventingStorage<S>`** decorator wraps any `Storage` implementation with automatic event
  publishing after `save_graph`, `save_node`, and `delete_node` — inner backend unchanged.
- **`NatsEventConfig`** schema type with `From` conversion to `platform_nats::NatsConfig`.
- `nats-events` feature adds `platform-nats` and `bytes` workspace dependencies.

#### Orchestration Integration (`orchestration` feature)

- **`orchestration::pipeline_context_from_event(event, payload, state, schema_dir)`** bridges
  `KogralEvent` to `stratum-orchestrator`'s `PipelineContext::new` for downstream pipeline
  triggering.
- `orchestration` feature activates `stratum-orchestrator` and `stratum-state` workspace deps;
  implies `nats-events`.

#### Nickel Config Schema Updates

- `secondary.surrealdb` sub-record replaces flat `url`/`namespace`/`database` fields:
  `graph` and `hot` engine selectors with `engine`, `path`/`url` fields per engine type.
- `nats` top-level block in `StorageConfig` for optional JetStream configuration.
- `SurrealEngineValidator` contract and `SurrealEngineConfig` / `SurrealDbBackendConfig` /
  `NatsEventConfig` type contracts added to `schemas/kogral/contracts.ncl`.
- `schemas/kogral/defaults.ncl` updated with dual-engine defaults and `nats` disabled block.

#### Nickel Import Resolver

- `config::nickel::resolve_nickel_imports(path)` runs `ncl-import-resolver` on
  `resolver-manifest.json` (when present) before `nickel export`. No-op when manifest absent.

### Changed

#### BREAKING: MCP Protocol Rebranding
- **All MCP URIs, tool names, and prompts updated to KOGRAL namespace**
  - **URI Scheme**: `kb://` → `kogral://`
    - `kb://project/notes` → `kogral://project/notes`
    - `kb://project/decisions` → `kogral://project/decisions`
    - `kb://project/guidelines` → `kogral://project/guidelines`
    - `kb://project/patterns` → `kogral://project/patterns`
    - `kb://shared/guidelines` → `kogral://shared/guidelines`
    - `kb://shared/patterns` → `kogral://shared/patterns`
  - **Tool Names**: `kb/*` → `kogral/*`
    - `kb/search` → `kogral/search`
    - `kb/add_note` → `kogral/add_note`
    - `kb/add_decision` → `kogral/add_decision`
    - `kb/link` → `kogral/link`
    - `kb/get_guidelines` → `kogral/get_guidelines`
    - `kb/list_graphs` → `kogral/list_graphs`
    - `kb/export` → `kogral/export`
    - `kb/find_blocks` → `kogral/find_blocks`
    - `kb/find_todos` → `kogral/find_todos`
    - `kb/find_cards` → `kogral/find_cards`
  - **Prompt Names**: `kb/*` → `kogral/*`
    - `kb/summarize_project` → `kogral/summarize_project`
    - `kb/find_related` → `kogral/find_related`
  - **Server Name**: `kb-mcp` → `kogral-mcp`
  - **Migration Required**: Users must update `~/.config/claude/config.json`:

    ```diff
    {
      "mcpServers": {
    -   "kb-mcp": {
    -     "command": "/path/to/kb-mcp",
    +   "kogral-mcp": {
    +     "command": "/path/to/kogral-mcp",
    -       "env": { "KB_DIR": "/path/to/.kb" }
    +       "env": { "KOGRAL_DIR": "/path/to/.kogral" }
        }
      }
    }
    ```

  - **Rationale**: Complete branding consistency across all interfaces. Pre-release status (v0.x) allows breaking changes without user pain.
  - **Files updated**:
    - `crates/kb-mcp/src/resources.rs`: All URI definitions and parsing
    - `crates/kb-mcp/src/tools.rs`: All tool name definitions and dispatch
    - `crates/kb-mcp/src/prompts.rs`: All prompt name definitions and dispatch
    - `crates/kb-mcp/src/main.rs`: Server name registration
    - `crates/kb-mcp/tests/integration_test.rs`: Test assertions updated
    - `docs/apps/mcp-quickguide.md`: All examples and references
    - `docs/kogral/core-concepts.md`: MCP protocol documentation
    - `docs/api/mcp-tools.md`: API reference documentation
    - `README.md`: MCP integration examples

#### Branding
- **Rebranded to KOGRAL** (**KO**wledge **GRA**phs, **L**ocal-first)
  - Updated project name from "Knowledge Base" to "KOGRAL" across all documentation
  - New tagline: "Git-native knowledge graphs for developer teams"
  - Updated description emphasizing structured knowledge management for developers
  - Documentation folder renamed: `docs/kb/` → `docs/kogral/`
  - Documentation files renamed:
    - `docs/kogral/what-is-kb.md` → `what-is-kogral.md`
    - `docs/kogral/why-kb.md` → `why-kogral.md`
  - Technical names maintained for compatibility: `kb-core`, `kb-cli`, `kb-mcp` (crate names), `kb` (CLI binary)
  - Hybrid naming strategy: KOGRAL as primary brand, `kb.vapora.dev` as future alias
  - Files updated:
    - `README.md`: Title, description, and branding
    - `Cargo.toml` (workspace): Homepage, description, repository metadata
    - `crates/kb-core/Cargo.toml`: Package description
    - `crates/kb-cli/Cargo.toml`: Package description
    - `crates/kb-mcp/Cargo.toml`: Package description
    - `crates/kb-core/src/lib.rs`: Module documentation comments
    - `crates/kb-core/src/models.rs`: Type documentation comments
    - `crates/kb-cli/src/main.rs`: CLI description and messages
    - `docs/kogral/what-is-kogral.md`: Full KOGRAL branding with acronym explanation
    - `docs/kogral/core-concepts.md`: All "KB" references replaced with "KOGRAL"
    - `docs/README.md`: Title and section headers
    - `docs/book.toml`: mdBook title and description
    - `docs/SUMMARY.md`: Navigation titles
    - `.claude/CLAUDE.md`: Project description and documentation links

#### BREAKING: Filesystem Paths and Environment Variables
- **Directory structure renamed from `.kb/` to `.kogral/`**
  - Project directory: `.kb/` → `.kogral/`
  - Shared directory: `.kb-shared/` → `.kogral-shared/`
  - Configuration paths updated throughout codebase
  - All storage paths use `.kogral/` prefix
- **Environment variable renamed from `KB_DIR` to `KOGRAL_DIR`**
  - MCP server configuration requires `KOGRAL_DIR` environment variable
  - CLI tools respect `KOGRAL_DIR` for custom directory location
- **Migration Required**:
  1. Rename existing directory: `mv .kb .kogral`
  2. Update environment variables: `KB_DIR` → `KOGRAL_DIR`
  3. Update config files referencing `.kb/` paths
- **Rationale**: Complete branding consistency across all filesystem operations. Pre-release status allows breaking changes.
- **Files updated**:
  - All Rust source files in `crates/`: Path constants and variable names
  - `crates/kb-core/src/config/loader.rs`: Configuration path discovery
  - `crates/kb-core/src/storage/filesystem.rs`: Storage paths
  - `crates/kb-cli/src/main.rs`: CLI messages and examples
  - `crates/kb-mcp/src/main.rs`: Environment variable handling
  - All documentation in `docs/`: Examples and instructions
  - `README.md`: Setup and usage examples
  - `.claude/CLAUDE.md`: Project configuration paths

### Added

#### Core Infrastructure
- Initial project structure with Cargo workspace (kb-core, kb-cli, kb-mcp)
- Core graph models (Node, Edge, Graph) with 6 node types and 6 relationship types
- Config-driven architecture using Nickel schemas (NCL → JSON → Rust)
- Markdown parser with Logseq compatibility (YAML frontmatter + wikilinks)
- Storage backends: Filesystem (default), In-Memory, SurrealDB (feature-gated)
- Embedding integration via rig-core (OpenAI, Claude, Ollama) + fastembed local fallback
- Tera template engine for document generation and export

#### Logseq Blocks Support (ADR-004)
- **Block Data Structures** (`crates/kb-core/src/models.rs`):
  - `TaskStatus` enum: TODO, DOING, DONE, LATER, NOW, WAITING, CANCELLED
  - `BlockProperties` struct: tags, task status, custom properties, block/page references
  - `Block` struct: hierarchical content blocks with properties and children
  - `Node.blocks` field: Optional cached block structure (lazy parsing)
- **BlockParser** (`crates/kb-core/src/block_parser.rs`):
  - Bidirectional parser: Markdown outliner ↔ Block structures
  - Supports: task markers, #tags, custom properties (key:: value), ((block-refs)), [[page-refs]]
  - Hierarchical parsing based on indentation
  - Round-trip fidelity for Logseq import/export
  - 15 comprehensive tests covering parsing, serialization, and round-trips
- **Block Query API**:
  - `Node::get_blocks()`: Lazy parsing from content with caching
  - `Node::find_blocks_by_tag()`: Find blocks with specific tag
  - `Node::find_all_todos()`: Find all task blocks
  - `Node::find_blocks_by_status()`: Find blocks by task status
  - `Node::find_blocks_by_property()`: Find blocks with custom property
  - `Graph::find_blocks_by_tag()`: Search blocks across all nodes
  - `Graph::find_all_todos()`: Find TODOs across entire graph
  - 6 integration tests for Node and Graph block queries
- **SurrealDB Schema** (`schemas/surrealdb/blocks.surql`):
  - `block` table with indexes on node_id, block_id, tags, status, parent_id
  - Full-text search index on content
  - Support for hierarchical queries and tag-based filtering
  - Migration queries and example usage patterns
- **Logseq Import/Export** (`crates/kb-core/src/import/logseq.rs`, `crates/kb-core/src/export/logseq.rs`):
  - Round-trip import: Logseq markdown → KB Node with blocks
  - Property parsing: key:: value format, tags ([[tag]] and #tag), references, timestamps
  - Export to Logseq page/journal formats with block serialization
  - 12 comprehensive tests covering import/export round-trips
- **MCP Block Tools** (`crates/kb-mcp/src/tools.rs`):
  - `kb/find_blocks`: Search blocks by tag, task status, or custom property
  - `kb/find_todos`: Find all TODO blocks across KB
  - `kb/find_cards`: Find all flashcard blocks (#card tag)
  - Fixed borrowing issues with two-phase data collection pattern
- **Configuration Schema** (`schemas/kb-config.ncl`, `config/*.ncl`):
  - Added `BlocksConfig` schema with 4 settings:
    - `enabled`: Enable/disable Logseq blocks parsing (default: false, opt-in)
    - `parse_on_import`: Auto-parse blocks from Logseq imports (default: true)
    - `serialize_on_export`: Serialize blocks to outliner format on export (default: true)
    - `enable_mcp_tools`: Enable block-related MCP tools (default: true)
  - Updated all config files (defaults.ncl, production.ncl, minimal.ncl)
  - Validated: All configs export to JSON successfully
- **Documentation**:
  - Complete technical design: `docs/architecture/logseq-blocks-design.md`
  - Architectural Decision Record: `docs/architecture/adrs/004-logseq-blocks-support.md`
  - 6-phase implementation plan documented
  - Use cases and examples for flashcards, task tracking, and block references

#### Applications
- CLI tool (kb-cli) with 13 commands (init, add, search, link, sync, serve, etc.)
- MCP server (kb-mcp) with 10 tools (7 core + 3 block tools), 6 resources, 2 prompts for Claude Code integration
- NuShell maintenance scripts (6 scripts: sync, backup, reindex, import/export Logseq, stats)

#### Configuration System (Nickel)
- Complete Nickel configuration schema following typedialog/provisioning pattern
- Schema contracts (`schemas/kb/contracts.ncl`) - Type definitions with validation
- Default values (`schemas/kb/defaults.ncl`) - Base configuration values
- Composition helpers (`schemas/kb/helpers.ncl`) - merge_with_override, compose_config
- Mode overlays (`schemas/kb/modes/{dev,prod,test}.ncl`) - Environment-specific tuning
- Runtime configurations (`.kb-config/core/kb.ncl`, `.kb-config/platform/*.ncl`)
- Double validation pattern: Nickel contracts + serde deserialization
- Config export to JSON for Rust consumption (`.kb-config/targets/*.json`)

#### Templates
- Tera templates for 6 document types (note, decision, guideline, pattern, journal, execution)
- 4 export format templates (Logseq page/journal, summary, JSON)
- Customizable template system with frontmatter generation

#### Documentation (mdBook)
- Complete mdBook documentation structure with SUMMARY.md navigation
- 4 SVG diagrams for visual guides:
  - `architecture-overview.svg` - Complete system architecture
  - `core-concepts.svg` - Node types and relationships
  - `config-composition.svg` - Configuration flow (Schema → Defaults → Mode → User → JSON)
  - `storage-architecture.svg` - Hybrid storage strategy
- 11 major documentation sections (kb/, guides/, architecture/, setup/, config/, storage/, ai/, templates/, cli/, apps/, api/, contributing/)
- ADR documentation (ADR-001: Nickel vs TOML, ADR-002: FastEmbed + AI providers, ADR-003: Hybrid storage)
- Use cases documentation (8 real-world scenarios)
- MCP quick guide (5-minute setup for Claude Code)
- Config-driven architecture deep dive
- Storage architecture deep dive with code examples
- docs/README.md - Comprehensive guide for using and building documentation

#### Build System
- Modular justfile system (7 modules: build, test, dev, ci, nickel, docs, scripts)
- Nickel validation and export recipes (`just nickel::validate-all`, `just nickel::export-all`)
- mdBook documentation recipes (`just docs::serve`, `just docs::build`, `just docs::test`)
- CI/CD pipeline recipes (format, lint, test, build validation)
- NuShell script validation and execution recipes

#### Project Documentation
- Enhanced README.md with complete documentation links and visual diagram references
- Directory structure documentation with detailed explanations
- Quick start guide with MCP integration
- Integration guides (Vapora, Logseq, Claude Code)
- CHANGELOG.md following Keep a Changelog format

### Changed
- Updated README.md with comprehensive documentation links (mdBook, guides, architecture, config)
- Enhanced directory structure documentation with .kb-config/ and schemas/kb/ details
- Improved documentation section with visual diagrams and just recipe references
- **Blocks Feature Documentation**:
  - README.md: Added Logseq blocks section with examples, features, and configuration
  - README.md: Updated architecture diagram to include BlockParser
  - README.md: Added 3 new MCP tools to integration section
  - docs/kb/core-concepts.md: Added comprehensive "Logseq Content Blocks" section with examples, use cases, and queries
  - docs/api/mcp-tools.md: Added "Block Tools" section documenting kb/find_blocks, kb/find_todos, kb/find_cards
  - docs/api/mcp-tools.md: Updated overview from 7 to 10 tools

### Infrastructure
- Cargo workspace with 3 crates
- Feature flags: `filesystem` (default), `surrealdb`, `fastembed`, `full`
- 90 tests across workspace (81 kb-core + 5 kb-mcp + 4 kb-cli)
  - 15 tests for block parser
  - 12 tests for Logseq import/export
  - 6 tests for block queries
- mdBook integration for documentation (serves on [http://localhost:3000](http://localhost:3000))
- Nickel → JSON → Rust configuration pipeline
- Justfile modules: build, test, dev, ci, nickel, docs, scripts
- CI/CD pipeline recipes (format, lint, test, build validation)
- mdBook documentation structure with SUMMARY.md
- NuShell script validation recipes

## [1.0.0] - TBD

Initial release (planned).

### Planned Features
- Full FilesystemStorage implementation with file watching
- Complete Query Engine with semantic search
- Guideline inheritance resolution (shared → local override priority)
- SurrealDB schema and complete backend implementation
- Sync mechanism: bidirectional Filesystem ↔ SurrealDB
- FastEmbed local embeddings integration
- API provider integrations (OpenAI, Claude, Ollama) for embeddings
- Cross-graph queries (project + shared knowledge bases)
- Template customization system
- Export to multiple formats (Logseq, JSON, Markdown)
- Import from Logseq graphs
- MCP protocol compliance testing
- Production-ready CLI tool
- Performance benchmarks
- Security audit

---

## Version History

### Version Numbering

This project follows semantic versioning (MAJOR.MINOR.PATCH):

- **MAJOR**: Incompatible API changes
- **MINOR**: New features (backward compatible)
- **PATCH**: Bug fixes (backward compatible)

### Release Schedule

- **Alpha** (0.x.x): API unstable, features incomplete
- **Beta** (1.0.0-beta.x): API stable, features complete, testing phase
- **Stable** (1.0.0+): Production ready

### Feature Roadmap

#### Phase 1: Foundation ✅ (Completed)
- [x] Core models and graph structure
- [x] Nickel configuration system
- [x] Markdown parser with Logseq compatibility
- [x] Storage trait abstraction
- [x] CLI structure with clap
- [x] MCP server protocol implementation
- [x] NuShell maintenance scripts
- [x] Documentation structure

#### Phase 2: Integration (In Progress)
- [ ] Full FilesystemStorage with file watching
- [ ] SurrealDB backend implementation
- [ ] Query engine with text + semantic search
- [ ] Embedding provider integrations
- [ ] Guideline inheritance resolution
- [ ] Sync mechanism implementation

#### Phase 3: Polish (Planned)
- [ ] Performance optimization
- [ ] Comprehensive error handling
- [ ] Production testing
- [ ] Security audit
- [ ] API stabilization
- [ ] Release preparation

---

## Migration Guides

### Upgrading from 0.x to 1.0

(To be documented when 1.0 is released)

---

## Contributing

See [CONTRIBUTING.md](docs/contributing/development.md) for development setup and guidelines.

---

## License

(To be determined)