No description
  • Rust 96.4%
  • PLpgSQL 1.5%
  • Shell 0.7%
  • Nushell 0.4%
  • Dockerfile 0.3%
  • Other 0.7%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
claude-bot 7b911c65e8
Some checks are pending
CI / Format (push) Waiting to run
CI / Clippy + Test + Deny (push) Waiting to run
fix(mcp, processor): mcp authorization and job queue work, stubs are gone, docs match the code
librosys-mcp
- test-webhook panicked on parse: --secret and --signature shared -s.
- Logs went to stdout with the MCP JSON-RPC responses; they go to stderr.
- serve logged DATABASE_URL with its password; it logs database, host, port.
- serve ended when stdin closed, taking the webhook and worker with it; they
  now run until SIGINT/SIGTERM and serve returns their errors.
- The webhook refused Gitea's native X-Gitea-Signature (bare hex); bare hex
  and sha256=<hex> are accepted.
- Jobs dequeued NORMAL, LOW, HIGH (text order); priority is ranked explicitly.
- Cedar evaluated with no entities and no context, so no Role policy could
  match: cedar::decide builds User with Role parents from AuthzRequest
  principal_roles (authz_middleware passes the JWT roles) and turns the
  request context into the Cedar context. All active policy rows load, and an
  API key is looked up by its own hash.
- REST handlers extracted Extension<Extensions>, which is never inserted;
  they extract Extensions (router test: 401 without user, 500 before).
- Migration 004 could not complete after 003 (index name clash, api_keys
  shape, VARCHAR/TEXT function overloads); it alters 003's api_keys in place
  and is re-runnable. 005 converts TIMESTAMP columns to TIMESTAMPTZ, which the
  code decodes. Neither was run against PostgreSQL.
- Cedar resource, JWT audience and server name: libro-mcp -> librosys-mcp.

librosys-processor
- Built-in editorial rule messages are overridable by rule id
  (analysis.messages; unknown ids refused).
- docx_reader: list level, bullet vs numbered, and real table size.
- hybrid_extractor::merge_sources inserts PDF text the DOCX lacks as text
  boxes in reading order, skipping page furniture and hyphenation noise.

Removed: the DD-Pasos deployment (k8s, Tekton, Istio,
LINUX_PIPELINE_DESIGN.md), the first work's key-terms.toml, the unread
.libro.toml.example, a stale member Cargo.lock and .dockerignore.
READMEs, docs, Dockerfiles and setup_security.sh rewritten against --help
and the source.
2026-09-17 17:29:50 +01:00
.cargo chore: config update 2026-09-17 12:33:48 +01:00
.config docs: record content-type taxonomy ADR and update README/CHANGELOG 2026-04-08 23:57:45 +01:00
.github/workflows chore: initial librosys workspace 2026-04-08 01:31:42 +01:00
.woodpecker docs: record content-type taxonomy ADR and update README/CHANGELOG 2026-04-08 23:57:45 +01:00
crates fix(mcp, processor): mcp authorization and job queue work, stubs are gone, docs match the code 2026-09-17 17:29:50 +01:00
justfiles docs: record content-type taxonomy ADR and update README/CHANGELOG 2026-04-08 23:57:45 +01:00
scripts/nu chore(justfile): agent git recipes delegate to the constellation root 2026-09-17 13:12:32 +01:00
.clippy.toml chore: initial librosys workspace 2026-04-08 01:31:42 +01:00
.gitignore chore(gitignore): the dev-internal overlay git-dir stays out of the public repo 2026-09-17 12:31:03 +01:00
.pre-commit-config.yaml chore: complete ontoref migrations 2026-04-08 04:28:37 +01:00
.rustfmt.toml chore: initial librosys workspace 2026-04-08 01:31:42 +01:00
.taplo.toml chore: initial librosys workspace 2026-04-08 01:31:42 +01:00
.yamllint-ci.yml chore: initial librosys workspace 2026-04-08 01:31:42 +01:00
Cargo.lock feat(config): consumer configuration is NCL, read through ncl-load 2026-09-17 12:29:22 +01:00
Cargo.toml fix(mcp, processor): mcp authorization and job queue work, stubs are gone, docs match the code 2026-09-17 17:29:50 +01:00
CHANGELOG.md fix(mcp, processor): mcp authorization and job queue work, stubs are gone, docs match the code 2026-09-17 17:29:50 +01:00
deny.toml chore: initial librosys workspace 2026-04-08 01:31:42 +01:00
justfile chore(justfile): agent git recipes delegate to the constellation root 2026-09-17 13:12:32 +01:00
README.md docs: record content-type taxonomy ADR and update README/CHANGELOG 2026-04-08 23:57:45 +01:00

librosys

Generic Rust framework for editorial and educational document pipelines. Provides reusable crates consumed via path dependency by implementation projects.

Architecture

librosys enforces a consumer-inversion constraint: framework crates contain zero references to any consumer project. Consumers implement the public traits and inject their logic via Arc<dyn Trait>.

Consumer project
    └── implements: ContentLoader, PedagogyEngine, ChapterConfig
    └── depends on: librosys-mcp, librosys-rag, librosys-suggestions
                         ↓                   ↓
                 librosys-rag-core     librosys-processor
                 (trait interfaces)    (ODT/DOCX/typst)

Crates

Crate Type Responsibility
librosys-rag-core lib EmbeddingProvider, VectorStore, Retriever trait interfaces
librosys-rag lib RAG implementation: fastembed (ONNX) + instant-distance vector index
librosys-mcp lib + bin MCP server framework; ContentLoader, PedagogyEngine, ChapterConfig public traits; Cedar authorization
librosys-processor lib + bin ODT/DOCX extraction, text corrections, typst generation, bookmarks, textboxes
librosys-suggestions lib + bin AI suggestion generation and merge: LanguageTool, Claude, OpenAI, n8n webhooks
librosys-watcher lib + bin File watcher; debounced git commits to author-edits; writes .watcher/status.json

Public API (stable traits)

These traits are the framework's public contract. Breaking changes require updating all consumers.

librosys-mcp

  • ContentLoader — load/query structured document sections
  • PedagogyEngine — pedagogy-aware content enrichment
  • ChapterConfig — chapter-level configuration injection

librosys-rag-core

  • EmbeddingProvider — text → Vec<f32>, with embed_batch support
  • VectorStore — insert, search, persist, load
  • Retriever — query interface over a VectorStore

Implementations live in the consuming project. The framework ships Mock* implementations for use in framework-level tests only.

Content Type Taxonomy

SectionContent carries six typed arrays that consumers populate and the framework indexes for RAG retrieval:

Type Key fields RAG use
concept term, definition, related_terms vocabulary lookup
exercise instruction, difficulty practice retrieval
reflection question, topic metacognitive prompts
example text, context illustrative retrieval
guideline behavior, indicator authoring rules
glossary-entry term, definition, chapter_origin cross-chapter glossary

Schemas live in reflection/schemas/. Adding a new type requires a schema file there and a new match arm in ingest.rs chunk routing. See ADR 0001.

Build

# Debug / release
cargo build --workspace
cargo build --workspace --release

# Tests (372+ tests, 6 pre-existing ignored)
cargo test --workspace
cargo test -p librosys-processor

# Quality gate
cargo clippy --workspace -- -D warnings
cargo fmt --all

# License and advisory check
cargo deny check

# Full CI gate (fmt-check + clippy + test + deny)
just ci

Shorthand aliases: just b (release build), just t (test), just c (clippy), just f (fmt).

Just recipes are organized into modules under justfiles/: build.just, ci.just, dev.just, test.just. The root justfile imports them all.

Features

librosys-suggestions exposes optional features to avoid pulling in unused HTTP clients:

librosys-suggestions = { path = "...", features = ["languagetool", "claude"] }
Feature Enables
core (default) Base suggestion types + reqwest
languagetool LanguageTool HTTP analyzer
claude Claude semantic analyzer
openai OpenAI analyzer
n8n n8n webhook dispatcher (HMAC signing)
all All of the above

RAG cache

The first run using librosys-rag downloads the ONNX model to fastembed_cache/. This directory is git-ignored and requires internet access on first use only.

Adding a consumer

  1. Add path dependencies in consumer Cargo.toml:
    librosys-mcp = { path = "../librosys/crates/librosys-mcp" }
    librosys-rag = { path = "../librosys/crates/librosys-rag" }
    
  2. Implement ContentLoader and PedagogyEngine in the consumer crate.
  3. Verify consumer still compiles after framework changes:
    cd ../your-consumer && cargo build --workspace 2>&1 | grep "^error"
    

License

MIT — see Cargo.toml workspace metadata.