adr(025): Phase 4 findings — eager function-body parse + transitional fallback
Two amendments from Phase 4 implementation (2026-04-17): 1. New rationale: Nushell parses use statements inside function bodies at module-load time, not call time. Subprocess boundary is the only true lazy-load mechanism. Confirmed empirically. 2. Single-entry provisioning-cli.nu tested and rejected for hot paths (3.1s vs 0.08-0.15s thin handlers). All 15 dispatcher wrappers fire at module-load time regardless of which command runs. 3. Constraints amended: bash-wrapper-has-no-runner-reference updated to permit provisioning-cli.nu fallback; new universal-fallback-is- transitional constraint documents 22 unmapped commands as migration debt; every-registry-command-has-thin-handler made directional.
This commit is contained in:
Jesús Pérez
parent
334f351fc5
commit
8e721582a7
GPG key ID:
9F243E355E0BC939
1 changed files with 168 additions and 0 deletions
168
adrs/adr-025-unified-lazy-loading.ncl
Normal file
168
adrs/adr-025-unified-lazy-loading.ncl
Normal file
|
|
@ -0,0 +1,168 @@
|
|||
let d = import "adr-defaults.ncl" in
|
||||
|
||||
d.make_adr {
|
||||
id = "adr-025",
|
||||
title = "Unified Lazy-Loading: Empty mod.nu + Selective Imports + Bash-Direct Dispatch",
|
||||
status = 'Accepted,
|
||||
date = "2026-04-17",
|
||||
amended = "2026-04-17",
|
||||
amendment_summary = "Phase 4 implementation findings: (1) function-body `use` is eager in Nushell 0.112 — subprocess boundary is the only true lazy-load; (2) hard-fail default replaced with provisioning-cli.nu universal fallback for unmapped commands (transitional); (3) constraint bash-wrapper-has-no-runner-reference updated to permit provisioning-cli.nu; (4) added universal-fallback-is-transitional constraint documenting migration debt.",
|
||||
|
||||
context = "The `prvng` CLI has two incompatible module-loading modes coexisting in `provisioning/core/nulib/`. Fat mode: commands routed through `core/nulib/provisioning` (the Nu runner, 492 lines) which star-imports `lib_provisioning *` and `main_provisioning *` at top level — this forces Nushell's parser to resolve every `export use X *` transitively through each `mod.nu`, parsing the AST of all 466 reachable .nu files on every invocation. Cold start: ~60s. Thin mode: 13 ad-hoc handlers (`provisioning-<cmd>.nu`) with selective imports, bypassing the runner. Cold start: <1s. The asymmetry is structural: `lib_provisioning/mod.nu` re-exports 13 subtrees via `export use X *`, and `main_provisioning/mod.nu` re-exports ~30 command modules the same way. Every new command risks landing in the fat path by default. ADR-022 (ncl-sync) eliminated Nickel export latency but explicitly deferred the module-parse problem as orthogonal; this ADR addresses that remaining cost. A related constraint: the bash wrapper `core/cli/provisioning` already resolves three alias layers (single-char expansion, multi-letter case aliases, registry-declared aliases) and five special argument-shaping cases (subcommand order inversion for `create|new`, `ssh` shortcut with implicit `--run`, `volume` positional skip, `platform logs` interactive stdin, `server <sub> --help` inline interception) before Nu is invoked. Moving alias resolution into Nu would reintroduce the exact double-parse cost this ADR exists to eliminate.",
|
||||
|
||||
decision = "Adopt a single loading mode: lazy-by-default, enforced structurally. (1) Root module indices `lib_provisioning/mod.nu` and `main_provisioning/mod.nu` contain no `export use` statements — only a comment documenting the no-star policy. (2) Every Nu file in `core/nulib/` uses selective imports exclusively: `use lib_provisioning/<path>/<file>.nu [fn1 fn2]` — never `use X *` against the root. A transitivity rule is part of the contract: any file imported selectively must itself use only selective imports; a selective import against a file containing `use ../other.nu *` is not thin because the star propagates during parse. (3) All selective imports use absolute paths from `nulib/` root, never relative paths (`../`, `./`). This preserves the work across file movements in the follow-up restructure (ADR-026): a single `rg | sed` suffices to relocate importers when files move, whereas relative imports would require per-file recalculation. (4) The Nu runner `core/nulib/provisioning` is deleted in full. Dispatch becomes the responsibility of the bash wrapper alone. (5) Every top-level command in `commands-registry.ncl` has a dedicated thin handler `core/nulib/provisioning-<cmd>.nu` following the canonical pattern in `provisioning-status.nu`. (6) The bash wrapper's three alias layers and five special argument-shaping cases are preserved verbatim — only the dispatch target inside each `case` branch changes from `$RUNNER` to the thin handler path. (7) The default branch of the bash dispatch becomes a hard failure (`exit 2`) — no silent fat-path fallback. (8) A pre-commit hook prevents reintroduction of root star-imports and relative-path imports.",
|
||||
|
||||
rationale = [
|
||||
{
|
||||
claim = "Nushell's parser resolves `export use X *` transitively at parse time — the cost is 100% upfront, not lazy",
|
||||
detail = "When Nu parses `use lib_provisioning *`, it walks every `export use Y *` declared in `lib_provisioning/mod.nu`, recursively, building the full AST of every reachable file before executing a single line. With 466 .nu files reachable from the two root mod.nu files, cold start reaches ~60s regardless of what the command actually needs. Selective imports (`use path/to/file.nu [fn]`) parse only that single file and its direct dependencies. The difference is structural, not a tuning parameter.",
|
||||
},
|
||||
{
|
||||
claim = "Function-body `use` statements are also parsed eagerly — the subprocess boundary is the only true lazy-load mechanism in Nushell 0.112",
|
||||
detail = "During Phase 4 implementation (2026-04-17), the 'lazy-inside-function' approach was tested: moving `use servers/create.nu *` inside a `match` arm so it would only fire when that arm executed. Empirical measurement showed zero improvement — calling the OTHER arm (which never reaches the `use`) still incurred the full 755ms parse cost of `create.nu`. Nushell parses all `use` statements in a file at module-load time regardless of lexical scope: top-level, `def` body, `match` arm, or `if` block. The only mechanism that achieves genuine lazy-loading is a subprocess boundary — a separate Nu process that sources only the module it needs. This is exactly the thin-handler pattern: each `provisioning-<cmd>.nu` is a subprocess that parses one domain module. Consequence: a single-entry Nu script that imports N domain modules via a dispatcher will always pay the cost of all N modules at startup, even with selective imports inside wrapper functions. The bash wrapper is the correct dispatch layer not merely for alias resolution but because it provides the process-per-command isolation that Nu cannot provide internally.",
|
||||
},
|
||||
{
|
||||
claim = "The bash wrapper is the correct dispatch layer — moving alias logic to Nu would reintroduce double parsing",
|
||||
detail = "Alias resolution is a lookup over command names. If the lookup table itself lives in a Nu module, Nu must parse enough of its module graph to access the table before it can decide which module to actually execute. That is the double parse this ADR eliminates. Bash resolves aliases in O(1) via `case` statements with no language runtime required. Until Nu implements lazy module resolution at the interpreter level, bash is the only place alias expansion can happen without paying the parse cost twice.",
|
||||
},
|
||||
{
|
||||
claim = "One dispatch model with enforced guardrails is safer than two with documentation",
|
||||
detail = "The existing 13 thin handlers are patches layered on top of the fat path. Every new command since their creation has been a coin-flip: contributors who read the pattern ship thin, contributors who don't ship fat. The fat path is invisible at PR review because it looks identical to a thin command from the outside. Structural enforcement (empty mod.nu + pre-commit hook blocking star-imports) makes regression syntactically impossible, not culturally discouraged.",
|
||||
},
|
||||
{
|
||||
claim = "Deleting the Nu runner removes a dispatch layer — bash → thin handler is one process, bash → runner → dispatch would be two",
|
||||
detail = "Every Nu process incurs ~200ms of interpreter startup plus module parse. A bash-to-runner-to-handler chain pays this twice: once for the runner (which must parse whatever it uses for dispatch), once for the handler. The canonical thin-handler pattern (`provisioning-status.nu`) already demonstrates that a single Nu process with selective imports completes in <1s including actual work. The runner adds no value: bash can already route by command name.",
|
||||
},
|
||||
{
|
||||
claim = "Transitivity rule is load-bearing — without it, selective imports decay silently",
|
||||
detail = "A developer writes `use lib_provisioning/utils/settings.nu [load_settings]` believing it is thin. If `settings.nu` contains `use ../providers/registry.nu *`, Nu parses the registry tree as well. The caller's import was selective but the transitive reach is not. Without a rule that requires all imported files to themselves be star-free, the refactor regresses incrementally as utility files gain new star imports. The rule is enforced by the same pre-commit hook that blocks root star-imports.",
|
||||
},
|
||||
{
|
||||
claim = "Absolute-path imports preserve this refactor's work across the follow-up restructure (ADR-026)",
|
||||
detail = "Relative imports encode the caller's location. `use ../utils/settings.nu [fn]` means different files depending on where the caller lives. When ADR-026 moves `lib_provisioning/utils/` to `primitives/utils/`, every relative importer breaks and must be individually recalculated. Absolute imports encode the target's location. `use lib_provisioning/utils/settings.nu [fn]` always refers to the same file; when the target moves, a single `rg -l 'lib_provisioning/utils/' | xargs sed -i 's|lib_provisioning/utils/|primitives/utils/|g'` updates every caller uniformly. This turns ADR-025 into structural groundwork that ADR-026 consumes, not a tax paid twice.",
|
||||
},
|
||||
{
|
||||
claim = "ADR-022 (ncl-sync) and this ADR are complementary and both required",
|
||||
detail = "ADR-022 eliminated Nickel export latency (2-5s per call × 124 call sites). This ADR eliminates Nu module parse latency (~60s for fat commands). Together they bring `prvng component list` from ~70s cold start to <1s, and `prvng deploy` from 15-30s to <5s. Neither alone is sufficient; ncl-sync only helps commands that read Nickel, module parse dominates every command regardless.",
|
||||
},
|
||||
],
|
||||
|
||||
consequences = {
|
||||
positive = [
|
||||
"Every `prvng <cmd> --help` completes in <1s cold start — uniform UX across the command surface",
|
||||
"New commands cannot accidentally land in a slow path: the fat path no longer exists",
|
||||
"Dispatch topology is explicit in the bash wrapper — grepping `case` statements enumerates the full CLI surface",
|
||||
"Pre-commit hook blocks star-import regressions mechanically; code review does not carry the burden",
|
||||
"Deleting the Nu runner removes 492 lines of duplicated dispatch logic and its module-import side effects",
|
||||
"Thin handlers are small (~30-60 lines) and follow a single template — contributor onboarding cost drops",
|
||||
],
|
||||
negative = [
|
||||
"Every command in `commands-registry.ncl` must have a corresponding thin handler; adding a command requires two artifacts (registry entry + handler file) instead of one (registry entry alone, historically)",
|
||||
"The `export-env` side effects currently triggered by star-imports in `lib_provisioning/cmd/env.nu` and `lib_provisioning/providers/registry.nu` must be invoked explicitly by handlers that depend on them — a per-handler decision, not an ambient guarantee",
|
||||
"Bash is now load-bearing for alias and argument shaping; contributors who prefer Nu must understand that the wrapper cannot be replaced without solving Nu's module-parse problem first",
|
||||
"The `platform logs` branch requires a dedicated thin handler that preserves interactive stdin (no `</dev/null` redirect) — one more per-case quirk to maintain",
|
||||
],
|
||||
},
|
||||
|
||||
alternatives_considered = [
|
||||
{
|
||||
option = "Leave the fat path in place and document it as the slow path",
|
||||
why_rejected = "The fat path is not reachable through documentation alone — it is the default route for any command not explicitly listed in the bash dispatch. Documentation does not prevent regression; structural enforcement does. Also, the fat path parses 466 files to run `--help`, which is indefensible regardless of how it is labeled.",
|
||||
},
|
||||
{
|
||||
option = "Replace the bash wrapper with a Nushell dispatcher to unify the language",
|
||||
why_rejected = "A Nu dispatcher must itself be parsed before it can dispatch. If it uses selective imports only, it can be fast — but so can the bash wrapper, and bash has lower startup cost (fork+exec vs interpreter init). Critically, the Nu dispatcher must either hard-code the alias table (couples it to registry changes) or read the registry (requires parsing enough Nu modules to have a file reader and JSON parser available before dispatch). The bash wrapper already has this via `grep` over the JSON cache with zero interpreter startup. Until Nu offers lazy module resolution, the bash wrapper is structurally superior for dispatch.",
|
||||
},
|
||||
{
|
||||
option = "Keep the Nu runner as a thin dispatcher that execs `provisioning-<cmd>.nu` per command",
|
||||
why_rejected = "This introduces a two-process chain (bash → Nu runner → Nu handler) with two interpreter startups. The runner adds no dispatch intelligence the bash wrapper lacks — it is pure indirection. Every case where the runner would decide `exec provisioning-<X>.nu`, the bash wrapper can already route directly with a `case` statement. Deleting the runner removes a parse step without losing any functionality.",
|
||||
},
|
||||
{
|
||||
option = "Single Nu entry `provisioning-cli.nu` routing all commands through `dispatch_command` (tested during Phase 4)",
|
||||
why_rejected = "Implemented and benchmarked: cold start 3.1s vs 0.08–0.15s for thin handlers (20x slower for hot paths). Root cause: `dispatcher.nu` declares 15 `_dispatch_*` wrapper functions, each with a `use commands/X.nu *` inside. Nushell parses all 15 `use` statements at module-load time regardless of which wrapper fires — confirmed empirically by calling branch `b` of a match block that only contains `use` in branch `a` and observing unchanged parse cost. The dispatcher's 15 domain modules are therefore always parsed, making single-entry cold start structurally irreducible. provisioning-cli.nu is retained as a universal fallback for unmapped commands (accepting the 3s cost for rare commands) but is not viable as the primary hot-path route. The thin-handler pattern remains the only way to achieve sub-200ms cold start for frequently-invoked commands.",
|
||||
},
|
||||
{
|
||||
option = "Migrate high-traffic commands to Rust binaries",
|
||||
why_rejected = "Orthogonal to module-loading cost and disproportionate to the problem. The Nu ecosystem provides structured-data pipelines that the CLI depends on (tables, pipelines, cell paths). Rewriting in Rust would either duplicate that runtime or abandon it. ADR-022 already demonstrated that targeted Rust daemons (ncl-sync) solve specific bottlenecks without wholesale rewrites. Module parse is not a bottleneck Rust solves — it is a Nu parser behavior that selective imports address natively.",
|
||||
},
|
||||
{
|
||||
option = "Feature-flag the new mode and ship both side by side during a migration window",
|
||||
why_rejected = "Violates the rationale that 'one dispatch model with enforced guardrails is safer than two with documentation'. A feature flag is dual-mode coexistence with a runtime switch — the exact shape of the current problem. The refactor is reversible via `git reset --hard origin/nickel` in `prvng_core.git`; a flag adds complexity without adding recoverability that the remote baseline does not already provide.",
|
||||
},
|
||||
],
|
||||
|
||||
constraints = [
|
||||
{
|
||||
id = "no-root-star-imports-in-core-nulib",
|
||||
claim = "No .nu file in provisioning/core/nulib/ may contain `use lib_provisioning *`, `use main_provisioning *`, or an equivalent relative-path star-import at the module root",
|
||||
scope = "provisioning/core/nulib/",
|
||||
severity = 'Hard,
|
||||
check = { tag = 'Grep, pattern = "^use (lib_provisioning|main_provisioning|\\.\\.?/) \\*$", paths = ["provisioning/core/nulib/"], must_be_empty = true },
|
||||
rationale = "Root star-imports force Nushell to parse the transitive AST of ~466 files at invocation. Selective imports (`use path/file.nu [fn1 fn2]`) parse only the target file. The empty mod.nu contract is unenforceable without this constraint on every consumer.",
|
||||
},
|
||||
{
|
||||
id = "mod-nu-files-contain-no-export-use",
|
||||
claim = "lib_provisioning/mod.nu and main_provisioning/mod.nu must not contain any `export use` statement",
|
||||
scope = "provisioning/core/nulib/lib_provisioning/mod.nu, provisioning/core/nulib/main_provisioning/mod.nu",
|
||||
severity = 'Hard,
|
||||
check = { tag = 'Grep, pattern = "^export use ", paths = ["provisioning/core/nulib/lib_provisioning/mod.nu", "provisioning/core/nulib/main_provisioning/mod.nu"], must_be_empty = true },
|
||||
rationale = "A single `export use X *` at the root reintroduces the transitive parse regardless of how selective individual consumers are. The mod.nu files are the structural guarantee that star-imports are architecturally absent.",
|
||||
},
|
||||
{
|
||||
id = "no-nu-runner-script",
|
||||
claim = "The file provisioning/core/nulib/provisioning (the Nu runner) must not exist",
|
||||
scope = "provisioning/core/nulib/",
|
||||
severity = 'Hard,
|
||||
check = { tag = 'FileExists, path = "provisioning/core/nulib/provisioning", present = false },
|
||||
rationale = "The Nu runner's only role was to perform dispatch after star-importing both root modules. With star-imports removed and dispatch delegated to the bash wrapper, the runner has no remaining responsibility. Its presence would signal an unresolved fat path and invite regression.",
|
||||
},
|
||||
{
|
||||
id = "bash-wrapper-has-no-runner-reference",
|
||||
claim = "provisioning/core/cli/provisioning must not reference the legacy Nu runner path `provisioning/core/nulib/provisioning` (without extension)",
|
||||
scope = "provisioning/core/cli/provisioning",
|
||||
severity = 'Hard,
|
||||
check = { tag = 'Grep, pattern = "core/nulib/provisioning[^-.]", paths = ["provisioning/core/cli/provisioning"], must_be_empty = true },
|
||||
rationale = "The legacy Nu runner (`nulib/provisioning`) star-imported both root modules and is deleted. Any reference to it reintroduces the fat path. NOTE: `provisioning-cli.nu` (with extension) is the permitted universal fallback for commands not yet assigned a dedicated thin handler — see constraint `universal-fallback-is-transitional`. The original target state specified a hard-fail `exit 2` default; the transitional state uses `provisioning-cli.nu` as fallback to avoid breaking the 22 registry commands that do not yet have thin handlers. The hard-fail target remains the end goal once all registry commands are wired.",
|
||||
},
|
||||
{
|
||||
id = "every-registry-command-has-thin-handler",
|
||||
claim = "Every top-level command in commands-registry.ncl must be routed explicitly by the bash wrapper to a thin handler or to provisioning-cli.nu. The count of commands routed to provisioning-cli.nu must not increase — only decrease as thin handlers are added.",
|
||||
scope = "provisioning/core/nulib/commands-registry.ncl, provisioning/core/nulib/provisioning-*.nu, provisioning/core/cli/provisioning",
|
||||
severity = 'Hard,
|
||||
check = { tag = 'FileExists, path = "provisioning/core/nulib/provisioning-status.nu", present = true },
|
||||
rationale = "The registry is the authoritative command list. As of 2026-04-17, 18 of 40 commands have dedicated thin handlers; 22 fall through to provisioning-cli.nu (3s cold start). The constraint is directional: each new command must be assigned a thin handler, and existing commands may migrate from cli.nu to a handler but not the reverse. The canonical reference is provisioning-status.nu; all handlers follow its selective-import pattern.",
|
||||
},
|
||||
{
|
||||
id = "universal-fallback-is-transitional",
|
||||
claim = "provisioning-cli.nu serves as the universal fallback for registry commands not yet assigned dedicated thin handlers. Its use in the bash wrapper `*)` catch-all and in any named command case is transitional — it must be replaced with a thin handler before a command is considered fully wired.",
|
||||
scope = "provisioning/core/nulib/provisioning-cli.nu, provisioning/core/cli/provisioning",
|
||||
severity = 'Soft,
|
||||
check = { tag = 'FileExists, path = "provisioning/core/nulib/provisioning-cli.nu", present = true },
|
||||
rationale = "provisioning-cli.nu imports the full dispatcher surface at function-call time (~3s cold start vs ~0.15s for a thin handler). It exists because hard-failing 22 unwired commands during the migration was worse than accepting 3s for those commands temporarily. Its cold-start cost is irreducible given Nushell's eager function-body parse: the dispatcher imports 15 domain modules, each parsed at load time regardless of which command is invoked. Any command that stays in the `*)` catch-all permanently is accepting a 20x performance penalty relative to the thin-handler baseline. provisioning-cli.nu is an explicit migration debt marker, not a permanent architecture.",
|
||||
},
|
||||
{
|
||||
id = "alias-expansion-stays-in-bash",
|
||||
claim = "Single-char and multi-letter short alias expansion must live in provisioning/core/cli/provisioning — never in Nu code",
|
||||
scope = "provisioning/core/cli/provisioning, provisioning/core/nulib/",
|
||||
severity = 'Hard,
|
||||
check = { tag = 'FileExists, path = "provisioning/core/cli/provisioning", present = true },
|
||||
rationale = "Moving alias resolution into Nu reintroduces the double-parse cost: Nu must parse enough of the module graph to access the alias table before it can dispatch. Bash resolves aliases before Nu starts, preserving the <1s cold-start invariant. This is the structural reason the bash wrapper is preserved rather than replaced.",
|
||||
},
|
||||
{
|
||||
id = "absolute-path-imports-only",
|
||||
claim = "All selective imports in provisioning/core/nulib/ must use absolute paths from nulib/ root — no relative paths (../ or ./)",
|
||||
scope = "provisioning/core/nulib/",
|
||||
severity = 'Hard,
|
||||
check = { tag = 'Grep, pattern = "^use \\.\\.?/", paths = ["provisioning/core/nulib/"], must_be_empty = true },
|
||||
rationale = "Relative imports couple each caller to the target's location at write time. When the follow-up restructure (ADR-026) moves files, every relative importer must be recalculated individually. Absolute imports decouple caller and target: a file move is a single rg+sed pass across the codebase. This constraint is what makes ADR-025's work reusable by ADR-026 instead of discarded.",
|
||||
},
|
||||
],
|
||||
|
||||
ontology_check = {
|
||||
decision_string = "Eliminate dual load-mode split in provisioning/core/nulib/ via empty mod.nu, selective imports only, bash-direct dispatch, one thin handler per registry command, structural guardrails via pre-commit hook",
|
||||
invariants_at_risk = ["config-driven-always"],
|
||||
verdict = 'Safe,
|
||||
},
|
||||
|
||||
related_adrs = ["adr-022-ncl-sync-daemon", "adr-023-ncl-export-wrapper"],
|
||||
}
|
||||
Loading…
Add table
Reference in a new issue