DCTS/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

DCTS is a Rust-based distributed computing scheduler for stellar atmosphere modeling. It discretizes a multi-dimensional parameter grid (Teff, log g, log He, log C, log N, log O) into independent compute points, runs TLUSTY (atmosphere) + SYNSPEC (synthetic spectrum) physics binaries on distributed worker nodes, and orchestrates convergence via seed passing and divergence fallback. Fully documented (in Chinese) under docs/.

Commands

# Build / check
cargo build --release                 # binaries: server, node, import_results
cargo check --workspace --all-targets

# Tests (per crate)
cargo test --workspace
cargo test -p common                  # physics/input/conv-check unit tests
cargo test -p mq                      # SQLite queue concurrency + requeue tests
cargo test -p server                  # axum integration tests (api_tests.rs, wf_migration_isolation.rs)
cargo test -p server --test api_tests           # single integration test file
cargo test -p server --test api_tests test_name # single test by name filter

# Lint / format (must be clean before commit — see contributing.md)
cargo fmt --all -- --check
cargo clippy --workspace --all-targets -- -D warnings

# Dashboard (Vite + native ESM, Node built-in test runner)
cd dashboard && npm install
npm run dev                           # dev server
npm run build                         # outputs dashboard/dist (served by server on /)
npm test                              # node --test "test/*.test.js"

Runtime

./target/release/server --workflow workflows/sdB_cno.yaml --port 8090
./target/release/node                 # worker; reads .env, DCTS_SERVER_URL

All config is via environment variables prefixed DCTS_ (see .env.example): DCTS_ADMIN_TOKEN, DCTS_NODE_ID, DCTS_SERVER_URL, DCTS_MAX_SLOTS, DCTS_HEARTBEAT_SEC, DCTS_DB_PATH, DCTS_QUEUE_DB_PATH, DCTS_SEEDS_DIR, etc. dotenvy loads .env. With DCTS_AUTH_DISABLE=1 auth is skipped (local debug only).

Architecture

Workspace crates (Cargo.toml): common (lib), server (bin), node (bin), mq (lib), tools/import_results (bin), plus dashboard/ (web UI).

Master-Worker topology (pull-based)

• server — Axum HTTP service. main.rs builds the router + background loops (offline detection, stale-task requeue). db/ module (SQLite + r2d2): db/mod.rs holds the Database struct, connection/migration infra, shared types & the consolidated unit tests; db/grid.rs/db/nodes.rs/db/tasks.rs/db/seeds.rs/db/workflows.rs/db/snapshots.rs hold the per-domain impl Database methods. scheduler.rs expands the grid and pushes pending points into the MQ queue. api/ submodules: node.rs (register/heartbeat), task.rs (claim/report), seed.rs (.7 download), data.rs (binary/runtime deps), workflow.rs (CRUD + start/stop), status.rs, admin.rs, auth.rs, rate_limit.rs.
• node — stateless worker daemon. worker.rs is the poll loop (heartbeat + claim + concurrent slot pool). executor.rs runs the physics chain in a per-task sandbox. reporter.rs uploads results + .7 seed via multipart. Bootstrap pulls missing runtime deps from master.
• mq — SqliteTaskQueue (WAL mode): transactional Push/Claim/Report/Stale-Requeue. Claim is atomic so one task goes to one worker.
• common — physics engine: config.rs (YAML parsing), gen_input5.rs/fort55_writer.rs/nst_writer.rs (TLUSTY input streams), conv_check.rs (parses fort.6 log, judges convergence), runner.rs (async subprocess with timeout), seed_finder.rs (nearest-neighbor seed matching), models.rs (shared types/enums), embedded.rs (bootstrap), logging.rs.

Task scheduling — deliberate pull model

The scheduler only pushes pending grid points into the MQ queue; it does NOT decide which node gets a task. Workers actively claim work whenever active_slots < max_slots and stop claiming when full — this achieves natural load balancing in steady state. Dequeue order is wave ASC, created_at ASC (low CNO hard-sum first, then FIFO). Do not "improve" this to server-side load distribution unless the sparse-state tradeoffs are explicitly accepted (see docs/architecture.md §5).

Task lifecycle

pending → queued → running → completed / failed. Statuses map to GridPointStatus in common/src/models.rs (completed supersedes the old converged alias, still parsed). Running tasks that exceed the stale timeout (~1800s) are requeued back to pending.

Physics pipeline (TLUSTY/SYNSPEC)

Each grid point runs a strategy chain (tlusty_strategies in the workflow YAML, e.g. ["cold_run", "seed_step"]):

• Cold run (default_cold_chain in common/src/runner.rs): lte (grey LTE guess) → nc (NLTE continuum, ilvlin=0, niter=10) → nl (NLTE full lines, ilvlin=100, must converge) → synspec.
• Seed step (default_seed_chain): skip the grey LTE stage, hot-start from a nearby converged atmosphere .7 as fort.8: seed_nc → nl → synspec.

On nl non-convergence the server pops the chain head and retries with the next strategy (trigger_strategy_fallback). seed_step requires a resolvable neighboring seed before dispatch; if none exists the grid point stays pending and self-heals once a seed appears. Only failed tasks with an actual state transition trigger fallback (a 2026-08-02 fix prevents duplicate failure reports from re-triggering). Nodes are strategy-unaware — they only execute strategies[0] plus the injected seed_point_name.

Seed finding (common/src/seed_finder.rs)

The seed pool is a global shared resource (seeds table + in-memory cache, cross-workflow). exact_family buckets by quantized (teff/5000, logg, loghe); within a family, candidates are ranked by directed CNO distance — increasing metal abundance (target richer) is penalized 4×, decreasing (target poorer) 1×, because adding metals destabilizes NLTE radiation equilibrium (historically 3–11% success vs 42–54% for metal-poor direction). Global fallback uses d = Δteff/5000 + Δlogg×2 + Δloghe×0.5 + Δcno×0.1 with d ≤ 3.0.

Web dashboard (dashboard/)

Native ESM modules (no framework), Vite build. main.js/router.js/state.js/api.js are the shell; utils/ (format, polling, yamlStage, errors) and components/, views/ (home, workflowDetail, detail/ for per-point panels). Tests are pure Node scripts in dashboard/test/ run via node --test. State is centralized in state.js with polling from utils/polling.js.

Conventions & gotchas

• Convergence tuning is empirical — parameters like niter=10 for nc are the result of documented measurements (see workflows/sdB_cno.yaml comments and docs/spectrum_correctness_analysis.md). Don't change them casually; validate against the guide before altering.
• Update docs when you change contracts: API changes → docs/api.md; DB schema → docs/database.md; physics/scheduling → docs/design.md, docs/task_engine_decoupling_design.md. The repo treats these as living specs.
• Keep serde_yaml pinned to 0.9.34 (the upstream crate is archived; the workspace pins the final stable version deliberately).
• DB migrations live in crates/server/src/migrations.rs; old SQLite DBs are migrated seamlessly at startup (see docs/database_refactor_design.md).
• Commit messages follow Conventional Commits with scopes common/server/node/mq/dashboard/all (see docs/contributing.md).