fmq
d16b3d3cdc
- server/db: 拆 4929 行 db.rs 单体为 db/ 目录，migrations.rs 引入 PRAGMA user_version
版本化迁移运行器（M1~M13）
- 任务引擎 Phase 6/7b/7c 改名收敛：EngineStageConfig→PhaseConfig、StagePolicy→ResumePolicy、
Converged→Completed、删除 task_type 列、success_method 拆 tlusty_/synspec_ 双列、
新增 tlusty_status/synspec_status 半失败阶段守卫
- 科学正确性加固：conv_check 任意行 NaN/Inf/溢出判无效（0 行容忍）、新增 spec_is_valid
校验 SYNSPEC 脏谱、itek_history 逐次迭代全量保真、fmt_abn powf 溢出饱和
- 用户配置真正接通：tlusty_chain/tlusty_input 由死字段经 调度器→TaskSpec→executor→runner
透传生效；config 加载期 validate + deny_unknown_fields + 解析失败记 warn
- 调度修复：H1 活锁（pending_strategies 跳过已失败策略）、种子查找错误不再静默降级冷启动
- dashboard: 阶段配置面板 tlusty_stage/synspec_stage、"已完成"标签、迭代诊断展示
- docs: 新增 database_refactor_design.md，同步 database/api/PIPELINE/workflow_detail
86 lines
7.5 KiB
Markdown
86 lines
7.5 KiB
Markdown
# CLAUDE.md
|
||
|
||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||
|
||
## Project Overview
|
||
|
||
DCTS is a Rust-based distributed computing scheduler for stellar atmosphere modeling. It discretizes a multi-dimensional parameter grid (Teff, log g, log He, log C, log N, log O) into independent compute points, runs TLUSTY (atmosphere) + SYNSPEC (synthetic spectrum) physics binaries on distributed worker nodes, and orchestrates convergence via seed passing and divergence fallback. Fully documented (in Chinese) under `docs/`.
|
||
|
||
## Commands
|
||
|
||
```bash
|
||
# Build / check
|
||
cargo build --release # binaries: server, node, import_results
|
||
cargo check --workspace --all-targets
|
||
|
||
# Tests (per crate)
|
||
cargo test --workspace
|
||
cargo test -p common # physics/input/conv-check unit tests
|
||
cargo test -p mq # SQLite queue concurrency + requeue tests
|
||
cargo test -p server # axum integration tests (api_tests.rs, wf_migration_isolation.rs)
|
||
cargo test -p server --test api_tests # single integration test file
|
||
cargo test -p server --test api_tests test_name # single test by name filter
|
||
|
||
# Lint / format (must be clean before commit — see contributing.md)
|
||
cargo fmt --all -- --check
|
||
cargo clippy --workspace --all-targets -- -D warnings
|
||
|
||
# Dashboard (Vite + native ESM, Node built-in test runner)
|
||
cd dashboard && npm install
|
||
npm run dev # dev server
|
||
npm run build # outputs dashboard/dist (served by server on /)
|
||
npm test # node --test "test/*.test.js"
|
||
```
|
||
|
||
## Runtime
|
||
|
||
```bash
|
||
./target/release/server --workflow workflows/sdB_cno.yaml --port 8090
|
||
./target/release/node # worker; reads .env, DCTS_SERVER_URL
|
||
```
|
||
|
||
All config is via environment variables prefixed `DCTS_` (see `.env.example`): `DCTS_ADMIN_TOKEN`, `DCTS_NODE_ID`, `DCTS_SERVER_URL`, `DCTS_MAX_SLOTS`, `DCTS_HEARTBEAT_SEC`, `DCTS_DB_PATH`, `DCTS_QUEUE_DB_PATH`, `DCTS_SEEDS_DIR`, etc. `dotenvy` loads `.env`. With `DCTS_AUTH_DISABLE=1` auth is skipped (local debug only).
|
||
|
||
## Architecture
|
||
|
||
Workspace crates (`Cargo.toml`): `common` (lib), `server` (bin), `node` (bin), `mq` (lib), `tools/import_results` (bin), plus `dashboard/` (web UI).
|
||
|
||
### Master-Worker topology (pull-based)
|
||
|
||
- **`server`** — Axum HTTP service. `main.rs` builds the router + background loops (offline detection, stale-task requeue). `db/` module (SQLite + r2d2): `db/mod.rs` holds the `Database` struct, connection/migration infra, shared types & the consolidated unit tests; `db/grid.rs`/`db/nodes.rs`/`db/tasks.rs`/`db/seeds.rs`/`db/workflows.rs`/`db/snapshots.rs` hold the per-domain `impl Database` methods. `scheduler.rs` expands the grid and pushes pending points into the MQ queue. `api/` submodules: `node.rs` (register/heartbeat), `task.rs` (claim/report), `seed.rs` (.7 download), `data.rs` (binary/runtime deps), `workflow.rs` (CRUD + start/stop), `status.rs`, `admin.rs`, `auth.rs`, `rate_limit.rs`.
|
||
- **`node`** — stateless worker daemon. `worker.rs` is the poll loop (heartbeat + claim + concurrent slot pool). `executor.rs` runs the physics chain in a per-task sandbox. `reporter.rs` uploads results + `.7` seed via multipart. Bootstrap pulls missing runtime deps from master.
|
||
- **`mq`** — `SqliteTaskQueue` (WAL mode): transactional Push/Claim/Report/Stale-Requeue. Claim is atomic so one task goes to one worker.
|
||
- **`common`** — physics engine: `config.rs` (YAML parsing), `gen_input5.rs`/`fort55_writer.rs`/`nst_writer.rs` (TLUSTY input streams), `conv_check.rs` (parses fort.6 log, judges convergence), `runner.rs` (async subprocess with timeout), `seed_finder.rs` (nearest-neighbor seed matching), `models.rs` (shared types/enums), `embedded.rs` (bootstrap), `logging.rs`.
|
||
|
||
### Task scheduling — deliberate pull model
|
||
|
||
The scheduler only pushes pending grid points into the MQ queue; it does NOT decide which node gets a task. Workers actively claim work whenever `active_slots < max_slots` and stop claiming when full — this achieves natural load balancing in steady state. Dequeue order is `wave ASC, created_at ASC` (low CNO hard-sum first, then FIFO). Do not "improve" this to server-side load distribution unless the sparse-state tradeoffs are explicitly accepted (see `docs/architecture.md §5`).
|
||
|
||
### Task lifecycle
|
||
|
||
`pending → queued → running → completed / failed`. Statuses map to `GridPointStatus` in `common/src/models.rs` (`completed` supersedes the old `converged` alias, still parsed). Running tasks that exceed the stale timeout (~1800s) are requeued back to `pending`.
|
||
|
||
## Physics pipeline (TLUSTY/SYNSPEC)
|
||
|
||
Each grid point runs a **strategy chain** (`tlusty_strategies` in the workflow YAML, e.g. `["cold_run", "seed_step"]`):
|
||
|
||
- **Cold run** (`default_cold_chain` in `common/src/runner.rs`): `lte` (grey LTE guess) → `nc` (NLTE continuum, `ilvlin=0, niter=10`) → `nl` (NLTE full lines, `ilvlin=100`, must converge) → `synspec`.
|
||
- **Seed step** (`default_seed_chain`): skip the grey LTE stage, hot-start from a nearby converged atmosphere `.7` as `fort.8`: `seed_nc` → `nl` → `synspec`.
|
||
|
||
On `nl` non-convergence the server pops the chain head and retries with the next strategy (`trigger_strategy_fallback`). `seed_step` requires a resolvable neighboring seed before dispatch; if none exists the grid point stays pending and self-heals once a seed appears. Only `failed` tasks with an actual state transition trigger fallback (a 2026-08-02 fix prevents duplicate failure reports from re-triggering). Nodes are strategy-unaware — they only execute `strategies[0]` plus the injected `seed_point_name`.
|
||
|
||
### Seed finding (`common/src/seed_finder.rs`)
|
||
|
||
The seed pool is a global shared resource (seeds table + in-memory cache, cross-workflow). `exact_family` buckets by quantized `(teff/5000, logg, loghe)`; within a family, candidates are ranked by **directed CNO distance** — increasing metal abundance (target richer) is penalized 4×, decreasing (target poorer) 1×, because adding metals destabilizes NLTE radiation equilibrium (historically 3–11% success vs 42–54% for metal-poor direction). Global fallback uses `d = Δteff/5000 + Δlogg×2 + Δloghe×0.5 + Δcno×0.1` with `d ≤ 3.0`.
|
||
|
||
## Web dashboard (`dashboard/`)
|
||
|
||
Native ESM modules (no framework), Vite build. `main.js`/`router.js`/`state.js`/`api.js` are the shell; `utils/` (format, polling, yamlStage, errors) and `components/`, `views/` (home, workflowDetail, detail/ for per-point panels). Tests are pure Node scripts in `dashboard/test/` run via `node --test`. State is centralized in `state.js` with polling from `utils/polling.js`.
|
||
|
||
## Conventions & gotchas
|
||
|
||
- **Convergence tuning is empirical** — parameters like `niter=10` for `nc` are the result of documented measurements (see `workflows/sdB_cno.yaml` comments and `docs/spectrum_correctness_analysis.md`). Don't change them casually; validate against the guide before altering.
|
||
- **Update docs when you change contracts**: API changes → `docs/api.md`; DB schema → `docs/database.md`; physics/scheduling → `docs/design.md`, `docs/task_engine_decoupling_design.md`. The repo treats these as living specs.
|
||
- **Keep `serde_yaml` pinned to 0.9.34** (the upstream crate is archived; the workspace pins the final stable version deliberately).
|
||
- **DB migrations** live in `crates/server/src/migrations.rs`; old SQLite DBs are migrated seamlessly at startup (see `docs/database_refactor_design.md`).
|
||
- Commit messages follow Conventional Commits with scopes `common`/`server`/`node`/`mq`/`dashboard`/`all` (see `docs/contributing.md`). |