Asfmq
49784739fa
架构重构： - Agent Runtime 由单文件拆为 runtime/ 目录 12 模块（熔断/流式执行/Token预算/文件缓存/权限等） - Agent Tools 由单文件拆为 tools/ 目录 20+ 模块（filesystem/astro/memory/skill/subagent/team 等） - 解析器体系重构（common.rs 836行变更），各解析器同步升级 - Download 服务重构（562行），反爬策略强化 - LLM 客户端重构（266行），流式调用优化 新子系统： - Hooks 生命周期系统（9种事件类型，PreToolUse/PostToolUse 支持输入输出拦截） - Skills 双层加载系统（system-reminder 轻量注入 + LoadSkillTool 按需加载，notify 文件监听热更新） - Memory 项目记忆管理（类型/提取/去重/衰减/保活/选择策略/护栏 7 模块） - SubAgent 上下文隔离子代理运行器（独立 ReAct 循环 + Hook 管道） - Team 多智能体团队协作（文件 inbox 通信、lead/teammate 协调） - TaskBoard DAG 任务依赖管理 - Trajectory 会话轨迹、Terminal 终止信号、Autonomous 自主模式、Background 异步通知 数据库： - agent_tasks 表（DAG 依赖模式，blocked_by JSON 数组） - agent_audit_log 表（工具调用审计：名称/状态/耗时/输出预览） - agent_identity 迁移（消息/审计/任务的 agent_name 归属，agent_team_members 团队注册表） API： - GET /chat/metrics 聚合指标端点 - GET /chat/sessions/:id/audit 会话审计查询 - GET /chat/questions + POST /chat/answer 人机交互问答 工程： - 新增依赖：serde_yaml、notify、glob、walkdir、lru - Skills 目录含 methodology/plotting/presentation 三个初始 SKILL.md - CLAUDE.md 完整项目架构文档
164 lines
9.1 KiB
Markdown
164 lines
9.1 KiB
Markdown
# CLAUDE.md
|
|
|
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
|
|
## Build, Lint & Test Commands
|
|
|
|
```bash
|
|
# Build (debug)
|
|
cargo build
|
|
|
|
# Build (release with optimizations)
|
|
cargo build --release
|
|
|
|
# Build (release-min profile: size-optimized LTO)
|
|
cargo build --profile release-min
|
|
|
|
# Run (debug, starts server on http://localhost:8000)
|
|
cargo run
|
|
|
|
# Run with Obscura in-process browser (no external binaries needed)
|
|
cargo run --features obscura-inprocess
|
|
|
|
# Run CLI binary
|
|
cargo run --bin astroresearch_cli
|
|
|
|
# Run health check tool
|
|
cargo run --bin health_check # read-only scan
|
|
cargo run --bin health_check -- --fix # auto-repair
|
|
|
|
# Lint
|
|
cargo clippy
|
|
|
|
# Format
|
|
cargo fmt
|
|
|
|
# All tests
|
|
cargo test
|
|
|
|
# Unit tests only
|
|
cargo test --lib
|
|
|
|
# Run a specific test
|
|
cargo test test_name
|
|
|
|
# Frontend (cd dashboard first)
|
|
npm run dev # HMR dev server on :5173, proxies /api to :8000
|
|
npm run build # TypeScript check + Vite production build → dashboard/dist/
|
|
npm run lint # ESLint
|
|
```
|
|
|
|
## Architecture Overview
|
|
|
|
**Stack**: Rust Axum backend (port 8000) + React/Vite/TypeScript frontend (port 5173 in dev). In production, the Rust binary serves the pre-built `dashboard/dist/` via `ServeDir` and `ServeFile` fallback, so there is a single process.
|
|
|
|
### Source Layer Map
|
|
|
|
```
|
|
src/
|
|
├── main.rs # Axum server entry: logging, DB pool, migrations, vec0 table,
|
|
│ # client/service construction, route registration, AppState assembly
|
|
├── lib.rs # Config struct + from_env() loading from .env
|
|
├── api/ # HTTP handlers, AppState, StandardPaper type
|
|
│ ├── mod.rs # AppState (shared state), StandardPaper, handlers re-exports
|
|
│ ├── agent.rs # SSE chat_agent endpoint, session CRUD, metrics, audit log
|
|
│ ├── papers.rs # Search, download, parse, translate, embed, citations, library, export
|
|
│ ├── notes.rs # Highlight/note CRUD
|
|
│ ├── sync.rs # Meta-sync and asset-batch endpoints
|
|
│ ├── targets.rs # Target query/associate/extract, RAG chat, figure chat
|
|
│ └── helpers.rs # Shared DB helpers, format conversion, path validation
|
|
├── agent/ # ReAct-based research agent (LLM-driven tool-use loop)
|
|
│ ├── tools/ # AgentTool trait, ToolRegistry, tool implementations per domain file
|
|
│ ├── runtime/ # ReAct loop engine, streaming, session management, context building
|
|
│ ├── compact/ # Context compression (micro/auto/manual layers)
|
|
│ ├── hooks.rs # Lifecycle events (PreToolUse/PostToolUse/Stop/etc.)
|
|
│ ├── skills.rs # SkillRegistry: loads skill SKILL.md files from skills/ directory
|
|
│ ├── subagent.rs # Context-isolated sub-agent runner for delegate_research
|
|
│ ├── background.rs# BgNotificationQueue for async slow-task (download/parse) notifications
|
|
│ └── team/ # Multi-agent team: file-based inbox, lead/teammate coordination
|
|
├── clients/ # External API wrappers
|
|
│ ├── llm.rs # LlmClient (OpenAI-compatible chat + streaming), EmbeddingClient
|
|
│ ├── ads.rs # NASA ADS API
|
|
│ ├── arxiv.rs # arXiv Atom XML API
|
|
│ └── qiniu.rs # Qiniu cloud storage
|
|
├── services/ # Business logic
|
|
│ ├── search.rs # Unified cross-source search (ADS + arXiv dedup)
|
|
│ ├── download.rs # PDF/HTML download with anti-bot measures and fallback chain
|
|
│ ├── parser/ # HTML/PDF → Markdown parsers (A&A, IOP, ar5iv, generic, PDF via MinerU)
|
|
│ ├── translation.rs# LLM bilingual translation with Trie-based astronomy glossary
|
|
│ ├── rag.rs # Embedding ingest + vector similarity retrieval + LLM answer generation
|
|
│ ├── target.rs # Celestial target extraction (IAU name regex) + CDS Sesame lookup
|
|
│ ├── chunker.rs # Markdown text chunking for embedding
|
|
│ ├── batch/ # Meta-sync (ADS bulk harvest) and asset-batch processing engines
|
|
│ ├── query_parser.rs# Advanced search query syntax parser
|
|
│ └── logging.rs # Pretty console + rolling file logger
|
|
└── bin/
|
|
├── health_check.rs # Library consistency checker and auto-repair
|
|
├── cli.rs # CLI interface
|
|
└── reparse.rs # Re-parse existing library items
|
|
```
|
|
|
|
### AppState — Central Shared State
|
|
|
|
All handlers access state via `Arc<AppState>`. Key fields:
|
|
|
|
- `db: SqlitePool` — SQLite connection pool (5 max connections, foreign keys enforced)
|
|
- `llm: LlmClient` / `embedding: EmbeddingClient` — OpenAI-compatible LLM clients
|
|
- `ads: AdsClient` / `arxiv: ArxivClient` — academic search clients
|
|
- `skill_registry: Arc<RwLock<SkillRegistry>>` — hot-reloaded agent skills
|
|
- `cancelled_runs: Arc<Mutex<HashSet<String>>>` — agent cancellation tokens
|
|
- `harvest_status` / `batch_status` — async batch operation status tracking
|
|
|
|
### Agent System Design
|
|
|
|
The agent (`src/agent/`) implements a **ReAct** (Thought → Action → Observation) loop:
|
|
|
|
1. **`AgentRuntime`** (`runtime/mod.rs`) orchestrates the loop: session create/resume → context build → ReAct loop → finalize
|
|
2. **Streaming**: LLM response is streamed via SSE (`AgentStreamEvent`) — thought, tool_call, tool_result, text_delta, usage, error, done
|
|
3. **Tools**: Each tool implements `AgentTool` trait (name, description, JSON Schema parameters, execute). 19 tools in default registry including read_file, grep_files, glob_files, run_bash, file_write, file_edit, search_papers, download_paper, parse_paper, get_paper_content, rag_search, query_target, save_note, todo_write, compress_context, load_skill, delegate_research, plus optional background and team tools
|
|
4. **Parallel execution**: Same-turn tool calls execute concurrently via `executor::execute_parallel`
|
|
5. **Context compression**: Three layers — micro (placeholder replacement), auto (LLM summarization when over threshold), manual (compress_context tool). Protected by `CompactionCircuitBreaker`
|
|
6. **Skills** (`skills.rs`): Two-layer loading — system-reminder lists names (~20 tokens each), LLM calls `load_skill` to inject full SKILL.md content
|
|
7. **Sub-agents** (`subagent.rs`): `delegate_research` spawns a context-isolated sub-agent with its own ReAct loop, returning only the final summary
|
|
8. **Teams** (`team/`): File-based inbox directory per session for lead/teammate message passing
|
|
9. **Background tasks** (`background.rs`): Slow ops (download, parse) can run async; results inject via `BgNotificationQueue` before next LLM call
|
|
|
|
Environment variables for agent tuning: `AGENT_MAX_STEPS` (default 8), `AGENT_TOOL_TIMEOUT_SECS` (default 120), `AGENT_MAX_TOOL_OUTPUT_CHARS` (default 4000), `AGENT_CONTEXT_CHAR_LIMIT` (default 16000), `AGENT_TOKEN_SOFT_LIMIT` / `AGENT_TOKEN_HARD_LIMIT`.
|
|
|
|
### Database
|
|
|
|
SQLite via `sqlx::sqlite`. Migrations in `migrations/` are auto-run on startup (`sqlx::migrate!("./migrations")`). Key tables: `papers`, `citations_references`, `notes`, `agent_sessions`, `agent_messages`, `agent_tasks`, `agent_audit_log`, `paper_chunks_content`. Vector embeddings use `sqlite-vec` (`vec_paper_chunks` virtual table, auto-registered before any DB connection).
|
|
|
|
The embedding dimension is controlled by `EMBEDDING_DIM` env var (default 1536). On dimension mismatch, the vec table and chunk content are dropped and recreated.
|
|
|
|
### Frontend (dashboard/)
|
|
|
|
React 19 + TypeScript + Vite + Tailwind CSS 4. Features are organized by domain:
|
|
|
|
- `features/search/` — Cross-source paper search panel
|
|
- `features/library/` — Local library management
|
|
- `features/reader/` — Bilingual reader with highlight annotations (KaTeX for math)
|
|
- `features/citation/` — Canvas-based force-directed citation graph
|
|
- `features/sync/` — Batch sync control panel
|
|
- `features/agent/` — Agent chat interface (SSE event consumption)
|
|
- `features/settings/` — System configuration
|
|
|
|
Dependencies: `react-markdown` + `rehype-katex` + `remark-math` for Markdown/LaTeX rendering, `framer-motion` for animations, `lucide-react` for icons.
|
|
|
|
### Obscura In-Process Browser
|
|
|
|
The `obscura-inprocess` feature compiles `obscura-browser` and `obscura-net` directly into the binary, eliminating the need for external browser binaries. This is used for bypassing Cloudflare/WAF on PDF download. Enabled via `--features obscura-inprocess`.
|
|
|
|
### Astronomy Glossary (dictionary.txt)
|
|
|
|
A 1MB+ bilingual astronomy terminology file loaded at startup into a Trie tree for longest-match glossary construction, used by the translation service to guide LLM translations with domain-accurate term mappings.
|
|
|
|
## Code Conventions
|
|
|
|
- Use `anyhow` for application errors, `thiserror` for library-style typed errors
|
|
- Environment variables via `dotenvy` + `std::env::var`, with defaults in `Config::from_env()`
|
|
- SQL queries use parameterized bindings (`sqlx::query("...").bind(...)`) — never string interpolation
|
|
- API handlers take `State(Arc<AppState>)` and return Axum-compatible responses
|
|
- Agent tools implement `AgentTool` trait; new tools register in `ToolRegistry::new()`
|
|
- Front-end build is triggered by `build.rs` (auto npm install + build when `dashboard/src/` changes)
|