feat: Agent 多模式系统、视觉模型集成、LLM 能力分层与 P3 性能收尾

核心架构变更:

  1. Agent 多模式系统替代 Coordinator
     - 移除 src/agent/coordinator/(Coordinator Agent/Worker/Tools,946 行)
     - 新建 src/agent/modes/:声明式模式抽象(AgentMode/ModeConfig/ToolSet)
     - 三种内置模式:
       - default:通用科研助手,零覆盖保持现有行为
       - deep-research:16 步、启用思考、research 权限、系统性调研
       - literature-reader:白名单工具、只读沙箱、结构化阅读
     - ModeRegistry + ModeConfig 预设 + ToolSet 过滤 + 身份/原则覆盖
     - AgentRuntime::with_mode() 统一入口,模式持久化到 session.mode 字段
     - GET /api/chat/modes 提供模式列表给前端选择器

  2. 视觉模型与图片分析
     - 新增 analyze_image 工具(340 行):本地/URL 图片 → 视觉模型流式分析
     - LlmClient::analyze_image_stream():SSE 增量实时推送
     - 配置:LLM_VISION_MODEL / LLM_VISION_API_KEY / LLM_VISION_API_BASE
     - 前端:粘贴/选择图片附件,重试时复用文件路径
     - Service 层移除 /chat/rag 和 /chat/figure 端点,统一走 Agent SSE
     - Body limit 提升至 100MB 适配大图上传

  3. LLM 三级能力分层
     - Tier 1 (Core) → Tier 2 (Medium) → Tier 3 (Fast),级联回退
     - medium_llm / fast_llm / vision_llm 注入 AppState
     - 资产批量翻译 → Medium LLM + Semaphore(3) 并发控制
     - 记忆提取/上下文压缩子代理 → Fast LLM
     - SubAgentRunner::with_llm_client() 支持注入专用 LLM

  4. 数据库与性能优化
     - SQLite 启用 WAL + busy_timeout(10s) 处理并发写入
     - RAG ingest:DELETE 合并为原子语句 + 批量事务写入
     - Meta sync:save_paper_to_db_tx() 事务化批量插入
     - 翻译词典:first_words HashSet 预过滤 + next_valid_index 跳跃优化
     - read_file 不截断输出 + skip_persist 防止级联磁盘持久化

  5. 工具系统增强
     - ToolContext 增加 tool_call_id + max_output_chars
     - ToolOutput 增加 skip_persist 标记
     - TextDelta SSE 携带可选 tool_call_id 支持工具的流式输出
     - ChatMessage::text() 辅助方法
This commit is contained in:
fmq
2026-06-24 19:52:27 +08:00
parent cec4b8cf7b
commit 85b6429c30
44 changed files with 2307 additions and 1578 deletions
+71 -4
View File
@@ -69,11 +69,13 @@ src/
│ └── helpers.rs # Shared DB helpers, format conversion, path validation
├── agent/ # ReAct-based research agent (LLM-driven tool-use loop)
│ ├── tools/ # AgentTool trait, ToolRegistry, 25+ tool implementations per domain file
│ ├── runtime/ # ReAct loop, streaming, session/context, token budget, error recovery,
│ │ # permission checker, file cache, system prompt assembly, circuit breaker
│ ├── runtime/ # ReAct loop core: session, context, streaming, executor, token_budget,
│ │ # permission, permission_profile, checkpoint, circuit_breaker, error_recovery,
│ │ # hardline, partitioner, file_cache, system_prompt, finalize, denial_tracker
│ ├── compact/ # Context compression (micro/auto/manual layers + collapse)
│ ├── memory/ # Persistent memory manager: extraction, dedup, decay, age, guardrails
│ ├── hooks.rs # Lifecycle events (PreToolUse/PostToolUse/Stop/etc.)
│ ├── hooks/ # Lifecycle hooks: registry, dispatch, builtins, matcher, traits (PreToolUse/PostToolUse/Stop/etc.)
│ ├── modes/ # Agent session modes: default, deep-research, literature-reader (identity/tools/config presets)
│ ├── skills.rs # SkillRegistry: hot-loads SKILL.md files from skills/ directory
│ ├── subagent.rs # Context-isolated sub-agent runner (subagent tool)
│ ├── team/ # Multi-agent team: file-based inbox, lead/teammate coordination
@@ -121,7 +123,7 @@ The agent (`src/agent/`) implements a **ReAct** (Thought → Action → Observat
1. **`AgentRuntime`** (`runtime/mod.rs`) orchestrates the loop: session create/resume → context build → ReAct loop → finalize
2. **Streaming**: LLM response is streamed via SSE (`AgentStreamEvent`) — thought, tool_call (with `id`), tool_result (with `tool_call_id`), text_delta, usage, error, done. Tool calls execute in parallel.
3. **Tools**: Each tool implements `AgentTool` trait (name, description, JSON Schema parameters, execute). Core tools: read_file, grep_files, glob_files, run_bash, file_write, file_edit, search_papers, download_paper, parse_paper, get_paper_content, rag_search, query_target, save_note, todo_write, compress_context, load_skill, subagent, ask_user, save_memory. Plus background tools (bg_task_run, bg_task_check) and team tools (spawn_teammate, send_teammate_message, team_broadcast, check_team_inbox).
3. **Tools**: Each tool implements `AgentTool` trait (name, description, JSON Schema parameters, execute). Core tools: read_file, grep_files, glob_files, run_bash, file_write, file_edit, search_papers, download_paper, parse_paper, get_paper_content, rag_search, query_target, save_note, todo_write, compress_context, load_skill, subagent, ask_user, save_memory, analyze_image (vision model, only when `LLM_VISION_MODEL` set). Plus background tools (bg_task_run, bg_task_check) and team tools (spawn_teammate, send_teammate_message, team_broadcast, check_team_inbox).
4. **Thinking mode**: `enable_thinking` flag propagates from `AgentChatRequest``AgentConfig``ToolContext``LlmClient::chat_stream`. Only enabled for Qwen/DashScope backends; frontend-controlled via the `thinking` request field.
5. **Tool call ID tracking**: LLM may not return tool_call IDs — `LlmClient` generates UUID fallbacks. `ToolCall` and `ToolResult` SSE events carry matching IDs for precise frontend pairing.
6. **ToolContext** (`tools/mod.rs`): Injected into every tool execution — holds `app_state`, `session_id`, `sse_tx` (for intermediate events), `enable_thinking`, `read_file_state` (file cache for dedup), `silent` (sub-agents skip permission prompts).
@@ -162,6 +164,71 @@ The `obscura-inprocess` feature compiles `obscura-browser` and `obscura-net` dir
A 1MB+ bilingual astronomy terminology file loaded at startup into a Trie tree for longest-match glossary construction, used by the translation service to guide LLM translations with domain-accurate term mappings.
### Agent Session Modes (`src/agent/modes/`)
Session-level modes configure the agent's identity, tool access, step limits, thinking, and permissions at creation time. Modes are pure-data constants defined at compile time — adding a new mode requires no logic changes.
Three built-in modes (registered in `ModeRegistry::builtins()`):
| Mode | ID | Tools | Max Steps | Thinking | Permission Profile |
|------|-----|-------|-----------|----------|---------------------|
| 通用科研助手 | `default` | All (unrestricted) | 8 (default) | user-controlled | none |
| 深度研究 | `deep-research` | All | 16 | forced on | `research` |
| 文献阅读助手 | `literature-reader` | Allowlist (read-only + literature) | 6 | forced off | `readonly` |
Key types in `modes/mod.rs`:
- **`AgentMode`** — static definition struct: id, name, description, icon, identity/principles overrides, extra sections, `ToolSet`, `ModeConfig`
- **`ToolSet`** — `All`, `Allowlist(&[&str])`, or `Except(&[&str])` — constrains which tools are available
- **`ModeConfig`** — optional overrides for `max_steps`, `enable_thinking`, `tool_timeout_secs`, `permission_profile`
- **`ModeRegistry`** — holds `&'static AgentMode` references; `get(id)` for lookup
The mode is stored in the `agent_sessions.mode` column (migration `20260624000000_add_session_mode.sql`, defaults to `'default'`). The frontend `ResearchAgentPanel` exposes mode selection.
**Mode vs Skill**: Modes are session-level ("who am I"), Skills are task-level ("how do I do X"). Modes affect initialization; the ReAct loop itself is mode-agnostic.
### Permission System (`src/agent/runtime/permission.rs`)
Tool execution is gated by a priority-ordered rule chain: **Deny > Allow > Ask** (first match wins). Rules support content-level pattern matching (e.g., `run_bash(rm *)`).
**Permission modes** (set via `AGENT_PERMISSION_MODE` env or mode's `permission_profile`):
- `default` — full rule chain evaluation
- `accept_edits` — auto-allow `file_write`/`file_edit` within the working directory
- `bypass` — skip all Ask checks (Deny rules still enforced)
- `dont_ask` — convert all Ask to Deny
**Permission profiles** (`src/agent/runtime/permission_profile.rs`) are named presets loaded by `AGENT_PERMISSION_MODE` or a mode's `permission_profile` field. Profiles `research` and `readonly` are used by deep-research and literature-reader modes respectively.
**Configuration** (in `.env`):
- `AGENT_PERMISSIONS_DENY` — comma-separated deny rules (e.g., `run_bash(rm *),run_bash(sudo *)`)
- `AGENT_PERMISSIONS_ALLOW` — comma-separated allow rules
- `AGENT_PERMISSIONS_ASK` — comma-separated ask rules
- `AGENT_PERMISSION_MODE` — default/accept_edits/bypass/dont_ask
### Multi-Tier LLM Configuration
The system supports three LLM tiers with cascade fallback:
| Tier | Env Prefix | Purpose | Fallback |
|------|-----------|---------|----------|
| Primary | `LLM_` | Main agent reasoning | — |
| Medium | `LLM_MEDIUM_` | Translation, RAG | Primary LLM config |
| Fast | `LLM_FAST_` | Memory extraction, background tasks | Medium LLM config |
Each tier has `_API_KEY`, `_API_BASE`, `_MODEL` variants. Unset tiers cascade to the next tier down.
**Fallback chain**: `LLM_FALLBACK_CHAIN` (comma-separated model names) provides automatic model rotation on repeated 529 errors. `LLM_FALLBACK_MODEL` is a single backup model tried before the chain.
### Vision Model (`analyze_image` tool)
When `LLM_VISION_MODEL` env is set, the `analyze_image` tool (`src/agent/tools/astro/analyze_image.rs`) is registered. It delegates image analysis to a dedicated vision model, enabling the main agent to use a text-only model while still processing images. Supports local paths (relative to library dir) and HTTP(S) URLs. Results stream via SSE `TextDelta` events. Also supports `LLM_VISION_API_KEY` and `LLM_VISION_API_BASE` (fall back to primary LLM config).
### Auto Memory Extraction
Controlled by env vars:
- `EXTRACT_MEMORY_ENABLED` (default `false`) — enables automatic memory extraction at session end and during compaction
- `EXTRACT_MEMORY_THROTTLE_TURNS` (default `3`) — extract every N turns
- `EXTRACT_MEMORY_MAX_STEPS` (default `3`) — sub-agent max steps for extraction
## Code Conventions
- Use `anyhow` for application errors, `thiserror` for library-style typed errors