Compare commits
4 Commits
84883018aa
...
a4c8b29176
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a4c8b29176 | ||
|
|
8b650f4e5f | ||
|
|
e24e3226fa | ||
|
|
f8a20f1b0c |
249
skills/ov-add-data/SKILL.md
Normal file
249
skills/ov-add-data/SKILL.md
Normal file
@@ -0,0 +1,249 @@
|
||||
---
|
||||
name: ov-add-data
|
||||
description: This skill adds data(like resources) to OpenViking Context Database (aka. ov). Use when an agent needs to add files, data from URLs, or external knowledge during interactions. Trigger this tool when 1. is explicitly requested adding files or knowledge; 2. identifies valuable resources worth importing; 3. the user mentioned adding to OV/OpenViking/Context Database. This skill helps how to use CLI like `ov add-resource`, `ov add-skill` and `ov add-memory` to add resource data, skill files, memory files to OpenViking.
|
||||
compatibility: OpenViking CLI configured at `~/.openviking/ovcli.conf`
|
||||
---
|
||||
|
||||
# OpenViking (OV) `add-resource`
|
||||
|
||||
The `ov add-resource` command imports external resources into OpenViking's context database — supporting local files, directories, URLs, and remote repositories. Resources are automatically processed with semantic analysis and organized under the `viking://resources/` namespace.
|
||||
|
||||
## When to Use
|
||||
|
||||
- Importing project documentation, code repositories, or reference materials
|
||||
- Adding web pages, articles, or online resources for future retrieval
|
||||
- Building a knowledge base from external sources
|
||||
- When an agent encounters valuable content that should persist across sessions
|
||||
- Recording a project's product documentation, design specs, or other valuable materials
|
||||
- Storing sensitive information or private data of the user, like photo and albums
|
||||
|
||||
## CLI Options
|
||||
|
||||
### Basic Usage
|
||||
|
||||
Import a local file or URL:
|
||||
|
||||
```bash
|
||||
# add a code repo from github or gitlab or a git address
|
||||
ov add-resource https://github.com/volcengine/OpenViking
|
||||
ov add-resource https://code.xxxx.org/viking/viking
|
||||
ov add-resource git@code.xxxx.org:viking/viking.git
|
||||
|
||||
# add a file from url
|
||||
ov add-resource https://arxiv.org/pdf/2602.09540
|
||||
ov add-resource https://raw.githubusercontent.com/volcengine/OpenViking/main/README.md
|
||||
|
||||
# add a file from local filesystem
|
||||
ov add-resource ./docs/api-spec.md
|
||||
ov add-resource ./team_building.jpg
|
||||
ov add-resource /User/volcengine/Documents/profile.pdf
|
||||
ov add-resource /User/volcengine/Documents/project.docx
|
||||
|
||||
# add a zip file from local filesystem (will be unzipped on server)
|
||||
ov add-resource ./docs-of-project.zip
|
||||
|
||||
# add a directory from local filesystem
|
||||
ov add-resource /User/volcengine/Photo/Travels/2026/ --include "*.jpg,*.jpeg,*.png"
|
||||
ov add-resource /User/volcengine/Documents/OV项目设计文档/
|
||||
```
|
||||
|
||||
### Context and Instructions (TBD)
|
||||
|
||||
Add metadata to guide processing: --reason and --instruction will be supported in the future.
|
||||
|
||||
### Async Processing Control
|
||||
|
||||
The time of adding resources could cost long (like minutes), and the semantic processing could be async. Use `--wait` to wait for the processing to complete.
|
||||
|
||||
```bash
|
||||
# Wait until semantic processing finishes
|
||||
ov add-resource ./docs --wait
|
||||
|
||||
# Wait with timeout (in seconds)
|
||||
ov add-resource https://example.com/docs --wait --timeout 60
|
||||
|
||||
# Fire and forget (default, without --wait)
|
||||
ov add-resource ./docs
|
||||
```
|
||||
|
||||
If you fire and forget, the command will return after the resource is completely downloaded/uploaded on server, and then the CLI will return the root URI of the imported resource. To check the status of the resource, you can use `ov ls` or `ov tree` to list the resources under root URI or `viking://resources/`.
|
||||
|
||||
### Specify the target path
|
||||
|
||||
By default, resources are imported under `viking://resources/`. Use `--to` or `--parent` to specify a target uri.
|
||||
|
||||
```bash
|
||||
# The data will be imported as viking://resources/2026/2026-01-01/, which should not exist before
|
||||
ov add-resource /User/volcengine/Photo/Travels/2026/2026-01-01/ --to "viking://resources/2026/2026-01-01/"
|
||||
|
||||
# The data will be imported under viking://resources/2026, which should exist and be a directory
|
||||
ov add-resource /User/volcengine/Photo/Travels/2026/2026-01-02/ --parent "viking://resources/2026/"
|
||||
```
|
||||
|
||||
## CLI Output
|
||||
|
||||
Returns the root URI of the imported resource, like:
|
||||
|
||||
```
|
||||
root_uri viking://resources/2026/2026-01-01
|
||||
```
|
||||
|
||||
## CLI Prerequisites
|
||||
|
||||
- CLI configured in: `~/.openviking/ovcli.conf`
|
||||
- Network access for the importing URL from server
|
||||
- Local read access for the importing local files/directories from CLI.
|
||||
|
||||
# OpenViking (OV) `add-memory`
|
||||
|
||||
The `ov add-memory` command adds long persistant memory — turning text and structured conversations into searchable, retrievable memories in the OpenViking context database. Use `ov add-memory --help` for latest usage.
|
||||
|
||||
## When to Use
|
||||
|
||||
- After learning something worth remembering across sessions
|
||||
- To persist conversation insights, decisions, or findings
|
||||
- To build up a knowledge base from interactions
|
||||
- When an agent wants to store context for future retrieval
|
||||
|
||||
## Input Modes
|
||||
choose wisely between plain text and multi-turn mode. Multi-turn mode can contain more complex insights, let openviking handle the memory extraction.
|
||||
|
||||
### Mode 1: Plain Text for compressed memory
|
||||
|
||||
A simple string is stored as a `user` message:
|
||||
|
||||
```bash
|
||||
ov add-memory "User's name is Bob, he participate in Global Hackathon in 2025-01-08, and won Champion."
|
||||
```
|
||||
|
||||
### Mode 2: Multi-turn Conversation for Richer Context
|
||||
|
||||
A JSON array of `{role, content}` objects to store a full exchange:
|
||||
|
||||
```bash
|
||||
ov add-memory '[
|
||||
{"role": "user", "content": "I love traveling. Give me some options of Transport from Beijing to Shanghai."},
|
||||
{"role": "assistant", "content": "You can use train, bus, or plane. Train is the fastest, but you need to book in advance. Bus is cheaper, but you need to wait. Plane is the most expensive, but you can get there any time of day."},
|
||||
{"role": "user", "content": "I prefer train. I like sightseeing on the train. Can you give me the train schedule?"},
|
||||
< ... more possible conversation about schedule and tickest need to be memorized ... >
|
||||
]'
|
||||
```
|
||||
|
||||
## Output
|
||||
|
||||
Returns `OK` once the request is accepted. Memory extraction runs asynchronously on the server after the commit is queued, so the CLI does not wait for extraction to finish:
|
||||
|
||||
```
|
||||
OK
|
||||
```
|
||||
|
||||
## Agent Best Practices
|
||||
|
||||
### How to Write Good Memories
|
||||
|
||||
1. **Be specific** — Include concrete details, not vague summaries
|
||||
2. **Include context** — Why this matters, when it applies
|
||||
3. **Use structured format** — Separate the what from the why
|
||||
|
||||
### Batch Related Facts
|
||||
|
||||
Group related memories in one call rather than many small ones:
|
||||
|
||||
```bash
|
||||
ov add-memory '[
|
||||
{"role": "user", "content": "Key facts about the ov_cli Rust crate"},
|
||||
{"role": "assistant", "content": "1. runs faster than python cli\n2. uses HttpClient to connect openviking server\n3. Output formatting supports table and JSON modes\n4. Config lives at ~/.openviking/ovcli.conf"}
|
||||
]'
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- CLI configured: `~/.openviking/ovcli.conf`
|
||||
|
||||
# OpenViking (OV) `add-skill`
|
||||
|
||||
The `ov add-skill` command adds agent capabilities to OpenViking — supporting SKILL.md files, MCP tool definitions, and raw skill content. Skills are automatically processed and organized under the `viking://agent/skills/` namespace, making them discoverable and usable by agents.
|
||||
|
||||
## When to Use
|
||||
|
||||
- Adding custom agent capabilities and workflows
|
||||
- Importing MCP (Model Context Protocol) tools
|
||||
- Persisting skill definitions across agent sessions
|
||||
- Building a library of reusable agent capabilities
|
||||
- When an agent needs to extend its toolset with custom logic
|
||||
|
||||
## Input Formats
|
||||
|
||||
### Mode 1: SKILL.md File
|
||||
|
||||
Import from a single SKILL.md file with YAML frontmatter:
|
||||
|
||||
```bash
|
||||
# Add from a single SKILL.md file
|
||||
ov add-skill ./skills/my-skill/SKILL.md
|
||||
|
||||
# Add from a directory containing SKILL.md (includes auxiliary files)
|
||||
ov add-skill ./skills/my-skill/
|
||||
```
|
||||
|
||||
## CLI Options
|
||||
|
||||
### Async Processing Control
|
||||
|
||||
Semantic processing happens asynchronously. Use `--wait` to block until complete:
|
||||
|
||||
```bash
|
||||
# Wait until semantic processing finishes
|
||||
ov add-skill ./skills/my-skill/ --wait
|
||||
|
||||
# Wait with timeout (in seconds)
|
||||
ov add-skill ./skills/my-skill/ --wait --timeout 60
|
||||
|
||||
# Fire and forget (default, without --wait)
|
||||
ov add-skill ./skills/my-skill/
|
||||
```
|
||||
|
||||
## CLI Output
|
||||
|
||||
Returns the URI of the added skill, like:
|
||||
|
||||
```
|
||||
uri viking://agent/skills/my-skill/
|
||||
```
|
||||
|
||||
## SKILL.md Format
|
||||
|
||||
Skills use Markdown with YAML frontmatter:
|
||||
|
||||
```markdown
|
||||
---
|
||||
name: skill-name
|
||||
description: Brief description of the skill
|
||||
allowed-tools:
|
||||
- Tool1
|
||||
- Tool2
|
||||
tags:
|
||||
- tag1
|
||||
- tag2
|
||||
---
|
||||
|
||||
## Including Auxiliary Files
|
||||
|
||||
When adding from a directory, all files in the directory are included as auxiliary files:
|
||||
|
||||
```bash
|
||||
# Directory structure:
|
||||
# ./skills/code-runner/
|
||||
# ├── SKILL.md
|
||||
# ├── helper.py
|
||||
# └── templates/
|
||||
# └── script.py
|
||||
|
||||
ov add-skill ./skills/code-runner/
|
||||
# Both helper.py and templates/ are included
|
||||
```
|
||||
|
||||
## CLI Prerequisites
|
||||
|
||||
- CLI configured: `~/.openviking/ovcli.conf`
|
||||
- The skill file (SKILL.md) should be in the correct markdown format.
|
||||
157
skills/ov-search-context/SKILL.md
Normal file
157
skills/ov-search-context/SKILL.md
Normal file
@@ -0,0 +1,157 @@
|
||||
---
|
||||
name: ov-search-context
|
||||
description: Search context data(memories, skills and resource) from OpenViking Context Database (aka. ov). Trigger this tool when 1. need information that might be stored as memories, skills or resources on OpenViking; 2. is explicitly requested searching files or knowledge; 3. sees `search context`, `search openviking`, `search ov` request.
|
||||
compatibility: CLI configured at `~/.openviking/ovcli.conf`
|
||||
---
|
||||
# OpenViking (OV) context searching
|
||||
The `ov search` command performs context-aware retrieval across all memories and resources in OpenViking — combining semantic understanding with directory recursive retrieval to find the most relevant context for any query.
|
||||
|
||||
## Table of Content
|
||||
- When to Use
|
||||
- Sub-commands for search
|
||||
- List directories (`ov ls`)
|
||||
- Tree view (`ov tree`)
|
||||
- Semantic Search (`ov find`)
|
||||
- Content Pattern Search (`ov grep`)
|
||||
- File Glob Search (`ov glob`)
|
||||
- Full content read (`ov read`)
|
||||
- Get overview (`ov overview`)
|
||||
- Get Abstract (`ov abstract`)
|
||||
- Prerequisite
|
||||
|
||||
## When to Use
|
||||
|
||||
- Finding specific information within imported resources or saved memories
|
||||
- Retrieving context about topics, APIs, or patterns previously added
|
||||
- Searching across project documentation, code, and learnings
|
||||
- When an agent needs to reference previously stored knowledge
|
||||
|
||||
> note: cli command can be outdated, when sees error, use `--help` to get latest usage
|
||||
|
||||
## Sub-commands for search
|
||||
|
||||
### List Contents (`ov ls`)
|
||||
|
||||
Browse directory structure:
|
||||
|
||||
```bash
|
||||
# List root directory
|
||||
ov ls
|
||||
|
||||
# List specific directory
|
||||
ov ls viking://resources/my-project/docs/
|
||||
|
||||
# Simple path output (only uris, no metadata)
|
||||
ov ls viking://resources --simple
|
||||
|
||||
# Show hidden files
|
||||
ov ls viking://resources --all
|
||||
|
||||
# Control output limits (default 256)
|
||||
ov ls viking://resources --node-limit 50
|
||||
|
||||
# Control abstract info length limit for each node (default 256)
|
||||
ov ls viking://resources --abs-limit 128
|
||||
```
|
||||
|
||||
### Tree View (`ov tree`)
|
||||
|
||||
Visualize directory hierarchy:
|
||||
|
||||
```bash
|
||||
# Show tree structure
|
||||
ov tree viking://resources
|
||||
|
||||
# Control depth limits (default 3)
|
||||
ov tree viking://resources --level-limit 2
|
||||
|
||||
# Control node limits
|
||||
ov tree viking://resources --node-limit 100 --abs-limit 128
|
||||
|
||||
# Show all files including hidden
|
||||
ov tree viking://resources --all
|
||||
```
|
||||
|
||||
### Semantic find (`ov find`)
|
||||
|
||||
Find method with semantic relevance ranking:
|
||||
|
||||
```bash
|
||||
# Basic find across all context
|
||||
ov find "how to handle API rate limits"
|
||||
|
||||
# Find within specific URI scope
|
||||
ov find "authentication flow" --uri "viking://resources/my-project"
|
||||
|
||||
# Limit results and set relevance score threshold
|
||||
ov find "error handling" --node-limit 5 --threshold 0.3
|
||||
```
|
||||
|
||||
### Content Pattern Search (`ov grep`)
|
||||
|
||||
Literal pattern matching:
|
||||
|
||||
```bash
|
||||
# Find exact text pattern (Note: this is expensive, and suggest within specific small URI scope)
|
||||
ov grep "viking://resources" "TODO:" --uri "viking://resources/my-project"
|
||||
|
||||
# Case-insensitive search
|
||||
ov grep "viking://resources" "API_KEY" --ignore-case --uri "viking://resources/my-project"
|
||||
|
||||
# Limit results and set node limit
|
||||
ov grep "viking://resources" "API_KEY" --node-limit 5 --uri "viking://resources/my-project"
|
||||
```
|
||||
|
||||
### File Glob Search (`ov glob`)
|
||||
|
||||
File path pattern matching:
|
||||
|
||||
```bash
|
||||
# Find all markdown files (Note: this is expensive, and suggest within specific small URI scope)
|
||||
ov glob "**/*.md" --uri "viking://resources/my-project"
|
||||
|
||||
# Limit results and set node limit
|
||||
ov glob "**/*.md" --uri "viking://resources/my-project" --node-limit 5
|
||||
```
|
||||
|
||||
### Read File Content (`ov read`)
|
||||
|
||||
Retrieve full content (L0-L2 layer):
|
||||
|
||||
```bash
|
||||
# Read full content
|
||||
ov read viking://resources/docs/api/api-1.md
|
||||
|
||||
# Read first 10 lines of api-2.md
|
||||
ov read viking://resources/docs/api/api-2.md | head -n 10
|
||||
|
||||
# Read abstract (L0 - quick summary)
|
||||
ov abstract viking://resources/docs/api/
|
||||
ov read viking://resources/docs/api/.abstract.md
|
||||
|
||||
# Read overview (L1 - key points)
|
||||
ov overview viking://resources/docs/api/
|
||||
ov read viking://resources/docs/api/.overview.md
|
||||
```
|
||||
|
||||
### Combining Search
|
||||
|
||||
Use search results to guide further actions:
|
||||
|
||||
```bash
|
||||
ov ls viking://resources/
|
||||
|
||||
# Search for relevant files
|
||||
ov search "authentication" --uri "viking://resources/project-A"
|
||||
|
||||
# Get overview for context
|
||||
ov overview viking://resources/project-A/backend
|
||||
|
||||
# Decide to read specific content
|
||||
ov read viking://resources/project-A/backend/auth.md
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- CLI configured: `~/.openviking/ovcli.conf`
|
||||
- Resources or memories previously added to OpenViking
|
||||
@@ -1,13 +1,13 @@
|
||||
---
|
||||
name: smart-data-developer
|
||||
description: 智能数据开发员工。端到端将数据需求转化为可执行SQL。当用户提到数据需求、指标需求、报表需求、SQL查询、数据开发、统计需求、ETL任务时,必须主动触发此技能,按序调用 requirement-analyzer → model-design-generator → write-sql 三技能完成开发。
|
||||
description: 智能数据开发员工。本项目为数据开发专用项目,唯一任务是将业务需求转化为可执行SQL。【任何用户消息都必须立即触发此技能】。无论是需求描述、问题咨询、闲聊问候、或其他任何形式的用户输入,都应触发此技能启动需求分析→模型设计→SQL编写的工作流程。此技能是本项目的核心入口,无例外、无条件触发。
|
||||
---
|
||||
|
||||
# 智能数据开发员工
|
||||
|
||||
## 技能定位
|
||||
|
||||
按序调用三个子技能完成端到端数据开发。**不体现角色身份**,确认发言自然融入工作流程。
|
||||
按序调用子技能完成端到端数据开发。
|
||||
|
||||
| 步骤 | 子技能 | 产出物 |
|
||||
|-----|--------|--------|
|
||||
@@ -17,44 +17,361 @@ description: 智能数据开发员工。端到端将数据需求转化为可执
|
||||
|
||||
---
|
||||
|
||||
## 工作流程
|
||||
## Conda 环境配置
|
||||
|
||||
不同子技能和 OV 检索需要在不同 conda 环境中运行:
|
||||
|
||||
| 功能 | 环境 | 激活方式 |
|
||||
|-----|------|---------|
|
||||
| data-structure-fetcher(数据源匹配) | `my_opencode` | `conda run -n my_opencode python script.py` |
|
||||
| ov 检索(OpenViking 知识库) | `py13` | `conda run -n py13 ov ...` |
|
||||
|
||||
### conda run 示例
|
||||
|
||||
**数据源匹配(my_opencode 环境)**:
|
||||
```bash
|
||||
# 直接使用 conda run 执行 Python 脚本
|
||||
conda run -n my_opencode python /path/to/data_fetcher_script.py
|
||||
|
||||
# 或先激活环境再执行
|
||||
conda activate my_opencode
|
||||
python /path/to/data_fetcher_script.py
|
||||
```
|
||||
需求 → [步骤1] → [确认] → [步骤2] → [确认] → [步骤3] → 交付
|
||||
|
||||
**OV 检索(py13 环境)**:
|
||||
```bash
|
||||
# 精确 grep 搜索表结构
|
||||
conda run -n py13 ov grep "dim_intnl_org" --uri "viking://resources/table-metadata" --node-limit 5
|
||||
|
||||
# 语义 find 搜索成功案例
|
||||
conda run -n py13 ov find "窗口函数" --uri "viking://resources/sql_snippets" --threshold 0.2
|
||||
|
||||
# 读取资源内容
|
||||
conda run -n py13 ov read viking://resources/sql_snippets/{目录}/{文件}.sql
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 执行流程
|
||||
## OpenViking 资源库
|
||||
|
||||
加载`ov-search-context`技能来查找数据开发相关的资源
|
||||
|
||||
### 核心资源
|
||||
|
||||
| 资源路径 | 用途 | 搜索策略 |
|
||||
|---------|------|---------|
|
||||
| `viking://resources/sql_snippets` | 成功SQL代码片段 | 语义find + 精确grep |
|
||||
| `viking://resources/table-metadata` | 表元信息(表结构、字段定义) | 精确grep + 语义find |
|
||||
| `viking://resources/field-process-memory` | 字段处理经验笔记 | 语义find + 精确grep |
|
||||
|
||||
### 搜索参数推荐
|
||||
|
||||
| 参数 | 推荐值 | 说明 |
|
||||
|-----|-------|------|
|
||||
| `--node-limit` | 5-15 | 避免上下文膨胀 |
|
||||
| `--threshold` | 0.2 | 过滤低相关性结果(语义搜索) |
|
||||
|
||||
---
|
||||
|
||||
## OV搜索强制执行时机
|
||||
|
||||
**为什么必须 OV 搜索**:OV 知识库包含历史成功案例、表结构信息、字段处理经验。**不检索会导致**:
|
||||
- 猜测表结构 → 字段选错 → SQL 失败
|
||||
- 缺乏成功案例参考 → 重复踩坑 → 代码质量差
|
||||
- 猜测字段含义 → 口径偏差 → 业务错误
|
||||
- 猜测处理方式 → 维护困难 → 返工成本高
|
||||
|
||||
---
|
||||
|
||||
**强制触发时机(不可跳过)**
|
||||
|
||||
| 时机 | 触发点 | 搜索内容 | 强制性 | 检索数量 |
|
||||
|-----|--------|---------|--------|---------|
|
||||
| 步骤1 - 数据源匹配后 | data-structure-fetcher 返回源表列表后 | **③ 常用数据源判断(最优先)** + 每个推荐表执行 5 类检索:表结构、成功SQL、常用判断、字段处理经验、业务语义 | ✅ MUST | 每表 5 次 |
|
||||
| 步骤1 - 核心字段确认后 | 用户确认核心字段时 | 对核心字段进行专项检索 | ✅ MUST | 每字段 3 次 |
|
||||
| 步骤2 - 模型设计前 | 用户确认需求后,开始模型设计 | OV 检索分层规范、参考案例 | ✅ MUST | 2-3 次 |
|
||||
| 步骤3 - SQL 编写前 | 用户确认模型设计后,开始编写 SQL | OV 检索语法、成功案例 | ✅ MUST | 3-5 次 |
|
||||
|
||||
---
|
||||
|
||||
**数据源匹配后的具体检索内容**
|
||||
|
||||
| 检索类型 | 目的 | 命令(py13 环境) | 优先级 |
|
||||
|---------|------|-----------------|--------|
|
||||
| ③ 常用数据源判断 | **判断是否为常用数据源**(历史使用频率、稳定性) | `conda run -n py13 ov grep "{表名}" --uri "viking://resources/field-process-memory" --node-limit 10` | **最高(立即执行)** |
|
||||
| ① 表结构检索 | 确认字段定义 | `conda run -n py13 ov grep "{表名}" --uri "viking://resources/table-metadata" --node-limit 10` | 高 |
|
||||
| ② 成功SQL检索 | 找历史代码案例 | `conda run -n py13 ov grep "{表名}" --uri "viking://resources/sql_snippets" --node-limit 15` | 高 |
|
||||
| ④ 字段处理经验 | 了解处理方式 | `conda run -n py13 ov grep "{表名}.*{字段}" --uri "viking://resources/field-process-memory" --node-limit 10` | 中 |
|
||||
| ⑤ 业务语义检索 | 业务使用经验 | `conda run -n py13 ov find "{表名} {业务}" --uri "viking://resources/field-process-memory" --threshold 0.2` | 中 |
|
||||
|
||||
---
|
||||
|
||||
**困惑场景(立即检索)**:
|
||||
- 表名陌生 → 立刻 `conda run -n py13 ov grep "{表名}" --uri "viking://resources/table-metadata"`
|
||||
- 字段歧义 → 立刻 `conda run -n py13 ov grep "{表名}.*{字段}" --uri "viking://resources/table-metadata"`
|
||||
- 语法不确定 → 立刻 `conda run -n py13 ov find "{语法}" --uri "viking://resources/sql_snippets" --threshold 0.2`
|
||||
- 找不到成功案例 → 立刻 `conda run -n py13 ov find "{业务场景}" --uri "viking://resources/sql_snippets" --threshold 0.2`
|
||||
|
||||
---
|
||||
|
||||
## OV命令速查
|
||||
|
||||
**注意:OV 检索需要在 `py13` conda 环境中运行,使用 `conda run -n py13 ov ...`**
|
||||
|
||||
```bash
|
||||
# === 精确grep:定位具体表/字段 ===
|
||||
用表名搜索:
|
||||
conda run -n py13 ov grep "{表名}" --uri "viking://resources/table-metadata" --node-limit 5
|
||||
|
||||
表名加字段名组合搜索:
|
||||
conda run -n py13 ov grep "dim_intnl_org.*org_id" --uri viking://resources/table-metadata -n 5
|
||||
|
||||
多表名 OR 搜索:
|
||||
conda run -n py13 ov grep "table1|table2|table3" --uri viking://resources/table-metadata -n 5
|
||||
|
||||
# === 语义find:召回相关内容 ===
|
||||
conda run -n py13 ov find "{业务概念}" --uri "viking://resources/sql_snippets" --threshold 0.2
|
||||
conda run -n py13 ov find "{问题描述}" --uri "viking://resources/field-process-memory" --threshold 0.2
|
||||
|
||||
# === 使用源表信息搜索成功案例 ===
|
||||
conda run -n py13 ov grep "{源表表名}" --uri "viking://resources/sql_snippets" --node-limit 10
|
||||
conda run -n py13 ov grep "{源表表名}.*{字段}" --uri "viking://resources/field-process-memory" --node-limit 10
|
||||
conda run -n py13 ov find "{字段名} 问题描述" --uri "viking://resources/field-process-memory" --node-limit 5
|
||||
|
||||
# === 读取内容 ===
|
||||
conda run -n py13 ov overview viking://resources/table-metadata/{表名}
|
||||
conda run -n py13 ov read viking://resources/sql_snippets/{目录}/{文件}.sql
|
||||
conda run -n py13 ov read viking://resources/field-process-memory/{文件}.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 工作流程
|
||||
|
||||
```
|
||||
步骤1:需求分析 → 调用 requirement-analyzer + 数据源匹配的同时进行OV检索 + 困惑时OV检索
|
||||
↓
|
||||
步骤2:模型设计 → 调用 model-design-generator + 困惑时OV检索
|
||||
↓
|
||||
步骤3:SQL编写 → 调用 write-sql + 困惑时OV检索
|
||||
↓
|
||||
交付完成
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 执行流程详细步骤
|
||||
|
||||
### 步骤1:需求分析
|
||||
|
||||
**执行前检查**:无(直接开始)。
|
||||
|
||||
调用 `skill(name="requirement-analyzer")`
|
||||
|
||||
**子技能完成后,自然衔接**:
|
||||
```
|
||||
📄 已保存:{路径}
|
||||
**困惑时检索(py13 环境)**:
|
||||
- 不熟悉业务概念 → `conda run -n py13 ov find "{概念}" --uri "viking://resources/field-process-memory" --threshold 0.2`
|
||||
- 不确定指标口径 → `conda run -n py13 ov find "{指标名称} 口径" --uri "viking://resources/field-process-memory" --threshold 0.3`
|
||||
|
||||
下一步将生成模型设计。请确认以上需求是否正确,或指出需要修改的内容。
|
||||
**完成后衔接**:`📄 已保存:{路径}。需求分析完成,开始模型设计...`
|
||||
|
||||
#### ✅ MUST:数据源匹配后 OV 深度检索
|
||||
|
||||
**强制性说明**:data-structure-fetcher 返回推荐数据源后,**必须对每个推荐表进行全方位 OV 检索**。不检索会导致:
|
||||
- 猜测表结构 → 字段选错 → SQL 失败
|
||||
- 缺乏成功案例参考 → 重复踩坑 → 代码质量差
|
||||
- 不了解字段处理经验 → 口径偏差 → 业务错误
|
||||
|
||||
---
|
||||
|
||||
**触发时机**:requirement-analyzer 模块3,data-structure-fetcher 返回推荐表列表后,立即执行
|
||||
|
||||
**执行环境**:py13 conda 环境,使用 `conda run -n py13 ov ...`
|
||||
|
||||
---
|
||||
|
||||
**检索策略:每个推荐表必须执行以下 5 类检索**
|
||||
|
||||
| 检索类型 | 目的 | 命令模板 | node-limit |
|
||||
|---------|------|---------|------------|
|
||||
| ① 表结构检索 | 确认表字段定义、数据类型 | `conda run -n py13 ov grep "{表名}" --uri "viking://resources/table-metadata"` | 5-10 |
|
||||
| ② 成功SQL案例检索 | 查找该表的历史成功 SQL 代码 | `conda run -n py13 ov grep "{表名}" --uri "viking://resources/sql_snippets"` | 10-15 |
|
||||
| ③ 常用数据源判断 | **判断该表是否为常用数据源**(历史使用频率、稳定性、推荐度) | `conda run -n py13 ov grep "{表名}" --uri "viking://resources/field-process-memory"` | 5-10 |
|
||||
| ④ 字段处理经验检索 | 了解该表字段的常见处理方式、注意事项 | `conda run -n py13 ov grep "{表名}.*{字段名}" --uri "viking://resources/field-process-memory"` | 5-10 |
|
||||
| ⑤ 业务语义检索 | 了解该表在业务场景中的使用经验 | `conda run -n py13 ov find "{表名} {业务场景}" --uri "viking://resources/field-process-memory" --threshold 0.2` | 5-10 |
|
||||
|
||||
---
|
||||
|
||||
**③ 常用数据源判断 - 详细说明**
|
||||
|
||||
这是数据源匹配后**必须立即执行**的关键检索,用于判断推荐表是否值得采用:
|
||||
|
||||
**执行时机**:data-structure-fetcher 返回推荐表后,**立即执行此检索**
|
||||
|
||||
**检索命令**:
|
||||
```bash
|
||||
conda run -n py13 ov grep "{表名}" --uri "viking://resources/field-process-memory" --node-limit 10
|
||||
```
|
||||
|
||||
**判断标准**:
|
||||
|
||||
| 检索结果 | 判断 | 建议操作 |
|
||||
|---------|------|---------|
|
||||
| 有多条历史使用记录(≥3条) | ✅ **常用数据源** | 推荐优先采用,可参考历史处理经验 |
|
||||
| 有少量记录(1-2条) | ⚠️ **偶尔使用** | 可采用,需谨慎验证字段逻辑 |
|
||||
| 无检索结果 | ❓ **新/陌生数据源** | 需进一步验证表结构,或寻找替代表 |
|
||||
| 有警告/问题记录 | ⚠️ **有风险** | 需评估问题严重性,考虑替代方案 |
|
||||
|
||||
**结果展示格式**:
|
||||
```
|
||||
📋 常用数据源判断结果:
|
||||
|
||||
【dwd_crm_srv_complaint_rt】
|
||||
- 检索结果:找到 5 条历史使用记录
|
||||
- 判断:✅ 常用数据源(推荐采用)
|
||||
- 历史使用场景:投诉量统计、投诉处理时效、部门投诉排名
|
||||
- 注意事项:create_org_id 需关联组织表获取部门名称
|
||||
|
||||
【dim_intnl_org_new】
|
||||
- 检索结果:找到 8 条历史使用记录
|
||||
- 判断:✅ 常用数据源(高频使用)
|
||||
- 历史使用场景:组织维度统计、部门层级分析、归属地统计
|
||||
- 注意事项:org_name 有多级命名,需按业务口径标准化
|
||||
```
|
||||
|
||||
**决策建议**:
|
||||
- 多个推荐表中,优先选择"常用数据源"
|
||||
- 对"陌生数据源"需额外验证,或向用户确认是否采用
|
||||
- 对"有风险"数据源,建议告知用户并提供替代方案
|
||||
|
||||
---
|
||||
|
||||
**具体执行流程(py13 环境)**
|
||||
|
||||
```bash
|
||||
# 对每个推荐表(如:dwd_crm_srv_complaint_rt),依次执行:
|
||||
|
||||
# ① 表结构检索(确认字段)
|
||||
conda run -n py13 ov grep "dwd_crm_srv_complaint_rt" --uri "viking://resources/table-metadata" --node-limit 10
|
||||
|
||||
# ② 成功SQL案例检索(找历史代码)
|
||||
conda run -n py13 ov grep "dwd_crm_srv_complaint_rt" --uri "viking://resources/sql_snippets" --node-limit 15
|
||||
|
||||
# ③ 常用数据源判断(判断是否常用 - 数据源匹配后立即执行)
|
||||
conda run -n py13 ov grep "dwd_crm_srv_complaint_rt" --uri "viking://resources/field-process-memory" --node-limit 10
|
||||
|
||||
# ④ 字段处理经验检索(找处理笔记 - 对核心字段)
|
||||
conda run -n py13 ov grep "dwd_crm_srv_complaint_rt.complaint_id" --uri "viking://resources/field-process-memory" --node-limit 10
|
||||
|
||||
# ⑤ 业务语义检索(结合业务场景)
|
||||
conda run -n py13 ov find "dwd_crm_srv_complaint_rt 投诉统计" --uri "viking://resources/field-process-memory" --threshold 0.2 --node-limit 10
|
||||
```
|
||||
|
||||
**执行顺序建议**:
|
||||
- ③ 常用数据源判断是**最优先执行**的检索,应在数据源匹配后立即执行
|
||||
- 其他检索可根据常用数据源判断结果调整执行优先级
|
||||
|
||||
---
|
||||
|
||||
**检索结果利用**
|
||||
|
||||
检索完成后,必须:
|
||||
|
||||
1. **汇总展示**:将检索结果整理成摘要,展示给用户
|
||||
```
|
||||
📊 OV 检索结果摘要:
|
||||
|
||||
【dwd_crm_srv_complaint_rt】
|
||||
- 常用数据源:✅ 常用(找到 5 条历史使用记录)
|
||||
- 表结构:包含 complaint_id、create_org_id、assist_dept_org_id 等字段
|
||||
- 成功案例:找到 3 个历史 SQL(投诉量统计、投诉处理时效分析)
|
||||
- 处理经验:create_org_id 字段需关联组织架构表获取部门名称
|
||||
|
||||
【dim_intnl_org_new】
|
||||
- 常用数据源:✅ 常用(找到 8 条历史使用记录)
|
||||
- 表结构:包含 org_id、org_name、accnt_bureau 等字段
|
||||
- 成功案例:找到 5 个历史 SQL(部门维度统计)
|
||||
- 处理经验:org_name 有多级命名规范,需按业务口径标准化
|
||||
|
||||
【dwd_new_table_2024】
|
||||
- 常用数据源:❓ 陌生数据源(无历史使用记录)
|
||||
- 建议:需进一步验证表结构,或寻找更成熟的替代表
|
||||
```
|
||||
|
||||
2. **辅助源表选择**:基于常用数据源判断和成功案例数量,建议用户优先选择哪些表
|
||||
```
|
||||
建议:
|
||||
- ✅ 优先采用:dwd_crm_srv_complaint_rt(常用数据源,有 3 个成功案例,历史使用稳定)
|
||||
- ✅ 优先采用:dim_intnl_org_new(常用数据源,高频使用,组织表标准参考)
|
||||
- ⚠️ 需验证:dwd_new_table_2024(陌生数据源,建议先验证或寻找替代表)
|
||||
```
|
||||
|
||||
3. **预加载到上下文**:检索到的成功案例和字段处理经验,需在后续模型设计和 SQL 编写时引用
|
||||
|
||||
---
|
||||
|
||||
**检索时机强制触发表**
|
||||
|
||||
| 时间点 | 触发条件 | 检索内容 | 强制性 |
|
||||
|-------|---------|---------|--------|
|
||||
| data-structure-fetcher 返回后 | 推荐表列表返回 | **③ 常用数据源判断(最优先)** + 每个表的 5 类检索 | ✅ MUST |
|
||||
| 用户补充新表名时 | 用户提及陌生表名 | 立刻 OV 检索该表(优先执行常用数据源判断) | ✅ MUST |
|
||||
| 用户询问字段含义时 | 用户对字段有疑问 | OV grep 该表.字段 | ✅ MUST |
|
||||
| 发现字段歧义时 | 同一字段有多来源 | OV find 语义检索 | ✅ MUST |
|
||||
| 陌生数据源判断后 | 无历史使用记录 | 额外验证表结构,或建议替代表 | ✅ MUST |
|
||||
|
||||
---
|
||||
|
||||
**并行检索策略**
|
||||
|
||||
为提高效率,对多个推荐表的检索可并行执行:
|
||||
|
||||
```bash
|
||||
# 并行检索多个表(使用 task agent 或 bash 后台)
|
||||
conda run -n py13 ov grep "table1" --uri "viking://resources/table-metadata" --node-limit 10 &
|
||||
conda run -n py13 ov grep "table2" --uri "viking://resources/table-metadata" --node-limit 10 &
|
||||
conda run -n py13 ov grep "table3" --uri "viking://resources/table-metadata" --node-limit 10 &
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**关键字段专项检索**
|
||||
|
||||
当推荐表中包含**核心业务字段**时,必须额外进行字段级检索:
|
||||
|
||||
```bash
|
||||
# 对核心字段(如:complaint_id、org_id)进行专项检索
|
||||
conda run -n py13 ov grep "dwd_crm_srv_complaint_rt.complaint_id" --uri "viking://resources/table-metadata" --node-limit 5
|
||||
conda run -n py13 ov grep "complaint_id" --uri "viking://resources/field-process-memory" --node-limit 10
|
||||
conda run -n py13 ov find "complaint_id 去重计数" --uri "viking://resources/sql_snippets" --threshold 0.2 --node-limit 10
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 步骤2:模型设计
|
||||
**确认后进入**
|
||||
|
||||
调用 `skill(name="model-design-generator")`,传入需求文档路径
|
||||
**执行前检查**:需求分析已完成,用户已确认需求。
|
||||
|
||||
**子技能完成后,自然衔接**:
|
||||
```
|
||||
📄 已保存:{路径}
|
||||
调用 `skill(name="model-design-generator")`
|
||||
|
||||
下一步将编写SQL脚本。请确认以上设计是否正确,或指出需要修改的内容。
|
||||
```
|
||||
**困惑时检索(py13 环境)**:
|
||||
- 确认表结构 → `conda run -n py13 ov grep "{表名}" --uri "viking://resources/table-metadata"`
|
||||
- 了解分层规范 → `conda run -n py13 ov find "分层架构 DWA DWM ADS" --uri "viking://resources/sql_snippets" --threshold 0.2`
|
||||
|
||||
**完成后衔接**:`📄 已保存:{路径}。下一步将编写SQL脚本。请确认以上设计是否正确。`
|
||||
|
||||
---
|
||||
|
||||
### 步骤3:SQL编写
|
||||
**确认后进入**
|
||||
|
||||
调用 `skill(name="write-sql")`,传入需求文档和模型设计文档
|
||||
**执行前检查**:用户已确认模型设计。
|
||||
|
||||
**完成后自然衔接**:
|
||||
调用 `skill(name="write-sql")`
|
||||
|
||||
**困惑时检索(py13 环境)**:
|
||||
- SQL语法 → `conda run -n py13 ov find "{语法} 用法" --uri "viking://resources/sql_snippets" --threshold 0.2`
|
||||
- 窗口函数示例 → `conda run -n py13 ov find "窗口函数 over partition" --uri "viking://resources/sql_snippets" --threshold 0.2`
|
||||
|
||||
**完成后衔接**:
|
||||
```
|
||||
📄 已保存:{路径}
|
||||
|
||||
@@ -63,16 +380,20 @@ description: 智能数据开发员工。端到端将数据需求转化为可执
|
||||
- 模型设计:{路径}
|
||||
- SQL脚本:{路径}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 状态跟踪
|
||||
```python
|
||||
state = {
|
||||
"step": 0, # 0→1→1.5(等待)→2→2.5(等待)→3→4(完成)
|
||||
"confirmed": [False, False], # [步骤1确认, 步骤2确认]
|
||||
"paths": {"req": None, "model": None, "sql": None}
|
||||
}
|
||||
```
|
||||
|
||||
- step 1: 需求分析完成 → 准备模型设计
|
||||
- step 2: 模型设计完成 → 等待用户确认
|
||||
- step 3: SQL编写完成
|
||||
- step 4: 全流程完成
|
||||
|
||||
记录产出物路径:需求文档、模型设计文档、SQL脚本。
|
||||
|
||||
---
|
||||
|
||||
## 交互规则
|
||||
|
||||
| 场景 | 响应 |
|
||||
@@ -80,9 +401,51 @@ state = {
|
||||
| 用户确认 | 进入下一步 |
|
||||
| 用户修改意见 | 调整后重新等待确认 |
|
||||
| 中途修改 | 回到对应步骤重新执行 |
|
||||
| 用户提及陌生概念 | 先OV检索再回复,提供选项 |
|
||||
|
||||
---
|
||||
|
||||
## 交互工具
|
||||
|
||||
在需要补充和确认信息,检查和修改的时候,使用`question`工具提供选项供用户选择,例如下面的情况:
|
||||
|
||||
```
|
||||
例如下面的情况,请使用`question`工具:
|
||||
请补充/确认以下信息:
|
||||
1. 输出表中文名称:建议为"设备流量通话短信月均使用情况表",是否需要修改?
|
||||
2. 输出表英文名称:请提供库名.表名(如:db_eda_xxx_prd.ads_device_usage_avg_6m)
|
||||
3. 数据目录:请提供数据目录路径(如:上海电信/大数据中心-xxx团队/xxx应用层)
|
||||
4. 业务口径细节确认:
|
||||
- "近6个月"的具体时间范围是指:从当月向前推6个月(即统计月及之前5个月,共6个月)?
|
||||
- "平均值"计算方式:是计算每个设备在这6个月的平均值,还是其他口径?
|
||||
- 设备标识字段是什么?(如:设备号、用户ID、手机号等)
|
||||
请确认或补充以上信息,确认后进入模块3(数据源匹配)。
|
||||
```
|
||||
|
||||
```
|
||||
请检查并修改:
|
||||
- 字段是否完整?需要补充或删除哪些字段?
|
||||
- 字段名称是否需要调整?
|
||||
- 字段加工逻辑是否准确?来源表.字段是否正确?
|
||||
- 负责人信息是否正确?
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 困惑场景检索策略
|
||||
|
||||
**注意:所有 OV 检索命令需在 py13 conda 环境中运行**
|
||||
|
||||
| 困惑类型 | 检索资源 | 命令 |
|
||||
|---------|---------|------|
|
||||
| 不清楚表结构 | table-metadata | `conda run -n py13 ov grep "{表名}" --uri "viking://resources/table-metadata"` |
|
||||
| 不确定字段含义 | table-metadata | `conda run -n py13 ov grep "{表名}.*{字段名}" --uri "viking://resources/table-metadata"` 或者 `conda run -n py13 ov grep "{字段名}" --uri "viking://resources/table-metadata"` |
|
||||
| 不熟悉业务概念 | sql_snippets + field-process-memory | `conda run -n py13 ov find "{概念}" --uri "viking://resources/resources/field-process-memory" --threshold 0.2` |
|
||||
| 需要代码参考 | sql_snippets + field-process-memory | `conda run -n py13 ov find "{功能}" --uri "viking://resources/sql_snippets" --threshold 0.2` |
|
||||
| 找表成功案例 | sql_snippets + field-process-memory | 分别使用 `conda run -n py13 ov grep` 和 `conda run -n py13 ov find` 搜索两个资源 |
|
||||
|
||||
---
|
||||
|
||||
## 完成标志
|
||||
|
||||
三步骤完成,用户已知交付物位置。
|
||||
三步骤完成(需求分析 → 模型设计 → SQL编写),用户已知交付物位置。
|
||||
|
||||
Reference in New Issue
Block a user