Commit english docs
parent
1e05d4e444
commit
03d404e24f
@ -0,0 +1,212 @@
|
|||||||
|
---
|
||||||
|
title: "Chapter 1: ChatModel and Message (Console)"
|
||||||
|
---
|
||||||
|
|
||||||
|
## Introduction to the Eino Framework
|
||||||
|
|
||||||
|
**What is Eino?**
|
||||||
|
|
||||||
|
Eino is an AI application development framework (Agent Development Kit) implemented in Go, designed to help developers quickly build scalable, maintainable AI applications.
|
||||||
|
|
||||||
|
**What problems does Eino solve?**
|
||||||
|
|
||||||
|
1. **Model abstraction**: Unifies interfaces across different LLM providers (OpenAI, Ark, Claude, etc.) so switching models doesn't require modifying business code
|
||||||
|
2. **Capability composition**: Implements replaceable, composable capability units (conversation, tools, retrieval, etc.) through Component interfaces
|
||||||
|
3. **Orchestration framework**: Provides orchestration abstractions like Agent, Graph, and Chain, supporting complex multi-step AI workflows
|
||||||
|
4. **Runtime support**: Built-in capabilities for streaming output, interrupt and resume, state management, Callback observability, and more
|
||||||
|
|
||||||
|
**Eino's main repositories:**
|
||||||
|
- **eino** (this repository): Core library, defines interfaces, orchestration abstractions, and ADK
|
||||||
|
- **eino-ext**: Extension library, provides concrete implementations of various Components (OpenAI, Ark, Milvus, etc.)
|
||||||
|
- **eino-examples**: Example code repository, includes this quickstart series
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ChatWithEino: An Intelligent Assistant for Chatting with Eino Documentation
|
||||||
|
|
||||||
|
**What is ChatWithEino?**
|
||||||
|
|
||||||
|
ChatWithEino is an intelligent assistant built on the Eino framework that helps developers learn the Eino framework and write Eino code. By accessing the Eino repository's source code, comments, and examples, it provides users with the most accurate and up-to-date technical support.
|
||||||
|
|
||||||
|
**Core capabilities:**
|
||||||
|
- **Conversational interaction**: Understands user questions about Eino and provides clear answers
|
||||||
|
- **Code access**: Directly reads Eino source code, comments, and examples to answer questions based on real implementations
|
||||||
|
- **Persistent sessions**: Supports multi-turn conversations, remembers context, and can resume sessions across processes
|
||||||
|
- **Tool calls**: Can perform operations like file reading and code searching
|
||||||
|
|
||||||
|
**Technical architecture:**
|
||||||
|
- **ChatModel**: Communicates with large language models (OpenAI, Ark, Claude, etc.)
|
||||||
|
- **Tool**: Capability extensions like filesystem access and code search
|
||||||
|
- **Memory**: Persistent storage for conversation history
|
||||||
|
- **Agent**: Unified execution framework that coordinates components working together
|
||||||
|
|
||||||
|
## Quickstart Documentation Series: Building ChatWithEino from Scratch
|
||||||
|
|
||||||
|
This documentation series takes you step by step, starting from the most basic ChatModel call, gradually building a fully functional ChatWithEino Agent.
|
||||||
|
|
||||||
|
**Learning path:**
|
||||||
|
|
||||||
|
| Chapter | Topic | Core Content | Capability Gained |
|
||||||
|
|---------|-------|-------------|-------------------|
|
||||||
|
| **Chapter 1** | ChatModel and Message | Understand Component abstraction, implement single-turn conversation | Basic conversation |
|
||||||
|
| **Chapter 2** | Agent and Runner | Introduce execution abstraction, implement multi-turn conversation | Session management |
|
||||||
|
| **Chapter 3** | Memory and Session | Persist conversation history, support session recovery | Persistence |
|
||||||
|
| **Chapter 4** | Tool and Filesystem | Add file access capability, read source code | Tool calling |
|
||||||
|
| **Chapter 5** | Middleware | Middleware mechanism for unified cross-cutting concerns | Enhanced extensibility |
|
||||||
|
| **Chapter 6** | Callback | Callback mechanism for monitoring Agent execution | Observability |
|
||||||
|
| **Chapter 7** | Interrupt and Resume | Interrupt and resume, support for long-running tasks | Enhanced reliability |
|
||||||
|
| **Chapter 8** | Graph and Tool | Use Graph to orchestrate complex workflows | Complex orchestration |
|
||||||
|
| **Chapter 9** | Skill | Use Skill middleware to load and reuse skill documents | Knowledge reuse |
|
||||||
|
| **Final Chapter** | A2UI | Agent-to-UI integration solution | Production-grade application |
|
||||||
|
|
||||||
|
**Why is it designed this way?**
|
||||||
|
|
||||||
|
Each chapter adds one core capability on top of the previous one, allowing you to:
|
||||||
|
1. **Understand each component's role**: Instead of showing all features at once, they are introduced gradually
|
||||||
|
2. **See the architecture evolution**: From simple to complex, understand why each abstraction is needed
|
||||||
|
3. **Master practical development skills**: Each chapter has runnable code for hands-on practice
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
The goal of this chapter is to understand Eino's Component abstraction, call a ChatModel with minimal code (with streaming output support), and master the basic usage of `schema.Message`.
|
||||||
|
|
||||||
|
## Code Location
|
||||||
|
|
||||||
|
- Entry code: [cmd/ch01/main.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/cmd/ch01/main.go)
|
||||||
|
|
||||||
|
## Why We Need the Component Interface
|
||||||
|
|
||||||
|
Eino defines a set of Component interfaces (`ChatModel`, `Tool`, `Retriever`, `Loader`, etc.), each describing a replaceable capability:
|
||||||
|
|
||||||
|
```go
|
||||||
|
type BaseChatModel interface {
|
||||||
|
Generate(ctx context.Context, input []*schema.Message, opts ...Option) (*schema.Message, error)
|
||||||
|
Stream(ctx context.Context, input []*schema.Message, opts ...Option) (
|
||||||
|
*schema.StreamReader[*schema.Message], error)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Benefits of using interfaces:**
|
||||||
|
|
||||||
|
1. **Swappable implementations**: `eino-ext` provides multiple implementations including OpenAI, Ark, Claude, Ollama, etc. Business code only depends on the interface, so switching models only requires changing the construction logic.
|
||||||
|
2. **Composable orchestration**: Orchestration layers like Agent, Graph, and Chain only depend on Component interfaces, not specific implementations. You can swap OpenAI for Ark without changing orchestration code.
|
||||||
|
3. **Mockable for testing**: Interfaces naturally support mocking, so unit tests don't need real model calls.
|
||||||
|
|
||||||
|
This chapter only covers `ChatModel`. Subsequent chapters will gradually introduce `Tool`, `Retriever`, and other Components.
|
||||||
|
|
||||||
|
## schema.Message: The Basic Unit of Conversation
|
||||||
|
|
||||||
|
`Message` is the fundamental data structure for conversations in Eino:
|
||||||
|
|
||||||
|
```go
|
||||||
|
type Message struct {
|
||||||
|
Role RoleType // system / user / assistant / tool
|
||||||
|
Content string // Text content
|
||||||
|
ToolCalls []ToolCall // Only assistant messages may have this
|
||||||
|
// ...
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Common constructors:
|
||||||
|
|
||||||
|
```go
|
||||||
|
schema.SystemMessage("You are a helpful assistant.")
|
||||||
|
schema.UserMessage("What is the weather today?")
|
||||||
|
schema.AssistantMessage("I don't know.", nil) // Second parameter is ToolCalls
|
||||||
|
schema.ToolMessage("tool result", "call_id")
|
||||||
|
```
|
||||||
|
|
||||||
|
**Role semantics:**
|
||||||
|
- `system`: System instructions, usually placed at the beginning of the messages
|
||||||
|
- `user`: User input
|
||||||
|
- `assistant`: Model reply
|
||||||
|
- `tool`: Tool call result (covered in later chapters)
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
### Get the Code
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://github.com/cloudwego/eino-examples.git
|
||||||
|
cd eino-examples/quickstart/chatwitheino
|
||||||
|
```
|
||||||
|
|
||||||
|
- Go version: Go 1.21+ (see `go.mod`)
|
||||||
|
- A callable ChatModel (defaults to OpenAI; Ark is also supported)
|
||||||
|
|
||||||
|
### Option A: OpenAI (Default)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export OPENAI_API_KEY="..."
|
||||||
|
export OPENAI_MODEL="gpt-4.1-mini" # OpenAI's 2025 new model; gpt-4o, gpt-4o-mini, etc. also work
|
||||||
|
# Optional:
|
||||||
|
# OPENAI_BASE_URL (proxy or compatible service)
|
||||||
|
# OPENAI_BY_AZURE=true (use Azure OpenAI)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Option B: Ark
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export MODEL_TYPE="ark"
|
||||||
|
export ARK_API_KEY="..."
|
||||||
|
export ARK_MODEL="..."
|
||||||
|
# Optional: ARK_BASE_URL
|
||||||
|
```
|
||||||
|
|
||||||
|
## Running
|
||||||
|
|
||||||
|
In the `examples/quickstart/chatwitheino` directory, run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
go run ./cmd/ch01 -- "Explain in one sentence what problem Eino's Component design solves?"
|
||||||
|
```
|
||||||
|
|
||||||
|
Output example (streamed progressively):
|
||||||
|
|
||||||
|
```text
|
||||||
|
[assistant] Eino's Component design solves the problem of...
|
||||||
|
```
|
||||||
|
|
||||||
|
## What the Entry Code Does
|
||||||
|
|
||||||
|
In execution order:
|
||||||
|
|
||||||
|
1. **Create ChatModel**: Selects OpenAI or Ark implementation based on the `MODEL_TYPE` environment variable
|
||||||
|
2. **Construct input messages**: `SystemMessage(instruction)` + `UserMessage(query)`
|
||||||
|
3. **Call Stream**: All ChatModel implementations must support `Stream()`, which returns `StreamReader[*Message]`
|
||||||
|
4. **Print results**: Iterates through the `StreamReader` to print the assistant reply frame by frame
|
||||||
|
|
||||||
|
Key code snippet (**Note: this is a simplified code snippet that cannot be run directly. For the complete code, please refer to** [cmd/ch01/main.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/cmd/ch01/main.go)):
|
||||||
|
|
||||||
|
```go
|
||||||
|
// Construct input
|
||||||
|
messages := []*schema.Message{
|
||||||
|
schema.SystemMessage(instruction),
|
||||||
|
schema.UserMessage(query),
|
||||||
|
}
|
||||||
|
|
||||||
|
// Call Stream (all ChatModels must implement this)
|
||||||
|
stream, err := cm.Stream(ctx, messages)
|
||||||
|
if err != nil {
|
||||||
|
log.Fatal(err)
|
||||||
|
}
|
||||||
|
defer stream.Close()
|
||||||
|
|
||||||
|
for {
|
||||||
|
chunk, err := stream.Recv()
|
||||||
|
if errors.Is(err, io.EOF) {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
log.Fatal(err)
|
||||||
|
}
|
||||||
|
fmt.Print(chunk.Content)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Chapter Summary
|
||||||
|
|
||||||
|
- **Component interface**: Defines replaceable, composable, and testable capability boundaries
|
||||||
|
- **Message**: The basic unit of conversation data, with semantics distinguished by role
|
||||||
|
- **ChatModel**: The most fundamental Component, providing two core methods: `Generate` and `Stream`
|
||||||
|
- **Implementation selection**: Switch between OpenAI/Ark and other implementations via environment variables or configuration, with no changes needed in business code
|
||||||
@ -0,0 +1,296 @@
|
|||||||
|
---
|
||||||
|
title: "Chapter 2: ChatModelAgent, Runner, AgentEvent (Console Multi-turn)"
|
||||||
|
---
|
||||||
|
|
||||||
|
The goal of this chapter is to introduce the ADK's execution abstractions (Agent + Runner) and implement a multi-turn conversation using a Console program.
|
||||||
|
|
||||||
|
## Code Location
|
||||||
|
|
||||||
|
- Entry code: [cmd/ch02/main.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/cmd/ch02/main.go)
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
Same as Chapter 1: you need to configure an available ChatModel (OpenAI or Ark).
|
||||||
|
|
||||||
|
## Running
|
||||||
|
|
||||||
|
In the `examples/quickstart/chatwitheino` directory, run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
go run ./cmd/ch02
|
||||||
|
```
|
||||||
|
|
||||||
|
After seeing the prompt, enter your question (empty line to exit):
|
||||||
|
|
||||||
|
```text
|
||||||
|
you> Hello, explain what an Agent is in Eino?
|
||||||
|
...
|
||||||
|
you> Summarize that in one sentence
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
## Key Concepts
|
||||||
|
|
||||||
|
### From Component to Agent
|
||||||
|
|
||||||
|
In Chapter 1, we learned about **Components**, which are replaceable, composable capability units in Eino:
|
||||||
|
|
||||||
|
- `ChatModel`: Call large language models
|
||||||
|
- `Tool`: Execute specific tasks
|
||||||
|
- `Retriever`: Retrieve information
|
||||||
|
- `Loader`: Load data
|
||||||
|
|
||||||
|
**The relationship between Component and Agent:**
|
||||||
|
|
||||||
|
- **Components don't form a complete AI application**: They are just capability units that need to be organized, orchestrated, and executed
|
||||||
|
- **An Agent is a complete AI application**: It encapsulates complete business logic and can run directly
|
||||||
|
- **Agents use Components internally**: The most essential ones are `ChatModel` (conversation capability) and `Tool` (execution capability)
|
||||||
|
|
||||||
|
**Why do we need Agents?**
|
||||||
|
|
||||||
|
With only Components, you would need to handle on your own:
|
||||||
|
- Managing conversation history
|
||||||
|
- Orchestrating the call flow (when to call the model, when to call tools)
|
||||||
|
- Handling streaming output
|
||||||
|
- Implementing interruption and recovery
|
||||||
|
- ...
|
||||||
|
|
||||||
|
**What does an Agent provide?**
|
||||||
|
|
||||||
|
- **A complete runtime framework**: Unified execution management through `Runner`
|
||||||
|
- **Standardized event stream output**: `Run() -> AsyncIterator[*AgentEvent]`, supporting streaming, interruption, and recovery
|
||||||
|
- **Extensible capabilities**: You can add tools, middleware, interrupt, etc.
|
||||||
|
- **Ready to use out of the box**: Once an Agent is created, you can run it directly without worrying about internal details
|
||||||
|
|
||||||
|
**This chapter's example:**
|
||||||
|
|
||||||
|
`ChatModelAgent` is the simplest Agent. It only uses a `ChatModel` internally, but already has the complete Agent capability framework. Subsequent chapters will show how to add `Tool` and other capabilities.
|
||||||
|
|
||||||
|
### Agent Interface
|
||||||
|
|
||||||
|
`Agent` is the core interface in ADK, defining the basic behavior of an intelligent agent:
|
||||||
|
|
||||||
|
```go
|
||||||
|
type Agent interface {
|
||||||
|
Name(ctx context.Context) string
|
||||||
|
Description(ctx context.Context) string
|
||||||
|
|
||||||
|
// Run executes the Agent and returns an event stream
|
||||||
|
Run(ctx context.Context, input *AgentInput, options ...AgentRunOption) *AsyncIterator[*AgentEvent]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Interface responsibilities:**
|
||||||
|
- `Name()` / `Description()`: Identify the Agent's name and description
|
||||||
|
- `Run()`: The core method for executing the Agent, receiving input messages and returning an event stream
|
||||||
|
|
||||||
|
**Design philosophy:**
|
||||||
|
- **Unified abstraction**: All Agents (ChatModelAgent, WorkflowAgent, SupervisorAgent, etc.) implement this interface
|
||||||
|
- **Event-driven**: Outputs the execution process through an event stream (`AsyncIterator[*AgentEvent]`), supporting streaming responses
|
||||||
|
- **Extensibility**: When adding tools, middleware, interrupt, and other capabilities later, the interface remains unchanged
|
||||||
|
|
||||||
|
### ChatModelAgent
|
||||||
|
|
||||||
|
`ChatModelAgent` is an implementation of the Agent interface, built on top of ChatModel:
|
||||||
|
|
||||||
|
```go
|
||||||
|
agent, err := adk.NewChatModelAgent(ctx, &adk.ChatModelAgentConfig{
|
||||||
|
Name: "Ch02ChatModelAgent",
|
||||||
|
Description: "A minimal ChatModelAgent with in-memory multi-turn history.",
|
||||||
|
Instruction: instruction,
|
||||||
|
Model: cm,
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
**ChatModel vs ChatModelAgent: The Essential Difference**
|
||||||
|
|
||||||
|
| Dimension | ChatModel | ChatModelAgent |
|
||||||
|
|-----------|-----------|----------------|
|
||||||
|
| **Role** | Component | Agent |
|
||||||
|
| **Interface** | `Generate() / Stream()` | `Run() -> AsyncIterator[*AgentEvent]` |
|
||||||
|
| **Output** | Directly returns message content | Returns an event stream (containing messages, control actions, etc.) |
|
||||||
|
| **Capabilities** | Pure model invocation | Extensible with tools, middleware, interrupt, etc. |
|
||||||
|
| **Use case** | Simple conversation scenarios | Complex agent applications |
|
||||||
|
|
||||||
|
**Why do we need ChatModelAgent?**
|
||||||
|
|
||||||
|
1. **Unified abstraction**: ChatModel is just one type of Component, while Agent is a higher-level abstraction that can compose multiple Components
|
||||||
|
2. **Event-driven**: Agent outputs an event stream, supporting streaming responses, interruption recovery, state transitions, and other complex scenarios
|
||||||
|
3. **Extensibility**: ChatModelAgent can have tools, middleware, interrupt, and other capabilities added, while ChatModel can only call the model
|
||||||
|
4. **Orchestration-friendly**: Agents can be uniformly managed by Runner, supporting checkpoint, recovery, and other runtime capabilities
|
||||||
|
|
||||||
|
**In simple terms:**
|
||||||
|
- **ChatModel** = "The component responsible for communicating with the large language model, abstracting away differences between model providers (OpenAI, Ark, Claude, etc.)"
|
||||||
|
- **ChatModelAgent** = "An intelligent agent built on top of the model; it can call the model, but can also do much more"
|
||||||
|
|
||||||
|
**Analogy:**
|
||||||
|
- **ChatModel** is like a "database driver": responsible for communicating with the database, abstracting away MySQL/PostgreSQL differences
|
||||||
|
- **ChatModelAgent** is like the "business logic layer": built on top of the database driver, but also includes business rules, transaction management, etc.
|
||||||
|
|
||||||
|
**Features:**
|
||||||
|
- Encapsulates the ChatModel invocation logic
|
||||||
|
- Provides a unified `Run() -> AgentEvent` output format
|
||||||
|
- Can have tools, middleware, and other capabilities added later
|
||||||
|
|
||||||
|
### Runner
|
||||||
|
|
||||||
|
`Runner` is the entry point for executing an Agent, responsible for managing the Agent's lifecycle:
|
||||||
|
|
||||||
|
```go
|
||||||
|
type Runner struct {
|
||||||
|
a Agent // The Agent to execute
|
||||||
|
enableStreaming bool
|
||||||
|
store CheckPointStore // State storage for interruption recovery
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Why do we need Runner?**
|
||||||
|
|
||||||
|
Although Agent provides a `Run()` method, calling it directly would lack many runtime capabilities:
|
||||||
|
|
||||||
|
1. **Lifecycle management**: Runner manages the Agent's startup, recovery, interruption, and other states
|
||||||
|
2. **Checkpoint support**: Works with `CheckPointStore` to implement interruption recovery (covered in later chapters)
|
||||||
|
3. **Unified entry point**: Provides convenient methods like `Run()` and `Query()`
|
||||||
|
4. **Event stream encapsulation**: Converts the Agent's event stream into a consumable `AsyncIterator[*AgentEvent]`
|
||||||
|
|
||||||
|
**Usage:**
|
||||||
|
|
||||||
|
```go
|
||||||
|
runner := adk.NewRunner(ctx, adk.RunnerConfig{
|
||||||
|
Agent: agent,
|
||||||
|
EnableStreaming: true,
|
||||||
|
})
|
||||||
|
|
||||||
|
// Method 1: Pass in a message list
|
||||||
|
events := runner.Run(ctx, history)
|
||||||
|
|
||||||
|
// Method 2: Convenience method, pass in a single query string
|
||||||
|
events := runner.Query(ctx, "Hello")
|
||||||
|
```
|
||||||
|
|
||||||
|
### AgentEvent
|
||||||
|
|
||||||
|
`AgentEvent` is the event unit returned by Runner:
|
||||||
|
|
||||||
|
```go
|
||||||
|
type AgentEvent struct {
|
||||||
|
AgentName string
|
||||||
|
RunPath []RunStep
|
||||||
|
|
||||||
|
Output *AgentOutput // Output content
|
||||||
|
Action *AgentAction // Control action
|
||||||
|
Err error // Execution error
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Main fields:**
|
||||||
|
- `event.Err`: Execution error
|
||||||
|
- `event.Output.MessageOutput`: Message or message stream (streaming)
|
||||||
|
- `event.Action`: Control actions like interrupt/transfer/exit (used in later chapters)
|
||||||
|
|
||||||
|
### AsyncIterator: How to Consume the Event Stream
|
||||||
|
|
||||||
|
`Runner.Run()` returns `*AsyncIterator[*AgentEvent]`, which is a non-blocking streaming iterator.
|
||||||
|
|
||||||
|
**Why use AsyncIterator instead of returning results directly?**
|
||||||
|
|
||||||
|
Because Agent execution is **streaming**: the model generates replies token by token, with Tool calls interspersed. If you wait for everything to complete before returning, the user would have to wait longer. `AsyncIterator` lets you consume each event in real time.
|
||||||
|
|
||||||
|
**How to consume:**
|
||||||
|
|
||||||
|
```go
|
||||||
|
// events is *AsyncIterator[*AgentEvent], returned by runner.Run()
|
||||||
|
events := runner.Run(ctx, history)
|
||||||
|
|
||||||
|
for {
|
||||||
|
event, ok := events.Next() // Get the next event, blocks until an event is available or the stream ends
|
||||||
|
if !ok {
|
||||||
|
break // Iterator closed, all events consumed
|
||||||
|
}
|
||||||
|
if event.Err != nil {
|
||||||
|
// Handle error
|
||||||
|
}
|
||||||
|
if event.Output != nil && event.Output.MessageOutput != nil {
|
||||||
|
// Handle message output (may be streaming)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Note:** Each `runner.Run()` creates a new iterator; it cannot be reused after being consumed once.
|
||||||
|
|
||||||
|
## Implementing Multi-turn Conversation
|
||||||
|
|
||||||
|
This chapter implements simple multi-turn conversation: user input -> model reply -> user continues input -> ...
|
||||||
|
|
||||||
|
**Implementation approach:**
|
||||||
|
|
||||||
|
Without tools, `ChatModelAgent` only completes one round of model invocation per `Run()` call. Multi-turn conversation is achieved by maintaining history on the caller side:
|
||||||
|
|
||||||
|
1. Use `history []*schema.Message` to store the accumulated conversation
|
||||||
|
2. On each user input: append the `UserMessage` to history
|
||||||
|
3. Call `runner.Run(ctx, history)` to get the event stream, consume it to get the assistant text
|
||||||
|
4. Append the assistant text back to history, then proceed to the next round
|
||||||
|
|
||||||
|
**Key code snippet (Note: this is a simplified code snippet that cannot be run directly. For the complete code, please refer to** [cmd/ch02/main.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/cmd/ch02/main.go)**)**:
|
||||||
|
|
||||||
|
```go
|
||||||
|
history := make([]*schema.Message, 0, 16)
|
||||||
|
|
||||||
|
for {
|
||||||
|
// 1. Read user input
|
||||||
|
line := readUserInput()
|
||||||
|
if line == "" {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
|
||||||
|
// 2. Append user message to history
|
||||||
|
history = append(history, schema.UserMessage(line))
|
||||||
|
|
||||||
|
// 3. Call Runner to execute Agent
|
||||||
|
events := runner.Run(ctx, history)
|
||||||
|
|
||||||
|
// 4. Consume event stream, collect assistant reply
|
||||||
|
content := collectAssistantFromEvents(events)
|
||||||
|
|
||||||
|
// 5. Append assistant message to history
|
||||||
|
history = append(history, schema.AssistantMessage(content, nil))
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Flow diagram:**
|
||||||
|
|
||||||
|
```
|
||||||
|
+------------------------------------------+
|
||||||
|
| Initialize history = [] |
|
||||||
|
+------------------------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| User inputs UserMessage |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| Append to history |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| runner.Run(history) |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| Consume event stream |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| Append AssistantMessage |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
(loop continues)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Chapter Summary
|
||||||
|
|
||||||
|
- **Agent interface**: Defines the basic behavior of an intelligent agent; the core is `Run() -> AsyncIterator[*AgentEvent]`
|
||||||
|
- **ChatModelAgent**: An Agent implementation based on ChatModel, providing a unified execution abstraction
|
||||||
|
- **Runner**: The execution entry point for Agents, managing lifecycle, checkpoint, event stream, and other runtime capabilities
|
||||||
|
- **AgentEvent**: An event-driven output unit, supporting streaming responses and control actions
|
||||||
|
- **Multi-turn conversation**: Implemented by maintaining history on the caller side; each `Run()` completes one round of conversation
|
||||||
@ -0,0 +1,303 @@
|
|||||||
|
---
|
||||||
|
title: "Chapter 3: Memory and Session (Persistent Conversations)"
|
||||||
|
---
|
||||||
|
|
||||||
|
The goal of this chapter is to implement persistent storage for conversation history, supporting session recovery across processes.
|
||||||
|
|
||||||
|
> **Warning: Business Layer Concepts vs Framework Concepts**
|
||||||
|
>
|
||||||
|
> The **Memory, Session, and Store concepts introduced in this chapter are business layer concepts**, **not core components of the Eino framework**.
|
||||||
|
>
|
||||||
|
> - **Eino framework level**: Only provides basic abstractions like `adk.Runner` and `schema.Message`; the framework itself does not concern itself with how conversation history is stored
|
||||||
|
> - **Business layer level**: Memory/Session/Store are business logic designed in this example project to implement persistent conversations, interacting with the Eino framework by assembling input for `adk.Runner`
|
||||||
|
>
|
||||||
|
> In other words, the Eino framework is only responsible for "how to process messages", while "how to store messages" is entirely up to the business layer. The implementation provided in this chapter is just a simple reference example — you can choose a completely different storage solution (database, Redis, cloud storage, etc.) based on your business needs.
|
||||||
|
|
||||||
|
## Code Location
|
||||||
|
|
||||||
|
- Entry code: [cmd/ch03/main.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/cmd/ch03/main.go)
|
||||||
|
- Memory implementation: [mem/store.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/mem/store.go)
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
Same as Chapter 1: you need to configure an available ChatModel (OpenAI or Ark).
|
||||||
|
|
||||||
|
## Running
|
||||||
|
|
||||||
|
In the `examples/quickstart/chatwitheino` directory, run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Create a new session
|
||||||
|
go run ./cmd/ch03
|
||||||
|
|
||||||
|
# Resume an existing session
|
||||||
|
go run ./cmd/ch03 --session <session-id>
|
||||||
|
```
|
||||||
|
|
||||||
|
Output example:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Created new session: 083d16da-6b13-4fe6-afb0-c45d8f490ce1
|
||||||
|
Session title: New Session
|
||||||
|
Enter your message (empty line to exit):
|
||||||
|
you> Hello, my name is Zhang San
|
||||||
|
[assistant] Hello Zhang San! Nice to meet you...
|
||||||
|
you> What's my name?
|
||||||
|
[assistant] Your name is Zhang San...
|
||||||
|
|
||||||
|
Session saved: 083d16da-6b13-4fe6-afb0-c45d8f490ce1
|
||||||
|
Resume with: go run ./cmd/ch03 --session 083d16da-6b13-4fe6-afb0-c45d8f490ce1
|
||||||
|
```
|
||||||
|
|
||||||
|
## From In-Memory to Persistent: Why We Need Memory
|
||||||
|
|
||||||
|
In Chapter 2, we implemented multi-turn conversation, but there was a problem: **conversation history only existed in memory**.
|
||||||
|
|
||||||
|
**Limitations of in-memory storage:**
|
||||||
|
- Conversation history is lost when the process exits
|
||||||
|
- Cannot resume sessions across devices or processes
|
||||||
|
- Cannot implement session management (listing, deleting, searching, etc.)
|
||||||
|
|
||||||
|
**The role of Memory:**
|
||||||
|
- **Memory is persistent storage for conversation history**: Saves conversations to disk or a database
|
||||||
|
- **Memory supports Session management**: Each Session represents a complete conversation
|
||||||
|
- **Memory is decoupled from Agent**: The Agent doesn't care about storage details, only about the message list
|
||||||
|
|
||||||
|
**Simple analogy:**
|
||||||
|
- **In-memory storage** = "scratch paper" (gone when the process exits)
|
||||||
|
- **Memory** = "notebook" (permanently saved, can be reviewed at any time)
|
||||||
|
|
||||||
|
## Key Concepts
|
||||||
|
|
||||||
|
> **Reminder**: The following Session, Store, and other concepts are all **business layer implementations** used to manage conversation history storage. The Eino framework itself does not provide these components — the business layer is responsible for managing the message list, then passing messages to `adk.Runner` for processing.
|
||||||
|
|
||||||
|
### Session (Business Layer Concept)
|
||||||
|
|
||||||
|
`Session` represents a complete conversation session:
|
||||||
|
|
||||||
|
```go
|
||||||
|
type Session struct {
|
||||||
|
ID string
|
||||||
|
CreatedAt time.Time
|
||||||
|
|
||||||
|
messages []*schema.Message // Conversation history
|
||||||
|
// ...
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Core methods:**
|
||||||
|
- `Append(msg)`: Append a message to the session and persist it
|
||||||
|
- `GetMessages()`: Get all messages
|
||||||
|
- `Title()`: Generate a session title from the first user message
|
||||||
|
|
||||||
|
### Store (Business Layer Concept)
|
||||||
|
|
||||||
|
`Store` manages persistent storage for multiple Sessions:
|
||||||
|
|
||||||
|
```go
|
||||||
|
type Store struct {
|
||||||
|
dir string // Storage directory
|
||||||
|
cache map[string]*Session // In-memory cache
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Core methods:**
|
||||||
|
- `GetOrCreate(id)`: Get or create a Session
|
||||||
|
- `List()`: List all Sessions
|
||||||
|
- `Delete(id)`: Delete a Session
|
||||||
|
|
||||||
|
### JSONL File Format
|
||||||
|
|
||||||
|
Each Session is stored as a `.jsonl` file:
|
||||||
|
|
||||||
|
```jsonl
|
||||||
|
{"type":"session","id":"083d16da-...","created_at":"2026-03-11T10:00:00Z"}
|
||||||
|
{"role":"user","content":"Hello, who am I?"}
|
||||||
|
{"role":"assistant","content":"Hello! I don't know who you are yet..."}
|
||||||
|
{"role":"user","content":"My name is Zhang San"}
|
||||||
|
{"role":"assistant","content":"Got it, Zhang San, nice to meet you!"}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Why JSONL?**
|
||||||
|
- **Simple**: One JSON object per line, easy to read and write
|
||||||
|
- **Extensible**: New messages can be appended without rewriting the entire file
|
||||||
|
- **Readable**: Can be viewed directly with a text editor
|
||||||
|
- **Fault-tolerant**: A corrupted line doesn't affect other lines
|
||||||
|
|
||||||
|
## Memory Implementation (Business Layer Example)
|
||||||
|
|
||||||
|
Below is a simple business layer implementation example that uses JSONL files to store conversation history. This is just one of many possible implementations — you can choose databases, Redis, or other storage solutions based on your actual needs.
|
||||||
|
|
||||||
|
### 1. Create a Store
|
||||||
|
|
||||||
|
```go
|
||||||
|
sessionDir := "./data/sessions"
|
||||||
|
store, err := mem.NewStore(sessionDir)
|
||||||
|
if err != nil {
|
||||||
|
log.Fatal(err)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Get or Create a Session
|
||||||
|
|
||||||
|
```go
|
||||||
|
sessionID := "083d16da-6b13-4fe6-afb0-c45d8f490ce1"
|
||||||
|
session, err := store.GetOrCreate(sessionID)
|
||||||
|
if err != nil {
|
||||||
|
log.Fatal(err)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Append a User Message
|
||||||
|
|
||||||
|
```go
|
||||||
|
userMsg := schema.UserMessage("Hello")
|
||||||
|
if err := session.Append(userMsg); err != nil {
|
||||||
|
log.Fatal(err)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. Get History and Call the Agent
|
||||||
|
|
||||||
|
```go
|
||||||
|
history := session.GetMessages()
|
||||||
|
events := runner.Run(ctx, history)
|
||||||
|
content := collectAssistantFromEvents(events)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5. Append the Assistant Message
|
||||||
|
|
||||||
|
```go
|
||||||
|
assistantMsg := schema.AssistantMessage(content, nil)
|
||||||
|
if err := session.Append(assistantMsg); err != nil {
|
||||||
|
log.Fatal(err)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Key code snippet (Note: this is a simplified code snippet that cannot be run directly. For the complete code, please refer to** [cmd/ch03/main.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/cmd/ch03/main.go)**)**:
|
||||||
|
|
||||||
|
```go
|
||||||
|
// Create or resume a Session
|
||||||
|
session, err := store.GetOrCreate(sessionID)
|
||||||
|
if err != nil {
|
||||||
|
log.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// User input
|
||||||
|
userMsg := schema.UserMessage(line)
|
||||||
|
if err := session.Append(userMsg); err != nil {
|
||||||
|
log.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Call the Agent
|
||||||
|
history := session.GetMessages()
|
||||||
|
events := runner.Run(ctx, history)
|
||||||
|
content := collectAssistantFromEvents(events)
|
||||||
|
|
||||||
|
// Save assistant reply
|
||||||
|
assistantMsg := schema.AssistantMessage(content, nil)
|
||||||
|
if err := session.Append(assistantMsg); err != nil {
|
||||||
|
log.Fatal(err)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## The Relationship Between Session and Agent: Business Layer and Framework Layer Collaboration
|
||||||
|
|
||||||
|
**Key understanding:**
|
||||||
|
- **Session is a business layer concept**: Implemented and managed by business code, responsible for storing and loading conversation history
|
||||||
|
- **Agent (Runner) is a framework layer concept**: Provided by the Eino framework, responsible for processing messages and generating replies
|
||||||
|
- **Their interaction point**: The business layer uses `session.GetMessages()` to get the message list, which is passed to `runner.Run(ctx, history)` for processing
|
||||||
|
|
||||||
|
**Architecture layers:**
|
||||||
|
|
||||||
|
```
|
||||||
|
+-------------------------------------------------------------+
|
||||||
|
| Business Layer (Your Code) |
|
||||||
|
| +-------------+ +--------------+ +---------------+ |
|
||||||
|
| | Session |--->| GetMessages() |--->| runner.Run() | |
|
||||||
|
| | (Storage) | | (Message List)| | (Framework) | |
|
||||||
|
| +-------------+ +--------------+ +---------------+ |
|
||||||
|
| ^ | |
|
||||||
|
| | v |
|
||||||
|
| +-------------+ +---------------+ |
|
||||||
|
| | Append() |<--------------------| Assistant Reply| |
|
||||||
|
| | (Save Msg) | +---------------+ |
|
||||||
|
| +-------------+ |
|
||||||
|
+-------------------------------------------------------------+
|
||||||
|
|
|
||||||
|
v
|
||||||
|
+-------------------------------------------------------------+
|
||||||
|
| Framework Layer (Eino Framework) |
|
||||||
|
| +-------------------------------------------------------+ |
|
||||||
|
| | adk.Runner: Receives message list, calls ChatModel, | |
|
||||||
|
| | returns reply | |
|
||||||
|
| +-------------------------------------------------------+ |
|
||||||
|
+-------------------------------------------------------------+
|
||||||
|
```
|
||||||
|
|
||||||
|
**Flow diagram:**
|
||||||
|
|
||||||
|
```
|
||||||
|
+------------------------------------------+
|
||||||
|
| User Input |
|
||||||
|
+------------------------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| session.Append() |
|
||||||
|
| Save user message |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| session.GetMessages() |
|
||||||
|
| Get complete history |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| runner.Run(history) |
|
||||||
|
| Agent processes msgs |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| Collect assistant |
|
||||||
|
| reply |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| session.Append() |
|
||||||
|
| Save assistant message|
|
||||||
|
+------------------------+
|
||||||
|
```
|
||||||
|
|
||||||
|
## Chapter Summary
|
||||||
|
|
||||||
|
**Framework Layer vs Business Layer:**
|
||||||
|
- **Eino framework layer**: Provides basic abstractions like `adk.Runner` and `schema.Message`; does not concern itself with how messages are stored
|
||||||
|
- **Business layer (this chapter's implementation)**: Memory/Session/Store are business layer concepts used to manage conversation history storage
|
||||||
|
|
||||||
|
**Business layer concepts:**
|
||||||
|
- **Memory**: Persistent storage for conversation history, supporting cross-process recovery
|
||||||
|
- **Session**: A complete conversation session, containing ID, creation time, and message list
|
||||||
|
- **Store**: Manages storage for multiple Sessions, supporting create, get, list, and delete operations
|
||||||
|
- **JSONL format**: A simple file format, easy to read, write, and extend
|
||||||
|
|
||||||
|
**Business layer and framework layer interaction:**
|
||||||
|
- The business layer is responsible for storing messages and uses `session.GetMessages()` to get the message list
|
||||||
|
- The message list is passed to the framework layer's `runner.Run(ctx, history)` for processing
|
||||||
|
- The reply returned by the framework layer is collected and then saved to storage by the business layer
|
||||||
|
|
||||||
|
> **Tip**: The implementation in this chapter is just one simple example among many storage solutions. In real projects, you can choose databases, Redis, cloud storage, or other solutions based on business needs, and even implement more complex features like session expiration cleanup, search, sharing, etc.
|
||||||
|
|
||||||
|
## Further Thinking: Choosing a Business Layer Storage Solution
|
||||||
|
|
||||||
|
The JSONL file storage solution provided in this chapter is suitable for simple single-machine applications. In real business scenarios, you may need to consider other storage solutions:
|
||||||
|
|
||||||
|
**Other storage implementations:**
|
||||||
|
- Database storage (MySQL, PostgreSQL, MongoDB)
|
||||||
|
- Redis storage (supports distributed setups)
|
||||||
|
- Cloud storage (S3, OSS)
|
||||||
|
|
||||||
|
**Advanced features:**
|
||||||
|
- Session expiration cleanup
|
||||||
|
- Session search
|
||||||
|
- Session export/import
|
||||||
|
- Session sharing
|
||||||
@ -0,0 +1,314 @@
|
|||||||
|
---
|
||||||
|
title: "Chapter 4: Tool and Filesystem Access"
|
||||||
|
---
|
||||||
|
|
||||||
|
The goal of this chapter is to add Tool capabilities to the Agent, enabling it to access the filesystem.
|
||||||
|
|
||||||
|
## Why We Need Tools
|
||||||
|
|
||||||
|
In the first three chapters, the Agent we implemented could only have conversations — it couldn't perform actual operations.
|
||||||
|
|
||||||
|
**Agent limitations:**
|
||||||
|
- Can only generate text replies
|
||||||
|
- Cannot access external resources (files, APIs, databases, etc.)
|
||||||
|
- Cannot execute actual tasks (calculations, queries, modifications, etc.)
|
||||||
|
|
||||||
|
**The role of Tools:**
|
||||||
|
- **Tools are capability extensions for the Agent**: They enable the Agent to perform concrete operations
|
||||||
|
- **Tools encapsulate specific implementations**: The Agent doesn't care how a Tool works internally, only about its input and output
|
||||||
|
- **Tools are composable**: An Agent can have multiple Tools and choose which to call as needed
|
||||||
|
|
||||||
|
**Simple analogy:**
|
||||||
|
- **Agent** = "intelligent assistant" (can understand instructions, but needs tools to execute)
|
||||||
|
- **Tool** = "toolbox" (file operations, network requests, database queries, etc.)
|
||||||
|
|
||||||
|
## Why We Need Filesystem Access
|
||||||
|
|
||||||
|
This example is ChatWithDoc (chat with documentation), with the goal of helping users learn the Eino framework and write Eino code. So, what is the best documentation?
|
||||||
|
|
||||||
|
**The answer is: the Eino repository's code itself.**
|
||||||
|
|
||||||
|
- **Code**: Source code shows the framework's actual implementation
|
||||||
|
- **Comments**: Code comments provide design rationale and usage instructions
|
||||||
|
- **Examples**: Example code demonstrates best practices
|
||||||
|
|
||||||
|
With filesystem access capabilities, the Agent can directly read Eino source code, comments, and examples, providing users with the most accurate and up-to-date technical support.
|
||||||
|
|
||||||
|
## Key Concepts
|
||||||
|
|
||||||
|
### Tool Interface
|
||||||
|
|
||||||
|
`Tool` is the interface in Eino that defines executable capabilities:
|
||||||
|
|
||||||
|
```go
|
||||||
|
// BaseTool provides tool metadata that ChatModel uses to decide whether and how to call the tool
|
||||||
|
type BaseTool interface {
|
||||||
|
Info(ctx context.Context) (*schema.ToolInfo, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// InvokableTool is a tool that can be executed by ToolsNode
|
||||||
|
type InvokableTool interface {
|
||||||
|
BaseTool
|
||||||
|
// InvokableRun executes the tool; arguments are a JSON-encoded string, returns a string result
|
||||||
|
InvokableRun(ctx context.Context, argumentsInJSON string, opts ...Option) (string, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// StreamableTool is the streaming variant of InvokableTool
|
||||||
|
type StreamableTool interface {
|
||||||
|
BaseTool
|
||||||
|
// StreamableRun executes the tool in streaming mode, returns a StreamReader
|
||||||
|
StreamableRun(ctx context.Context, argumentsInJSON string, opts ...Option) (*schema.StreamReader[string], error)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Interface hierarchy:**
|
||||||
|
- `BaseTool`: Base interface, only provides metadata
|
||||||
|
- `InvokableTool`: Executable tool (extends BaseTool)
|
||||||
|
- `StreamableTool`: Streaming tool (extends BaseTool)
|
||||||
|
|
||||||
|
### Backend Interface
|
||||||
|
|
||||||
|
`Backend` is an abstract interface in Eino for filesystem operations:
|
||||||
|
|
||||||
|
```go
|
||||||
|
type Backend interface {
|
||||||
|
// List file information in a directory
|
||||||
|
LsInfo(ctx context.Context, req *LsInfoRequest) ([]FileInfo, error)
|
||||||
|
|
||||||
|
// Read file content, supports line offset and limits
|
||||||
|
Read(ctx context.Context, req *ReadRequest) (*FileContent, error)
|
||||||
|
|
||||||
|
// Search for matching content in files
|
||||||
|
GrepRaw(ctx context.Context, req *GrepRequest) ([]GrepMatch, error)
|
||||||
|
|
||||||
|
// Match files based on glob patterns
|
||||||
|
GlobInfo(ctx context.Context, req *GlobInfoRequest) ([]FileInfo, error)
|
||||||
|
|
||||||
|
// Write file content
|
||||||
|
Write(ctx context.Context, req *WriteRequest) error
|
||||||
|
|
||||||
|
// Edit file content (string replacement)
|
||||||
|
Edit(ctx context.Context, req *EditRequest) error
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### LocalBackend
|
||||||
|
|
||||||
|
`LocalBackend` is the local filesystem implementation of Backend, directly accessing the operating system's filesystem:
|
||||||
|
|
||||||
|
```go
|
||||||
|
import localbk "github.com/cloudwego/eino-ext/adk/backend/local"
|
||||||
|
|
||||||
|
backend, err := localbk.NewBackend(ctx, &localbk.Config{})
|
||||||
|
```
|
||||||
|
|
||||||
|
**Features:**
|
||||||
|
- Directly accesses the local filesystem, implemented using Go standard library
|
||||||
|
- Supports all Backend interface methods
|
||||||
|
- Supports executing shell commands (ExecuteStreaming)
|
||||||
|
- Path safety: Requires absolute paths to prevent directory traversal attacks
|
||||||
|
- Zero configuration: Works out of the box with no additional setup
|
||||||
|
|
||||||
|
## Implementation: Using DeepAgent
|
||||||
|
|
||||||
|
This chapter uses the DeepAgent prebuilt Agent, which provides first-class configuration for Backend and StreamingShell, making it easy to register filesystem-related tools.
|
||||||
|
|
||||||
|
### From ChatModelAgent to DeepAgent: When to Switch?
|
||||||
|
|
||||||
|
Previous chapters used `ChatModelAgent`, which can handle multi-turn conversations. But to access the filesystem, we need to switch to `DeepAgent`.
|
||||||
|
|
||||||
|
**ChatModelAgent vs DeepAgent comparison:**
|
||||||
|
|
||||||
|
| Capability | ChatModelAgent | DeepAgent |
|
||||||
|
|-----------|----------------|-----------|
|
||||||
|
| Multi-turn conversation | Yes | Yes |
|
||||||
|
| Add custom Tools | Yes, manually register each Tool | Yes, manual or automatic registration |
|
||||||
|
| Filesystem access (Backend) | No, must manually create and register all file tools | Yes, first-class config, auto-registered |
|
||||||
|
| Command execution (StreamingShell) | No, must manually create | Yes, first-class config, auto-registered |
|
||||||
|
| Built-in task management | No | Yes, `write_todos` tool |
|
||||||
|
| Sub-Agent support | No | Yes |
|
||||||
|
|
||||||
|
**Selection guide:**
|
||||||
|
- Pure conversation scenarios (no external access) -> Use `ChatModelAgent`
|
||||||
|
- Need filesystem access or command execution -> Use `DeepAgent`
|
||||||
|
|
||||||
|
### Why Use DeepAgent?
|
||||||
|
|
||||||
|
Compared to using ChatModelAgent directly, DeepAgent's advantages:
|
||||||
|
|
||||||
|
1. **First-class configuration**: Backend and StreamingShell are first-class config options — just pass them in
|
||||||
|
2. **Automatic tool registration**: Configuring a Backend automatically registers filesystem tools, no manual creation needed
|
||||||
|
3. **Built-in task management**: Provides the `write_todos` tool for task planning and tracking
|
||||||
|
4. **Sub-Agent support**: Can configure specialized sub-Agents for specific tasks
|
||||||
|
5. **More powerful**: Integrates filesystem, command execution, and many other capabilities
|
||||||
|
|
||||||
|
### Code Implementation
|
||||||
|
|
||||||
|
```go
|
||||||
|
import (
|
||||||
|
localbk "github.com/cloudwego/eino-ext/adk/backend/local"
|
||||||
|
"github.com/cloudwego/eino/adk/prebuilt/deep"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Create LocalBackend
|
||||||
|
backend, err := localbk.NewBackend(ctx, &localbk.Config{})
|
||||||
|
|
||||||
|
// Create DeepAgent, automatically registers filesystem tools
|
||||||
|
agent, err := deep.New(ctx, &deep.Config{
|
||||||
|
Name: "Ch04ToolAgent",
|
||||||
|
Description: "ChatWithDoc agent with filesystem access via LocalBackend.",
|
||||||
|
ChatModel: cm,
|
||||||
|
Instruction: instruction,
|
||||||
|
Backend: backend, // Provides filesystem operation capabilities
|
||||||
|
StreamingShell: backend, // Provides command execution capabilities
|
||||||
|
MaxIteration: 50,
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
### Tools Automatically Registered by DeepAgent
|
||||||
|
|
||||||
|
When `Backend` and `StreamingShell` are configured, DeepAgent automatically registers the following tools:
|
||||||
|
|
||||||
|
- `read_file`: Read file content
|
||||||
|
- `write_file`: Write file content
|
||||||
|
- `edit_file`: Edit file content
|
||||||
|
- `glob`: Find files based on glob patterns
|
||||||
|
- `grep`: Search for content in files
|
||||||
|
- `execute`: Execute shell commands
|
||||||
|
|
||||||
|
## Code Location
|
||||||
|
|
||||||
|
- Entry code: [cmd/ch04/main.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/cmd/ch04/main.go)
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
Same as Chapter 1: you need to configure an available ChatModel (OpenAI or Ark).
|
||||||
|
|
||||||
|
This chapter also requires setting `PROJECT_ROOT` (optional, see running instructions below).
|
||||||
|
|
||||||
|
## Running
|
||||||
|
|
||||||
|
In the `examples/quickstart/chatwitheino` directory, run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Optional: Set the root directory path for the Eino core library
|
||||||
|
# When not set, the Agent defaults to using the current working directory (the chatwitheino directory) as root
|
||||||
|
# To let the Agent search the complete Eino codebase, point this to the eino core library root
|
||||||
|
export PROJECT_ROOT=/path/to/eino
|
||||||
|
|
||||||
|
# Verify the path is correct (you should see directories like adk, components, compose, etc.)
|
||||||
|
ls $PROJECT_ROOT
|
||||||
|
|
||||||
|
go run ./cmd/ch04
|
||||||
|
```
|
||||||
|
|
||||||
|
**`PROJECT_ROOT` explanation:**
|
||||||
|
|
||||||
|
- **When not set**: `PROJECT_ROOT` defaults to the current working directory (the directory containing `chatwitheino`), and the Agent can only access files in this example project. This is sufficient for quick experimentation.
|
||||||
|
- **When set**: Points to the Eino core library root directory, and the Agent can search the complete Eino framework codebase (core library, extension library, examples library). This is the full ChatWithEino usage scenario.
|
||||||
|
|
||||||
|
**Recommended three-repository directory structure (for the full experience):**
|
||||||
|
|
||||||
|
```
|
||||||
|
eino/ # PROJECT_ROOT (Eino core library)
|
||||||
|
├── adk/
|
||||||
|
├── components/
|
||||||
|
├── compose/
|
||||||
|
├── ext/ # eino-ext (extension components, e.g., OpenAI, Ark implementations)
|
||||||
|
├── examples/ # eino-examples (this repository, where this example is located)
|
||||||
|
│ └── quickstart/
|
||||||
|
│ └── chatwitheino/
|
||||||
|
└── ...
|
||||||
|
```
|
||||||
|
|
||||||
|
You can use the `dev_setup.sh` script to automatically set up the above directory structure:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Run in the eino root directory to automatically clone extension and example repos to the correct locations
|
||||||
|
bash scripts/dev_setup.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Output example:
|
||||||
|
|
||||||
|
```text
|
||||||
|
you> List the files in the current directory
|
||||||
|
[assistant] Let me list the files in the current directory...
|
||||||
|
[tool call] glob(pattern: "*")
|
||||||
|
[tool result] Found 5 files:
|
||||||
|
- main.go
|
||||||
|
- go.mod
|
||||||
|
- go.sum
|
||||||
|
- README.md
|
||||||
|
- cmd/
|
||||||
|
|
||||||
|
you> Read the contents of main.go
|
||||||
|
[assistant] Let me read the main.go file...
|
||||||
|
[tool call] read_file(file_path: "main.go")
|
||||||
|
[tool result] File contents:
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
**Note:** If you encounter Tool errors during execution that cause the Agent to stop, don't panic — this is normal. Tool errors are common, such as parameter errors, file not found, etc. How to gracefully handle Tool errors will be covered in detail in the next chapter.
|
||||||
|
|
||||||
|
## Tool Call Flow
|
||||||
|
|
||||||
|
When the Agent needs to call a Tool:
|
||||||
|
|
||||||
|
```
|
||||||
|
+------------------------------------------+
|
||||||
|
| User: List the files in the current dir |
|
||||||
|
+------------------------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| Agent analyzes intent |
|
||||||
|
| Decides to call glob |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| Generate Tool Call |
|
||||||
|
| {"pattern": "*"} |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| Execute Tool |
|
||||||
|
| glob("*") |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| Return Tool Result |
|
||||||
|
| {"files": [...]} |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| Agent generates reply |
|
||||||
|
| "Found 5 files..." |
|
||||||
|
+------------------------+
|
||||||
|
```
|
||||||
|
|
||||||
|
## Chapter Summary
|
||||||
|
|
||||||
|
- **Tool**: Capability extensions for the Agent, enabling it to perform concrete operations
|
||||||
|
- **Backend**: Abstract interface for filesystem operations, providing unified file operation capabilities
|
||||||
|
- **LocalBackend**: Local filesystem implementation of Backend, directly accessing the OS filesystem
|
||||||
|
- **DeepAgent**: A prebuilt advanced Agent with first-class configuration for Backend and StreamingShell
|
||||||
|
- **Automatic tool registration**: Configuring a Backend automatically registers filesystem tools
|
||||||
|
- **Tool call flow**: Agent analyzes intent -> Generates Tool Call -> Executes Tool -> Returns result -> Generates reply
|
||||||
|
|
||||||
|
## Further Thinking
|
||||||
|
|
||||||
|
**Other Tool types:**
|
||||||
|
- HTTP Tool: Call external APIs
|
||||||
|
- Database Tool: Query databases
|
||||||
|
- Calculator Tool: Perform calculations
|
||||||
|
- Code Executor Tool: Run code
|
||||||
|
|
||||||
|
**Other Backend implementations:**
|
||||||
|
- Other storage backends can be implemented based on the Backend interface
|
||||||
|
- For example: cloud storage, database storage, etc.
|
||||||
|
- LocalBackend already provides complete filesystem operation capabilities
|
||||||
|
|
||||||
|
**Custom Tool creation:**
|
||||||
|
|
||||||
|
If you need to create custom Tools, you can use `utils.InferTool` to automatically infer from functions. See:
|
||||||
|
- [Tool interface documentation](https://github.com/cloudwego/eino/tree/main/components/tool)
|
||||||
|
- [Tool creation examples](https://github.com/cloudwego/eino-examples/tree/main/components/tool)
|
||||||
@ -0,0 +1,441 @@
|
|||||||
|
---
|
||||||
|
title: "Chapter 5: Middleware"
|
||||||
|
---
|
||||||
|
|
||||||
|
The goal of this chapter is to understand the Middleware pattern and implement Tool error handling and ChatModel retry mechanisms.
|
||||||
|
|
||||||
|
## Why We Need Middleware
|
||||||
|
|
||||||
|
In Chapter 4, we added Tool capabilities to the Agent, enabling it to access the filesystem. But in real-world scenarios, **Tool errors and ChatModel errors are common**, for example:
|
||||||
|
|
||||||
|
- **Tool errors**: File not found, parameter errors, insufficient permissions, etc.
|
||||||
|
- **ChatModel errors**: API rate limiting (429), network timeouts, service unavailable, etc.
|
||||||
|
|
||||||
|
### Problem 1: Tool Errors Interrupt the Entire Flow
|
||||||
|
|
||||||
|
When a Tool execution fails, the error propagates directly to the Agent, causing the entire conversation to be interrupted:
|
||||||
|
|
||||||
|
```text
|
||||||
|
[tool call] read_file(file_path: "nonexistent.txt")
|
||||||
|
Error: open nonexistent.txt: no such file or directory
|
||||||
|
// Conversation interrupted, user needs to start over
|
||||||
|
```
|
||||||
|
|
||||||
|
### Problem 2: Model Calls May Fail Due to Rate Limiting
|
||||||
|
|
||||||
|
When the model API returns a 429 (Too Many Requests) error, the entire conversation is also interrupted:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Error: rate limit exceeded (429)
|
||||||
|
// Conversation interrupted
|
||||||
|
```
|
||||||
|
|
||||||
|
### Expected Behavior
|
||||||
|
|
||||||
|
These errors **should not directly terminate the Agent flow**. Instead, the error information should be passed to the model so it can self-correct and proceed to the next round. For example:
|
||||||
|
|
||||||
|
```text
|
||||||
|
[tool call] read_file(file_path: "nonexistent.txt")
|
||||||
|
[tool result] [tool error] open nonexistent.txt: no such file or directory
|
||||||
|
[assistant] Sorry, the file doesn't exist. Let me list the files in the current directory first...
|
||||||
|
[tool call] glob(pattern: "*")
|
||||||
|
```
|
||||||
|
|
||||||
|
### The Role of Middleware
|
||||||
|
|
||||||
|
The **Middleware pattern** can extend the behavior of Tools and ChatModel, making it ideal for solving this problem:
|
||||||
|
|
||||||
|
- **Middleware is an interceptor for the Agent**: Inserts custom logic before and after calls
|
||||||
|
- **Middleware can handle errors**: Converts errors into a format the model can understand
|
||||||
|
- **Middleware can implement retries**: Automatically retries failed operations
|
||||||
|
- **Middleware is composable**: Multiple Middlewares can be chained together
|
||||||
|
|
||||||
|
**Simple analogy:**
|
||||||
|
- **Agent** = "business logic"
|
||||||
|
- **Middleware** = "AOP aspects" (logging, retry, error handling, and other cross-cutting concerns)
|
||||||
|
|
||||||
|
## Code Location
|
||||||
|
|
||||||
|
- Entry code: [cmd/ch05/main.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/cmd/ch05/main.go)
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
Same as Chapter 1: you need to configure an available ChatModel (OpenAI or Ark). Additionally, you need to set `PROJECT_ROOT` as in Chapter 4:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export PROJECT_ROOT=/path/to/eino # Eino core library root directory
|
||||||
|
```
|
||||||
|
|
||||||
|
## Running
|
||||||
|
|
||||||
|
In the `examples/quickstart/chatwitheino` directory, run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Set the project root directory
|
||||||
|
export PROJECT_ROOT=/path/to/your/project
|
||||||
|
|
||||||
|
go run ./cmd/ch05
|
||||||
|
```
|
||||||
|
|
||||||
|
Output example:
|
||||||
|
|
||||||
|
```text
|
||||||
|
you> List the files in the current directory
|
||||||
|
[assistant] Let me list the files for you...
|
||||||
|
[tool call] list_files(directory: ".")
|
||||||
|
|
||||||
|
you> Read a nonexistent file
|
||||||
|
[assistant] Trying to read the file...
|
||||||
|
[tool call] read_file(file_path: "nonexistent.txt")
|
||||||
|
[tool result] [tool error] open nonexistent.txt: no such file or directory
|
||||||
|
[assistant] Sorry, the file doesn't exist...
|
||||||
|
```
|
||||||
|
|
||||||
|
## Key Concepts
|
||||||
|
|
||||||
|
### Middleware Interface
|
||||||
|
|
||||||
|
`ChatModelAgentMiddleware` is the middleware interface for Agent:
|
||||||
|
|
||||||
|
```go
|
||||||
|
type ChatModelAgentMiddleware interface {
|
||||||
|
// BeforeAgent is called before each agent run, allowing modification of
|
||||||
|
// the agent's instruction and tools configuration.
|
||||||
|
BeforeAgent(ctx context.Context, runCtx *ChatModelAgentContext) (context.Context, *ChatModelAgentContext, error)
|
||||||
|
|
||||||
|
// BeforeModelRewriteState is called before each model invocation.
|
||||||
|
// The returned state is persisted to the agent's internal state and passed to the model.
|
||||||
|
BeforeModelRewriteState(ctx context.Context, state *ChatModelAgentState, mc *ModelContext) (context.Context, *ChatModelAgentState, error)
|
||||||
|
|
||||||
|
// AfterModelRewriteState is called after each model invocation.
|
||||||
|
// The input state includes the model's response as the last message.
|
||||||
|
AfterModelRewriteState(ctx context.Context, state *ChatModelAgentState, mc *ModelContext) (context.Context, *ChatModelAgentState, error)
|
||||||
|
|
||||||
|
// WrapInvokableToolCall wraps a tool's synchronous execution with custom behavior.
|
||||||
|
// This method is only called for tools that implement InvokableTool.
|
||||||
|
WrapInvokableToolCall(ctx context.Context, endpoint InvokableToolCallEndpoint, tCtx *ToolContext) (InvokableToolCallEndpoint, error)
|
||||||
|
|
||||||
|
// WrapStreamableToolCall wraps a tool's streaming execution with custom behavior.
|
||||||
|
// This method is only called for tools that implement StreamableTool.
|
||||||
|
WrapStreamableToolCall(ctx context.Context, endpoint StreamableToolCallEndpoint, tCtx *ToolContext) (StreamableToolCallEndpoint, error)
|
||||||
|
|
||||||
|
// WrapEnhancedInvokableToolCall wraps an enhanced tool's synchronous execution.
|
||||||
|
// This method is only called for tools that implement EnhancedInvokableTool.
|
||||||
|
WrapEnhancedInvokableToolCall(ctx context.Context, endpoint EnhancedInvokableToolCallEndpoint, tCtx *ToolContext) (EnhancedInvokableToolCallEndpoint, error)
|
||||||
|
|
||||||
|
// WrapEnhancedStreamableToolCall wraps an enhanced tool's streaming execution.
|
||||||
|
// This method is only called for tools that implement EnhancedStreamableTool.
|
||||||
|
WrapEnhancedStreamableToolCall(ctx context.Context, endpoint EnhancedStreamableToolCallEndpoint, tCtx *ToolContext) (EnhancedStreamableToolCallEndpoint, error)
|
||||||
|
|
||||||
|
// WrapModel wraps a chat model with custom behavior.
|
||||||
|
// This method is called at request time when the model is about to be invoked.
|
||||||
|
WrapModel(ctx context.Context, m model.BaseChatModel, mc *ModelContext) (model.BaseChatModel, error)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Design philosophy:**
|
||||||
|
- **Decorator pattern**: Each Middleware wraps the original call, and can modify input, output, or errors
|
||||||
|
- **Onion model**: Requests pass through Middleware from outside to inside, responses return from inside to outside
|
||||||
|
- **Composable**: Multiple Middlewares execute in sequence
|
||||||
|
|
||||||
|
### Middleware Execution Order
|
||||||
|
|
||||||
|
`Handlers` (i.e., Middlewares) are wrapped in **array order**, forming an onion model:
|
||||||
|
|
||||||
|
```go
|
||||||
|
Handlers: []adk.ChatModelAgentMiddleware{
|
||||||
|
&middlewareA{}, // Outermost: wrapped first, intercepts requests first, but WrapModel takes effect last
|
||||||
|
&middlewareB{}, // Middle layer
|
||||||
|
&middlewareC{}, // Innermost: wrapped last
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Execution order for Tool calls:**
|
||||||
|
|
||||||
|
```
|
||||||
|
Request -> A.Wrap -> B.Wrap -> C.Wrap -> Actual Tool Execution -> C returns -> B returns -> A returns -> Response
|
||||||
|
```
|
||||||
|
|
||||||
|
**Practical advice:** Place `safeToolMiddleware` (error capture) at the innermost layer (end of the array) to ensure that interrupt errors thrown by other Middlewares can propagate outward correctly.
|
||||||
|
|
||||||
|
### SafeToolMiddleware
|
||||||
|
|
||||||
|
`SafeToolMiddleware` converts Tool errors into strings so the model can understand and handle them:
|
||||||
|
|
||||||
|
```go
|
||||||
|
type safeToolMiddleware struct {
|
||||||
|
*adk.BaseChatModelAgentMiddleware
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *safeToolMiddleware) WrapInvokableToolCall(
|
||||||
|
_ context.Context,
|
||||||
|
endpoint adk.InvokableToolCallEndpoint,
|
||||||
|
_ *adk.ToolContext,
|
||||||
|
) (adk.InvokableToolCallEndpoint, error) {
|
||||||
|
return func(ctx context.Context, args string, opts ...tool.Option) (string, error) {
|
||||||
|
result, err := endpoint(ctx, args, opts...)
|
||||||
|
if err != nil {
|
||||||
|
// Convert error to string instead of returning an error
|
||||||
|
return fmt.Sprintf("[tool error] %v", err), nil
|
||||||
|
}
|
||||||
|
return result, nil
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Effect:**
|
||||||
|
|
||||||
|
```text
|
||||||
|
[tool call] read_file(file_path: "nonexistent.txt")
|
||||||
|
[tool result] [tool error] open nonexistent.txt: no such file or directory
|
||||||
|
[assistant] Sorry, the file doesn't exist, please check the file path...
|
||||||
|
// Conversation continues, the model can adjust its strategy based on the error information
|
||||||
|
```
|
||||||
|
|
||||||
|
### ModelRetryConfig
|
||||||
|
|
||||||
|
`ModelRetryConfig` configures automatic retries for ChatModel:
|
||||||
|
|
||||||
|
```go
|
||||||
|
type ModelRetryConfig struct {
|
||||||
|
MaxRetries int // Maximum number of retries
|
||||||
|
IsRetryAble func(ctx context.Context, err error) bool // Determines if an error is retryable
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Usage (using DeepAgent as an example):**
|
||||||
|
|
||||||
|
```go
|
||||||
|
agent, err := deep.New(ctx, &deep.Config{
|
||||||
|
// ...
|
||||||
|
ModelRetryConfig: &adk.ModelRetryConfig{
|
||||||
|
MaxRetries: 5,
|
||||||
|
IsRetryAble: func(_ context.Context, err error) bool {
|
||||||
|
// 429 rate limiting errors are retryable
|
||||||
|
return strings.Contains(err.Error(), "429") ||
|
||||||
|
strings.Contains(err.Error(), "Too Many Requests") ||
|
||||||
|
strings.Contains(err.Error(), "qpm limit")
|
||||||
|
},
|
||||||
|
},
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
**Retry strategy:**
|
||||||
|
- Exponential backoff: Retry intervals increase with each attempt
|
||||||
|
- Configurable conditions: Use `IsRetryAble` to determine which errors are retryable
|
||||||
|
- Automatic recovery: No user intervention needed
|
||||||
|
|
||||||
|
## Middleware Implementation
|
||||||
|
|
||||||
|
### 1. Implement SafeToolMiddleware
|
||||||
|
|
||||||
|
```go
|
||||||
|
type safeToolMiddleware struct {
|
||||||
|
*adk.BaseChatModelAgentMiddleware
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *safeToolMiddleware) WrapInvokableToolCall(
|
||||||
|
_ context.Context,
|
||||||
|
endpoint adk.InvokableToolCallEndpoint,
|
||||||
|
_ *adk.ToolContext,
|
||||||
|
) (adk.InvokableToolCallEndpoint, error) {
|
||||||
|
return func(ctx context.Context, args string, opts ...tool.Option) (string, error) {
|
||||||
|
result, err := endpoint(ctx, args, opts...)
|
||||||
|
if err != nil {
|
||||||
|
// Don't convert interrupt errors, they need to continue propagating
|
||||||
|
if _, ok := compose.IsInterruptRerunError(err); ok {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
// Convert other errors to strings
|
||||||
|
return fmt.Sprintf("[tool error] %v", err), nil
|
||||||
|
}
|
||||||
|
return result, nil
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Implement Streaming Tool Error Handling
|
||||||
|
|
||||||
|
```go
|
||||||
|
func (m *safeToolMiddleware) WrapStreamableToolCall(
|
||||||
|
_ context.Context,
|
||||||
|
endpoint adk.StreamableToolCallEndpoint,
|
||||||
|
_ *adk.ToolContext,
|
||||||
|
) (adk.StreamableToolCallEndpoint, error) {
|
||||||
|
return func(ctx context.Context, args string, opts ...tool.Option) (*schema.StreamReader[string], error) {
|
||||||
|
sr, err := endpoint(ctx, args, opts...)
|
||||||
|
if err != nil {
|
||||||
|
if _, ok := compose.IsInterruptRerunError(err); ok {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
// Return a single-frame stream containing the error message
|
||||||
|
return singleChunkReader(fmt.Sprintf("[tool error] %v", err)), nil
|
||||||
|
}
|
||||||
|
// Wrap the stream to catch errors within it
|
||||||
|
return safeWrapReader(sr), nil
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Configure the Agent to Use Middleware
|
||||||
|
|
||||||
|
This chapter continues using the `DeepAgent` introduced in Chapter 4, registering Middleware in its `Handlers` field:
|
||||||
|
|
||||||
|
```go
|
||||||
|
agent, err := deep.New(ctx, &deep.Config{
|
||||||
|
Name: "Ch05MiddlewareAgent",
|
||||||
|
Description: "ChatWithDoc agent with safe tool middleware and retry.",
|
||||||
|
ChatModel: cm,
|
||||||
|
Instruction: agentInstruction,
|
||||||
|
Backend: backend,
|
||||||
|
StreamingShell: backend,
|
||||||
|
MaxIteration: 50,
|
||||||
|
Handlers: []adk.ChatModelAgentMiddleware{
|
||||||
|
&safeToolMiddleware{}, // Converts Tool errors to strings
|
||||||
|
},
|
||||||
|
ModelRetryConfig: &adk.ModelRetryConfig{
|
||||||
|
MaxRetries: 5,
|
||||||
|
IsRetryAble: func(_ context.Context, err error) bool {
|
||||||
|
return strings.Contains(err.Error(), "429") ||
|
||||||
|
strings.Contains(err.Error(), "Too Many Requests")
|
||||||
|
},
|
||||||
|
},
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
**Note**: The `Handlers` field (in the config) and "Middleware" (the concept discussed in documentation) are the same thing — `Handlers` is the config field name, while `ChatModelAgentMiddleware` is the interface name.
|
||||||
|
|
||||||
|
**Key code snippet (Note: this is a simplified code snippet that cannot be run directly. For the complete code, please refer to** [cmd/ch05/main.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/cmd/ch05/main.go)**)**:
|
||||||
|
|
||||||
|
```go
|
||||||
|
// SafeToolMiddleware catches Tool errors and converts them to strings
|
||||||
|
type safeToolMiddleware struct {
|
||||||
|
*adk.BaseChatModelAgentMiddleware
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *safeToolMiddleware) WrapInvokableToolCall(
|
||||||
|
_ context.Context,
|
||||||
|
endpoint adk.InvokableToolCallEndpoint,
|
||||||
|
_ *adk.ToolContext,
|
||||||
|
) (adk.InvokableToolCallEndpoint, error) {
|
||||||
|
return func(ctx context.Context, args string, opts ...tool.Option) (string, error) {
|
||||||
|
result, err := endpoint(ctx, args, opts...)
|
||||||
|
if err != nil {
|
||||||
|
if _, ok := compose.IsInterruptRerunError(err); ok {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return fmt.Sprintf("[tool error] %v", err), nil
|
||||||
|
}
|
||||||
|
return result, nil
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Configure DeepAgent (same as Chapter 4, with Handlers and ModelRetryConfig added)
|
||||||
|
agent, _ := deep.New(ctx, &deep.Config{
|
||||||
|
ChatModel: cm,
|
||||||
|
Backend: backend,
|
||||||
|
StreamingShell: backend,
|
||||||
|
MaxIteration: 50,
|
||||||
|
Handlers: []adk.ChatModelAgentMiddleware{
|
||||||
|
&safeToolMiddleware{},
|
||||||
|
},
|
||||||
|
ModelRetryConfig: &adk.ModelRetryConfig{
|
||||||
|
MaxRetries: 5,
|
||||||
|
IsRetryAble: func(_ context.Context, err error) bool {
|
||||||
|
return strings.Contains(err.Error(), "429")
|
||||||
|
},
|
||||||
|
},
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
## Middleware Execution Flow
|
||||||
|
|
||||||
|
```
|
||||||
|
+------------------------------------------+
|
||||||
|
| User: Read a nonexistent file |
|
||||||
|
+------------------------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| Agent analyzes intent |
|
||||||
|
| Decides to call |
|
||||||
|
| read_file |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| SafeToolMiddleware |
|
||||||
|
| Intercepts Tool call |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| Execute read_file |
|
||||||
|
| Returns error |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| SafeToolMiddleware |
|
||||||
|
| Converts error to |
|
||||||
|
| string |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| Return Tool Result |
|
||||||
|
| "[tool error] ..." |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| Agent generates reply |
|
||||||
|
| "Sorry, the file |
|
||||||
|
| doesn't exist..." |
|
||||||
|
+------------------------+
|
||||||
|
```
|
||||||
|
|
||||||
|
## Chapter Summary
|
||||||
|
|
||||||
|
- **Middleware**: An interceptor for the Agent that inserts custom logic before and after calls
|
||||||
|
- **SafeToolMiddleware**: Converts Tool errors to strings so the model can understand and handle them
|
||||||
|
- **ModelRetryConfig**: Configures automatic retries for ChatModel to handle temporary errors like rate limiting
|
||||||
|
- **Decorator pattern**: Middleware wraps the original call, and can modify input, output, or errors
|
||||||
|
- **Onion model**: Requests pass through Middleware from outside to inside, responses return from inside to outside
|
||||||
|
|
||||||
|
## Further Thinking
|
||||||
|
|
||||||
|
**Eino Built-in Middlewares:**
|
||||||
|
|
||||||
|
| Middleware | Description |
|
||||||
|
|------------|-------------|
|
||||||
|
| **reduction** | Tool output reduction — when tool output is too long, automatically truncates and offloads to the filesystem to prevent context overflow |
|
||||||
|
| **summarization** | Automatic conversation history summarization — when token count exceeds a threshold, automatically generates summaries to compress history |
|
||||||
|
| **skill** | Skill loading middleware — enables the Agent to dynamically load and execute predefined skills |
|
||||||
|
|
||||||
|
**Middleware chain example:**
|
||||||
|
|
||||||
|
```go
|
||||||
|
import (
|
||||||
|
"github.com/cloudwego/eino/adk/middlewares/reduction"
|
||||||
|
"github.com/cloudwego/eino/adk/middlewares/summarization"
|
||||||
|
"github.com/cloudwego/eino/adk/middlewares/skill"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Create reduction middleware: manages tool output length
|
||||||
|
reductionMW, _ := reduction.New(ctx, &reduction.Config{
|
||||||
|
Backend: filesystemBackend, // Storage backend
|
||||||
|
MaxLengthForTrunc: 50000, // Max length for single tool output
|
||||||
|
MaxTokensForClear: 30000, // Token threshold to trigger cleanup
|
||||||
|
})
|
||||||
|
|
||||||
|
// Create summarization middleware: automatically compresses conversation history
|
||||||
|
summarizationMW, _ := summarization.New(ctx, &summarization.Config{
|
||||||
|
Model: chatModel, // Model used to generate summaries
|
||||||
|
Trigger: &summarization.TriggerCondition{
|
||||||
|
ContextTokens: 190000, // Token threshold to trigger summarization
|
||||||
|
},
|
||||||
|
})
|
||||||
|
|
||||||
|
// Combine multiple middlewares (conceptual example; when using DeepAgent, replace adk.NewChatModelAgent with deep.New)
|
||||||
|
agent, _ := adk.NewChatModelAgent(ctx, &adk.ChatModelAgentConfig{
|
||||||
|
Handlers: []adk.ChatModelAgentMiddleware{ // Note: config field name is Handlers, conceptually equivalent to Middlewares
|
||||||
|
summarizationMW, // Outermost: conversation history summarization
|
||||||
|
reductionMW, // Middle layer: tool output reduction
|
||||||
|
},
|
||||||
|
})
|
||||||
|
```
|
||||||
@ -0,0 +1,351 @@
|
|||||||
|
---
|
||||||
|
title: "Chapter 6: Callback and Trace (Observability)"
|
||||||
|
---
|
||||||
|
|
||||||
|
The goal of this chapter is to understand the Callback mechanism and integrate CozeLoop to implement tracing and observability.
|
||||||
|
|
||||||
|
## Code Location
|
||||||
|
|
||||||
|
- Entry code: [cmd/ch06/main.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/cmd/ch06/main.go)
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
Same as Chapter 1: you need to configure an available ChatModel (OpenAI or Ark). Additionally, you need to set `PROJECT_ROOT` as in Chapter 4:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export PROJECT_ROOT=/path/to/eino # Eino core library root directory (defaults to the current directory if not set)
|
||||||
|
```
|
||||||
|
|
||||||
|
Optional: Configure CozeLoop for tracing:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export COZELOOP_WORKSPACE_ID=your_workspace_id
|
||||||
|
export COZELOOP_API_TOKEN=your_token
|
||||||
|
```
|
||||||
|
|
||||||
|
## Running
|
||||||
|
|
||||||
|
In the `examples/quickstart/chatwitheino` directory, run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Set the project root directory
|
||||||
|
export PROJECT_ROOT=/path/to/your/project
|
||||||
|
|
||||||
|
# Optional: Configure CozeLoop
|
||||||
|
export COZELOOP_WORKSPACE_ID=your_workspace_id
|
||||||
|
export COZELOOP_API_TOKEN=your_token
|
||||||
|
|
||||||
|
go run ./cmd/ch06
|
||||||
|
```
|
||||||
|
|
||||||
|
Output example:
|
||||||
|
|
||||||
|
```text
|
||||||
|
[trace] starting session: 083d16da-6b13-4fe6-afb0-c45d8f490ce1
|
||||||
|
you> Hello
|
||||||
|
[trace] chat_model_generate: model=gpt-4.1-mini tokens=150
|
||||||
|
[trace] tool_call: name=list_files duration=23ms
|
||||||
|
[assistant] Hello! How can I help you?
|
||||||
|
```
|
||||||
|
|
||||||
|
## From Black Box to White Box: Why We Need Callbacks
|
||||||
|
|
||||||
|
In the previous chapters, the Agent we implemented was a "black box": you input a question, get an answer, but what happened in between was unclear.
|
||||||
|
|
||||||
|
**Problems with a black box:**
|
||||||
|
- Don't know how many times the model was called
|
||||||
|
- Don't know how long Tool execution took
|
||||||
|
- Don't know how many tokens were consumed
|
||||||
|
- Difficult to locate the cause when something goes wrong
|
||||||
|
|
||||||
|
**The role of Callbacks:**
|
||||||
|
- **Callbacks are Eino's sidecar mechanism**: Consistent from component to compose (discussed below) to adk
|
||||||
|
- **Callbacks trigger at fixed points**: 5 key moments in a component's lifecycle
|
||||||
|
- **Callbacks extract real-time information**: Input, output, errors, streaming data, etc.
|
||||||
|
- **Callbacks are versatile**: Observation, logging, metrics, tracing, debugging, auditing, etc.
|
||||||
|
|
||||||
|
**Simple analogy:**
|
||||||
|
- **Agent** = "business logic" (main path)
|
||||||
|
- **Callback** = "sidecar hooks" (extract information at fixed points)
|
||||||
|
|
||||||
|
## Key Concepts
|
||||||
|
|
||||||
|
### Handler Interface
|
||||||
|
|
||||||
|
`Handler` is the core interface in Eino that defines callback handlers:
|
||||||
|
|
||||||
|
```go
|
||||||
|
type Handler interface {
|
||||||
|
// Non-streaming input (before the component starts processing)
|
||||||
|
OnStart(ctx context.Context, info *RunInfo, input CallbackInput) context.Context
|
||||||
|
|
||||||
|
// Non-streaming output (after the component returns successfully)
|
||||||
|
OnEnd(ctx context.Context, info *RunInfo, output CallbackOutput) context.Context
|
||||||
|
|
||||||
|
// Error (when the component returns an error)
|
||||||
|
OnError(ctx context.Context, info *RunInfo, err error) context.Context
|
||||||
|
|
||||||
|
// Streaming input (when the component receives streaming input)
|
||||||
|
OnStartWithStreamInput(ctx context.Context, info *RunInfo,
|
||||||
|
input *schema.StreamReader[CallbackInput]) context.Context
|
||||||
|
|
||||||
|
// Streaming output (when the component returns streaming output)
|
||||||
|
OnEndWithStreamOutput(ctx context.Context, info *RunInfo,
|
||||||
|
output *schema.StreamReader[CallbackOutput]) context.Context
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Design philosophy:**
|
||||||
|
- **Sidecar mechanism**: Does not interfere with the main flow, extracts information at fixed points
|
||||||
|
- **Full coverage**: All components are supported, from component to compose to adk
|
||||||
|
- **State passing**: OnStart -> OnEnd of the same Handler can pass state via context
|
||||||
|
- **Performance optimization**: Implementing the `TimingChecker` interface allows skipping unnecessary timings
|
||||||
|
|
||||||
|
**RunInfo structure:**
|
||||||
|
```go
|
||||||
|
type RunInfo struct {
|
||||||
|
Name string // Business name (node name or user-specified)
|
||||||
|
Type string // Implementation type (e.g., "OpenAI")
|
||||||
|
Component string // Component type (e.g., "ChatModel")
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Important notes:**
|
||||||
|
- Streaming callbacks must close the StreamReader, otherwise goroutine leaks will occur
|
||||||
|
- Do not modify Input/Output — they are shared by all downstream consumers
|
||||||
|
- RunInfo may be nil — check before using
|
||||||
|
|
||||||
|
### CozeLoop
|
||||||
|
|
||||||
|
CozeLoop is an open-source AI application observability platform by ByteDance, providing:
|
||||||
|
|
||||||
|
- **Tracing**: Complete call chain visualization
|
||||||
|
- **Metrics monitoring**: Latency, token consumption, error rates, etc.
|
||||||
|
- **Log aggregation**: Centralized management of all logs
|
||||||
|
- **Debug support**: Online viewing and debugging
|
||||||
|
|
||||||
|
**Integration:**
|
||||||
|
|
||||||
|
```go
|
||||||
|
import (
|
||||||
|
clc "github.com/cloudwego/eino-ext/callbacks/cozeloop"
|
||||||
|
"github.com/cloudwego/eino/callbacks"
|
||||||
|
"github.com/coze-dev/cozeloop-go"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Create CozeLoop client
|
||||||
|
client, err := cozeloop.NewClient(
|
||||||
|
cozeloop.WithAPIToken(apiToken),
|
||||||
|
cozeloop.WithWorkspaceID(workspaceID),
|
||||||
|
)
|
||||||
|
|
||||||
|
// Register as a global Callback
|
||||||
|
callbacks.AppendGlobalHandlers(clc.NewLoopHandler(client))
|
||||||
|
```
|
||||||
|
|
||||||
|
### Callback Trigger Timings
|
||||||
|
|
||||||
|
Callbacks are triggered at 5 key moments in a component's lifecycle. In the table below, `Timing*` are Eino internal constant names (used with the `TimingChecker` interface), and the corresponding Handler interface methods are shown on the right:
|
||||||
|
|
||||||
|
| Timing Constant | Handler Method | Trigger Point | Input/Output |
|
||||||
|
|-----------------|----------------|---------------|--------------|
|
||||||
|
| `TimingOnStart` | `OnStart` | Before the component starts processing | CallbackInput |
|
||||||
|
| `TimingOnEnd` | `OnEnd` | After the component returns successfully | CallbackOutput |
|
||||||
|
| `TimingOnError` | `OnError` | When the component returns an error | error |
|
||||||
|
| `TimingOnStartWithStreamInput` | `OnStartWithStreamInput` | When the component receives streaming input | StreamReader[CallbackInput] |
|
||||||
|
| `TimingOnEndWithStreamOutput` | `OnEndWithStreamOutput` | When the component returns streaming output | StreamReader[CallbackOutput] |
|
||||||
|
|
||||||
|
**Example: ChatModel call flow**
|
||||||
|
|
||||||
|
```
|
||||||
|
+------------------------------------------+
|
||||||
|
| ChatModel.Generate(ctx, messages) |
|
||||||
|
+------------------------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| OnStart | <- Input: CallbackInput (messages)
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| Model processing |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| OnEnd | <- Output: CallbackOutput (response)
|
||||||
|
+------------------------+
|
||||||
|
```
|
||||||
|
|
||||||
|
**Example: Streaming output flow**
|
||||||
|
|
||||||
|
```
|
||||||
|
+------------------------------------------+
|
||||||
|
| ChatModel.Stream(ctx, messages) |
|
||||||
|
+------------------------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| OnStart | <- Input: CallbackInput (messages)
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| Model processing |
|
||||||
|
| (streaming) |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+---------------------------+
|
||||||
|
| OnEndWithStreamOutput | <- Output: StreamReader[CallbackOutput]
|
||||||
|
+---------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| Return chunks one |
|
||||||
|
| by one |
|
||||||
|
+------------------------+
|
||||||
|
```
|
||||||
|
|
||||||
|
**Notes:**
|
||||||
|
- Streaming errors (errors mid-stream) do not trigger OnError — they are returned within the StreamReader
|
||||||
|
- OnStart -> OnEnd of the same Handler can pass state via context
|
||||||
|
- There is no guaranteed execution order between different Handlers
|
||||||
|
|
||||||
|
## Callback Implementation
|
||||||
|
|
||||||
|
### 1. Implement a Custom Callback Handler
|
||||||
|
|
||||||
|
Fully implementing the `Handler` interface requires implementing all 5 methods, which can be verbose. Eino provides the `callbacks.HandlerHelper` utility class to simplify the implementation:
|
||||||
|
|
||||||
|
```go
|
||||||
|
import "github.com/cloudwego/eino/callbacks"
|
||||||
|
|
||||||
|
// Use NewHandlerHelper to register callbacks you're interested in
|
||||||
|
handler := callbacks.NewHandlerHelper().
|
||||||
|
OnStart(func(ctx context.Context, info *callbacks.RunInfo, input callbacks.CallbackInput) context.Context {
|
||||||
|
log.Printf("[trace] %s/%s start", info.Component, info.Name)
|
||||||
|
return ctx
|
||||||
|
}).
|
||||||
|
OnEnd(func(ctx context.Context, info *callbacks.RunInfo, output callbacks.CallbackOutput) context.Context {
|
||||||
|
log.Printf("[trace] %s/%s end", info.Component, info.Name)
|
||||||
|
return ctx
|
||||||
|
}).
|
||||||
|
OnError(func(ctx context.Context, info *callbacks.RunInfo, err error) context.Context {
|
||||||
|
log.Printf("[trace] %s/%s error: %v", info.Component, info.Name, err)
|
||||||
|
return ctx
|
||||||
|
}).
|
||||||
|
Handler()
|
||||||
|
|
||||||
|
// Register as a global Callback
|
||||||
|
callbacks.AppendGlobalHandlers(handler)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Note**: `RunInfo` may be `nil` (e.g., top-level calls without RunInfo) — check before using.
|
||||||
|
|
||||||
|
### 2. Integrate CozeLoop
|
||||||
|
|
||||||
|
```go
|
||||||
|
func setupCozeLoop(ctx context.Context) (*cozeloop.Client, error) {
|
||||||
|
apiToken := os.Getenv("COZELOOP_API_TOKEN")
|
||||||
|
workspaceID := os.Getenv("COZELOOP_WORKSPACE_ID")
|
||||||
|
|
||||||
|
if apiToken == "" || workspaceID == "" {
|
||||||
|
return nil, nil // Skip if not configured
|
||||||
|
}
|
||||||
|
|
||||||
|
client, err := cozeloop.NewClient(
|
||||||
|
cozeloop.WithAPIToken(apiToken),
|
||||||
|
cozeloop.WithWorkspaceID(workspaceID),
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// Register as a global Callback
|
||||||
|
callbacks.AppendGlobalHandlers(clc.NewLoopHandler(client))
|
||||||
|
|
||||||
|
return client, nil
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Use in main
|
||||||
|
|
||||||
|
```go
|
||||||
|
func main() {
|
||||||
|
ctx := context.Background()
|
||||||
|
|
||||||
|
// Set up CozeLoop (optional)
|
||||||
|
client, err := setupCozeLoop(ctx)
|
||||||
|
if err != nil {
|
||||||
|
log.Printf("cozeloop setup failed: %v", err)
|
||||||
|
}
|
||||||
|
if client != nil {
|
||||||
|
defer func() {
|
||||||
|
time.Sleep(5 * time.Second) // Wait for data to be reported
|
||||||
|
client.Close(ctx)
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Create Agent and run...
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Key code snippet (Note: this is a simplified code snippet that cannot be run directly. For the complete code, please refer to** [cmd/ch06/main.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/cmd/ch06/main.go)**)**:
|
||||||
|
|
||||||
|
```go
|
||||||
|
// Set up CozeLoop tracing
|
||||||
|
cozeloopApiToken := os.Getenv("COZELOOP_API_TOKEN")
|
||||||
|
cozeloopWorkspaceID := os.Getenv("COZELOOP_WORKSPACE_ID")
|
||||||
|
if cozeloopApiToken != "" && cozeloopWorkspaceID != "" {
|
||||||
|
client, err := cozeloop.NewClient(
|
||||||
|
cozeloop.WithAPIToken(cozeloopApiToken),
|
||||||
|
cozeloop.WithWorkspaceID(cozeloopWorkspaceID),
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
log.Fatalf("cozeloop.NewClient failed: %v", err)
|
||||||
|
}
|
||||||
|
defer func() {
|
||||||
|
time.Sleep(5 * time.Second)
|
||||||
|
client.Close(ctx)
|
||||||
|
}()
|
||||||
|
callbacks.AppendGlobalHandlers(clc.NewLoopHandler(client))
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## The Value of Observability
|
||||||
|
|
||||||
|
### 1. Performance Analysis
|
||||||
|
|
||||||
|
With data collected through Callbacks, you can analyze:
|
||||||
|
- Model call latency distribution
|
||||||
|
- Tool execution time rankings
|
||||||
|
- Token consumption trends
|
||||||
|
|
||||||
|
### 2. Error Tracking
|
||||||
|
|
||||||
|
When the Agent encounters problems:
|
||||||
|
- View the complete call chain
|
||||||
|
- Locate which step caused the error
|
||||||
|
- Analyze the root cause
|
||||||
|
|
||||||
|
### 3. Cost Optimization
|
||||||
|
|
||||||
|
With token consumption data:
|
||||||
|
- Identify high-consumption conversations
|
||||||
|
- Optimize prompts to reduce tokens
|
||||||
|
- Choose more cost-effective models
|
||||||
|
|
||||||
|
## Chapter Summary
|
||||||
|
|
||||||
|
- **Callback**: Eino's observation hooks that trigger callbacks at key points
|
||||||
|
- **CozeLoop**: ByteDance's AI application observability platform
|
||||||
|
- **Global registration**: Register global Callbacks via `callbacks.AppendGlobalHandlers`
|
||||||
|
- **Non-intrusive**: Business code doesn't need modification — Callbacks trigger automatically
|
||||||
|
- **Observability value**: Performance analysis, error tracking, cost optimization
|
||||||
|
|
||||||
|
## Further Thinking
|
||||||
|
|
||||||
|
**Other Callback implementations:**
|
||||||
|
- OpenTelemetry Callback: Connect to standard observability protocols
|
||||||
|
- Custom logging Callback: Write to local files
|
||||||
|
- Metrics Callback: Connect to monitoring systems like Prometheus
|
||||||
|
|
||||||
|
**Advanced usage:**
|
||||||
|
- Implement sampling in Callbacks (only record some requests)
|
||||||
|
- Implement rate limiting in Callbacks (based on token consumption)
|
||||||
|
- Implement alerting in Callbacks (notify when error rate is too high)
|
||||||
@ -0,0 +1,346 @@
|
|||||||
|
---
|
||||||
|
title: "Chapter 7: Interrupt/Resume"
|
||||||
|
---
|
||||||
|
|
||||||
|
The goal of this chapter is to understand the Interrupt/Resume mechanism and implement a Tool approval flow that lets users confirm before sensitive operations.
|
||||||
|
|
||||||
|
## Code Location
|
||||||
|
|
||||||
|
- Entry code: [cmd/ch07/main.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/cmd/ch07/main.go)
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
Same as Chapter 1: you need to configure an available ChatModel (OpenAI or Ark). Additionally, you need to set `PROJECT_ROOT` as in Chapter 4:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export PROJECT_ROOT=/path/to/eino # Eino core library root directory (defaults to the current directory if not set)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Running
|
||||||
|
|
||||||
|
In the `examples/quickstart/chatwitheino` directory, run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Set the project root directory
|
||||||
|
export PROJECT_ROOT=/path/to/your/project
|
||||||
|
|
||||||
|
go run ./cmd/ch07
|
||||||
|
```
|
||||||
|
|
||||||
|
Output example:
|
||||||
|
|
||||||
|
```text
|
||||||
|
you> Please execute the command echo hello
|
||||||
|
|
||||||
|
Warning: Approval Required
|
||||||
|
Tool: execute
|
||||||
|
Arguments: {"command":"echo hello"}
|
||||||
|
|
||||||
|
Approve this action? (y/n): y
|
||||||
|
[tool result] hello
|
||||||
|
|
||||||
|
hello
|
||||||
|
```
|
||||||
|
|
||||||
|
## From Auto-Execution to Human Approval: Why We Need Interrupt
|
||||||
|
|
||||||
|
In the previous chapters, the Agent automatically executed all Tool calls, but in certain scenarios this is dangerous:
|
||||||
|
|
||||||
|
**Risks of auto-execution:**
|
||||||
|
- Deleting files: Accidentally deleting important data
|
||||||
|
- Sending emails: Sending incorrect content
|
||||||
|
- Executing commands: Running dangerous operations
|
||||||
|
- Modifying configuration: Breaking system settings
|
||||||
|
|
||||||
|
**The role of Interrupt:**
|
||||||
|
- **Interrupt is the Agent's pause mechanism**: Pauses before critical operations, waiting for user confirmation
|
||||||
|
- **Interrupt can carry information**: Shows the user the operation about to be executed
|
||||||
|
- **Interrupt is resumable**: Continues execution after user confirmation, returns an error if rejected
|
||||||
|
|
||||||
|
**Simple analogy:**
|
||||||
|
- **Auto-execution** = "autopilot" (fully trusting the system)
|
||||||
|
- **Interrupt** = "manual override" (critical decisions are made by humans)
|
||||||
|
|
||||||
|
## Key Concepts
|
||||||
|
|
||||||
|
### Interrupt Mechanism
|
||||||
|
|
||||||
|
`Interrupt` is the core mechanism in Eino for implementing human-agent collaboration.
|
||||||
|
|
||||||
|
**Core idea: Pause before executing critical operations, and continue after user confirmation.**
|
||||||
|
|
||||||
|
A Tool execution that requires approval is split into **two phases**:
|
||||||
|
|
||||||
|
1. **First call (trigger interrupt)**: The Tool saves the current arguments, then returns an interrupt signal. The Runner pauses execution and returns an Interrupt event to the caller.
|
||||||
|
2. **Resume after user approval**: The Runner calls the Tool again. This time the Tool detects it was "previously interrupted", reads the user's approval result, and executes (or rejects).
|
||||||
|
|
||||||
|
**Simplified pseudocode:**
|
||||||
|
|
||||||
|
```
|
||||||
|
func myTool(ctx, args):
|
||||||
|
if first call:
|
||||||
|
save args
|
||||||
|
return interrupt signal // Runner pauses, shows approval prompt
|
||||||
|
else: // Second call after Resume
|
||||||
|
if user approved:
|
||||||
|
return execute(saved args)
|
||||||
|
else:
|
||||||
|
return "Operation rejected by user"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Full code with key field explanations:**
|
||||||
|
|
||||||
|
```go
|
||||||
|
// Trigger an interrupt in a Tool
|
||||||
|
func myTool(ctx context.Context, args string) (string, error) {
|
||||||
|
// wasInterrupted: whether this is the second call after Resume (false on first call, true after Resume)
|
||||||
|
// storedArgs: arguments saved via StatefulInterrupt during the first call, retrievable after Resume
|
||||||
|
wasInterrupted, _, storedArgs := tool.GetInterruptState[string](ctx)
|
||||||
|
|
||||||
|
if !wasInterrupted {
|
||||||
|
// First call: trigger interrupt, saving args for use after Resume
|
||||||
|
return "", tool.StatefulInterrupt(ctx, &ApprovalInfo{
|
||||||
|
ToolName: "my_tool",
|
||||||
|
ArgumentsInJSON: args,
|
||||||
|
}, args) // Third argument is the state to save (retrievable via storedArgs after Resume)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Second call after Resume: read user's approval result
|
||||||
|
// isTarget: whether this Resume targets the current Tool (each Resume targets only one Tool)
|
||||||
|
// hasData: whether the Resume carries approval result data
|
||||||
|
// data: the user's approval result
|
||||||
|
isTarget, hasData, data := tool.GetResumeContext[*ApprovalResult](ctx)
|
||||||
|
if isTarget && hasData {
|
||||||
|
if data.Approved {
|
||||||
|
return doSomething(storedArgs) // Execute actual operation using saved arguments
|
||||||
|
}
|
||||||
|
return "Operation rejected by user", nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Other cases (isTarget=false means this Resume targets a different Tool): re-interrupt
|
||||||
|
return "", tool.StatefulInterrupt(ctx, &ApprovalInfo{
|
||||||
|
ToolName: "my_tool",
|
||||||
|
ArgumentsInJSON: storedArgs,
|
||||||
|
}, storedArgs)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### ApprovalMiddleware
|
||||||
|
|
||||||
|
`ApprovalMiddleware` is a general-purpose approval middleware that can intercept specific Tool calls:
|
||||||
|
|
||||||
|
```go
|
||||||
|
type approvalMiddleware struct {
|
||||||
|
*adk.BaseChatModelAgentMiddleware
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *approvalMiddleware) WrapInvokableToolCall(
|
||||||
|
_ context.Context,
|
||||||
|
endpoint adk.InvokableToolCallEndpoint,
|
||||||
|
tCtx *adk.ToolContext,
|
||||||
|
) (adk.InvokableToolCallEndpoint, error) {
|
||||||
|
// Only intercept Tools that require approval
|
||||||
|
if tCtx.Name != "execute" {
|
||||||
|
return endpoint, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
return func(ctx context.Context, args string, opts ...tool.Option) (string, error) {
|
||||||
|
wasInterrupted, _, storedArgs := tool.GetInterruptState[string](ctx)
|
||||||
|
|
||||||
|
if !wasInterrupted {
|
||||||
|
return "", tool.StatefulInterrupt(ctx, &commontool.ApprovalInfo{
|
||||||
|
ToolName: tCtx.Name,
|
||||||
|
ArgumentsInJSON: args,
|
||||||
|
}, args)
|
||||||
|
}
|
||||||
|
|
||||||
|
isTarget, hasData, data := tool.GetResumeContext[*commontool.ApprovalResult](ctx)
|
||||||
|
if isTarget && hasData {
|
||||||
|
if data.Approved {
|
||||||
|
return endpoint(ctx, storedArgs, opts...)
|
||||||
|
}
|
||||||
|
if data.DisapproveReason != nil {
|
||||||
|
return fmt.Sprintf("tool '%s' disapproved: %s", tCtx.Name, *data.DisapproveReason), nil
|
||||||
|
}
|
||||||
|
return fmt.Sprintf("tool '%s' disapproved", tCtx.Name), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Re-interrupt
|
||||||
|
return "", tool.StatefulInterrupt(ctx, &commontool.ApprovalInfo{
|
||||||
|
ToolName: tCtx.Name,
|
||||||
|
ArgumentsInJSON: storedArgs,
|
||||||
|
}, storedArgs)
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *approvalMiddleware) WrapStreamableToolCall(
|
||||||
|
_ context.Context,
|
||||||
|
endpoint adk.StreamableToolCallEndpoint,
|
||||||
|
tCtx *adk.ToolContext,
|
||||||
|
) (adk.StreamableToolCallEndpoint, error) {
|
||||||
|
// If the agent is configured with StreamingShell, execute uses streaming calls,
|
||||||
|
// so this method must be implemented to intercept it
|
||||||
|
if tCtx.Name != "execute" {
|
||||||
|
return endpoint, nil
|
||||||
|
}
|
||||||
|
return func(ctx context.Context, args string, opts ...tool.Option) (*schema.StreamReader[string], error) {
|
||||||
|
wasInterrupted, _, storedArgs := tool.GetInterruptState[string](ctx)
|
||||||
|
if !wasInterrupted {
|
||||||
|
return nil, tool.StatefulInterrupt(ctx, &commontool.ApprovalInfo{
|
||||||
|
ToolName: tCtx.Name,
|
||||||
|
ArgumentsInJSON: args,
|
||||||
|
}, args)
|
||||||
|
}
|
||||||
|
|
||||||
|
isTarget, hasData, data := tool.GetResumeContext[*commontool.ApprovalResult](ctx)
|
||||||
|
if isTarget && hasData {
|
||||||
|
if data.Approved {
|
||||||
|
return endpoint(ctx, storedArgs, opts...)
|
||||||
|
}
|
||||||
|
if data.DisapproveReason != nil {
|
||||||
|
return singleChunkReader(fmt.Sprintf("tool '%s' disapproved: %s", tCtx.Name, *data.DisapproveReason)), nil
|
||||||
|
}
|
||||||
|
return singleChunkReader(fmt.Sprintf("tool '%s' disapproved", tCtx.Name)), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
isTarget, _, _ = tool.GetResumeContext[any](ctx)
|
||||||
|
if !isTarget {
|
||||||
|
return nil, tool.StatefulInterrupt(ctx, &commontool.ApprovalInfo{
|
||||||
|
ToolName: tCtx.Name,
|
||||||
|
ArgumentsInJSON: storedArgs,
|
||||||
|
}, storedArgs)
|
||||||
|
}
|
||||||
|
|
||||||
|
return endpoint(ctx, storedArgs, opts...)
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### CheckPointStore
|
||||||
|
|
||||||
|
`CheckPointStore` is the key component for implementing interrupt recovery:
|
||||||
|
|
||||||
|
```go
|
||||||
|
type CheckPointStore interface {
|
||||||
|
// Save a checkpoint
|
||||||
|
Put(ctx context.Context, key string, checkpoint *Checkpoint) error
|
||||||
|
|
||||||
|
// Get a checkpoint
|
||||||
|
Get(ctx context.Context, key string) (*Checkpoint, error)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Why do we need CheckPointStore?**
|
||||||
|
- Save state on interrupt: Tool arguments, execution position, etc.
|
||||||
|
- Load state on resume: Continue execution from the interrupt point
|
||||||
|
- Support cross-process recovery: Can resume even after process restart
|
||||||
|
|
||||||
|
## Interrupt/Resume Implementation
|
||||||
|
|
||||||
|
### 1. Configure Runner with CheckPointStore
|
||||||
|
|
||||||
|
```go
|
||||||
|
runner := adk.NewRunner(ctx, adk.RunnerConfig{
|
||||||
|
Agent: agent,
|
||||||
|
EnableStreaming: true,
|
||||||
|
CheckPointStore: adkstore.NewInMemoryStore(), // In-memory storage
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Configure Agent with ApprovalMiddleware
|
||||||
|
|
||||||
|
```go
|
||||||
|
agent, err := deep.New(ctx, &deep.Config{
|
||||||
|
// ... other configuration
|
||||||
|
Handlers: []adk.ChatModelAgentMiddleware{
|
||||||
|
&approvalMiddleware{}, // Add approval middleware
|
||||||
|
&safeToolMiddleware{}, // Convert Tool errors to strings (interrupt errors continue to propagate upward)
|
||||||
|
},
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Handle Interrupt Events
|
||||||
|
|
||||||
|
```go
|
||||||
|
checkPointID := sessionID
|
||||||
|
|
||||||
|
events := runner.Run(ctx, history, adk.WithCheckPointID(checkPointID))
|
||||||
|
content, interruptInfo, err := printAndCollectAssistantFromEvents(events)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
if interruptInfo != nil {
|
||||||
|
// Note: it's recommended to use the same stdin reader for both "user input" and "approval y/n"
|
||||||
|
// to avoid approval input being treated as the next round's you> message
|
||||||
|
content, err = handleInterrupt(ctx, runner, checkPointID, interruptInfo, reader)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
_ = session.Append(schema.AssistantMessage(content, nil))
|
||||||
|
```
|
||||||
|
|
||||||
|
## Interrupt/Resume Execution Flow
|
||||||
|
|
||||||
|
```
|
||||||
|
+------------------------------------------+
|
||||||
|
| User: Execute command echo hello |
|
||||||
|
+------------------------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| Agent analyzes intent |
|
||||||
|
| Decides to call |
|
||||||
|
| execute |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| ApprovalMiddleware |
|
||||||
|
| Intercepts Tool call |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| Trigger Interrupt |
|
||||||
|
| Save state to Store |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| Return Interrupt event |
|
||||||
|
| Wait for user approval |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| User inputs y/n |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| runner.ResumeWith... |
|
||||||
|
| Resume execution |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| Execute command |
|
||||||
|
| or return rejection |
|
||||||
|
+------------------------+
|
||||||
|
```
|
||||||
|
|
||||||
|
## Chapter Summary
|
||||||
|
|
||||||
|
- **Interrupt**: The Agent's pause mechanism — pauses before critical operations to wait for confirmation
|
||||||
|
- **Resume**: Resumes execution — continues after user confirmation, or returns an error if rejected
|
||||||
|
- **ApprovalMiddleware**: A general-purpose approval middleware that intercepts specific Tool calls
|
||||||
|
- **CheckPointStore**: Saves interrupt state, supporting cross-process recovery
|
||||||
|
- **Human-agent collaboration**: Critical decisions are confirmed by humans, improving safety
|
||||||
|
|
||||||
|
## Further Thinking
|
||||||
|
|
||||||
|
**Other Interrupt scenarios:**
|
||||||
|
- Multi-option approval: User selects one of multiple options
|
||||||
|
- Parameter completion: User provides missing parameters
|
||||||
|
- Conditional branching: User decides the execution path
|
||||||
|
|
||||||
|
**Approval strategies:**
|
||||||
|
- Allowlist: Only require approval for sensitive operations
|
||||||
|
- Blocklist: Require approval for all operations, except safe ones
|
||||||
|
- Dynamic rules: Decide whether to require approval based on argument content
|
||||||
@ -0,0 +1,303 @@
|
|||||||
|
---
|
||||||
|
title: "Chapter 8: Graph Tool (Complex Workflows)"
|
||||||
|
---
|
||||||
|
|
||||||
|
The goal of this chapter is to understand the concept of Graph Tools, implement parallel chunk recall for large files, and introduce the compose package for building complex workflows.
|
||||||
|
|
||||||
|
## Code Location
|
||||||
|
|
||||||
|
- Entry code: [cmd/ch08/main.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/cmd/ch08/main.go)
|
||||||
|
- RAG implementation: [rag/rag.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/rag/rag.go)
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
Same as Chapter 1: you need to configure an available ChatModel (OpenAI or Ark).
|
||||||
|
|
||||||
|
## Running
|
||||||
|
|
||||||
|
In the `examples/quickstart/chatwitheino` directory, run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Set the project root directory
|
||||||
|
export PROJECT_ROOT=/path/to/your/project
|
||||||
|
|
||||||
|
go run ./cmd/ch08
|
||||||
|
```
|
||||||
|
|
||||||
|
Output example:
|
||||||
|
|
||||||
|
```text
|
||||||
|
you> Please analyze the WebSocket handshake section in the RFC6455 document
|
||||||
|
[assistant] Let me analyze the document for you...
|
||||||
|
[tool call] answer_from_document(file_path: "rfc6455.txt", question: "WebSocket handshake process")
|
||||||
|
[tool result] Found 3 relevant fragments, generating answer...
|
||||||
|
[assistant] According to the RFC6455 document, the WebSocket handshake process is as follows...
|
||||||
|
```
|
||||||
|
|
||||||
|
## From Simple Tools to Graph Tools: Why We Need Complex Workflows
|
||||||
|
|
||||||
|
In Chapter 4, we created simple Tools where each Tool performs a single task. But in real-world scenarios, many tasks require multiple steps working together.
|
||||||
|
|
||||||
|
**Limitations of simple Tools:**
|
||||||
|
- Single responsibility: Each Tool does only one thing
|
||||||
|
- No parallelism: Multiple independent tasks cannot execute simultaneously
|
||||||
|
- Hard to reuse: Complex logic is difficult to split and compose
|
||||||
|
|
||||||
|
**Important note: This chapter only showcases a small part of compose/graph/workflow capabilities.**
|
||||||
|
|
||||||
|
From a broader perspective, Eino's `compose` package provides very general-purpose, deterministic orchestration capabilities: you can organize any system that requires "deterministic business flows" into an executable pipeline using `compose`'s Graph/Chain/Workflow. It can **natively orchestrate all Eino components** (such as ChatModel, Prompt, Tools, Retriever, Embedding, Indexer, etc.), with a complete **callback** system and **interrupt/resume + checkpoint** support.
|
||||||
|
|
||||||
|
**The role of Graph Tools:**
|
||||||
|
- **Graph Tool is a Tool-wrapped compose workflow**: Wraps compilable orchestration artifacts like `compose.Graph / compose.Chain / compose.Workflow` into a Tool that an Agent can call
|
||||||
|
- **Supports parallelism/branching/composition**: Provided by compose (parallelism, branching, field mapping, subgraphs, etc.); Graph Tool simply exposes them as a Tool entry point
|
||||||
|
- **Supports state management and persistence**: Passes data between nodes, and saves/restores run state via checkpoints
|
||||||
|
- **Supports interrupt/resume**: Both workflow-internal interrupts (triggering interrupt within a node) and tool-level interrupt wrapping (nested interrupt scenarios)
|
||||||
|
|
||||||
|
**Simple analogy:**
|
||||||
|
- **Simple Tool** = "single-step operation" (read a file)
|
||||||
|
- **Graph Tool** = "pipeline" (read -> chunk -> score -> filter -> generate answer)
|
||||||
|
|
||||||
|
## Key Concepts
|
||||||
|
|
||||||
|
### compose.Workflow
|
||||||
|
|
||||||
|
`compose.Workflow` is the core component for building workflows in Eino:
|
||||||
|
|
||||||
|
```go
|
||||||
|
wf := compose.NewWorkflow[Input, Output]()
|
||||||
|
|
||||||
|
// Add nodes
|
||||||
|
wf.AddLambdaNode("load", loadFunc).AddInput(compose.START)
|
||||||
|
wf.AddLambdaNode("chunk", chunkFunc).AddInput("load")
|
||||||
|
wf.AddLambdaNode("score", scoreFunc).AddInput("chunk")
|
||||||
|
wf.AddLambdaNode("answer", answerFunc).AddInput("score")
|
||||||
|
|
||||||
|
// Connect to end node
|
||||||
|
wf.End().AddInput("answer")
|
||||||
|
```
|
||||||
|
|
||||||
|
**Core concepts:**
|
||||||
|
- **Node**: A processing unit in the workflow
|
||||||
|
- **Edge**: The data flow direction between nodes
|
||||||
|
- **START**: The workflow entry point
|
||||||
|
- **END**: The workflow exit point
|
||||||
|
|
||||||
|
### BatchNode
|
||||||
|
|
||||||
|
`BatchNode` is used for parallel processing of multiple tasks:
|
||||||
|
|
||||||
|
```go
|
||||||
|
scorer := batch.NewBatchNode(&batch.NodeConfig[Task, Result]{
|
||||||
|
Name: "ChunkScorer",
|
||||||
|
InnerTask: scoreOneChunk, // Processing function for a single task
|
||||||
|
MaxConcurrency: 5, // Maximum concurrency
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
**How it works:**
|
||||||
|
1. Receives a task list as input
|
||||||
|
2. Executes each task in parallel (limited by MaxConcurrency)
|
||||||
|
3. Collects and returns all results
|
||||||
|
|
||||||
|
### FieldMapping
|
||||||
|
|
||||||
|
`FieldMapping` is used to pass data across nodes:
|
||||||
|
|
||||||
|
```go
|
||||||
|
wf.AddLambdaNode("answer", answerFunc).
|
||||||
|
AddInputWithOptions("filter", // Get data from the filter node
|
||||||
|
[]*compose.FieldMapping{compose.ToField("TopK")},
|
||||||
|
compose.WithNoDirectDependency()).
|
||||||
|
AddInputWithOptions(compose.START, // Get data from the START node
|
||||||
|
[]*compose.FieldMapping{compose.MapFields("Question", "Question")},
|
||||||
|
compose.WithNoDirectDependency())
|
||||||
|
```
|
||||||
|
|
||||||
|
**Why do we need FieldMapping?**
|
||||||
|
- Pass data between non-adjacent nodes
|
||||||
|
- Merge multiple data sources into a single node
|
||||||
|
- Rename data fields
|
||||||
|
|
||||||
|
## Graph Tool Implementation
|
||||||
|
|
||||||
|
### 1. Define Input/Output Structures
|
||||||
|
|
||||||
|
```go
|
||||||
|
type Input struct {
|
||||||
|
FilePath string `json:"file_path" jsonschema:"description=Absolute path to the document"`
|
||||||
|
Question string `json:"question" jsonschema:"description=The question to answer"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type Output struct {
|
||||||
|
Answer string `json:"answer"`
|
||||||
|
Sources []string `json:"sources"`
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Build the Workflow
|
||||||
|
|
||||||
|
```go
|
||||||
|
func buildWorkflow(cm model.BaseChatModel) *compose.Workflow[Input, Output] {
|
||||||
|
wf := compose.NewWorkflow[Input, Output]()
|
||||||
|
|
||||||
|
// load: Read the file
|
||||||
|
wf.AddLambdaNode("load", compose.InvokableLambda(
|
||||||
|
func(ctx context.Context, in Input) ([]*schema.Document, error) {
|
||||||
|
data, err := os.ReadFile(in.FilePath)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return []*schema.Document{{Content: string(data)}}, nil
|
||||||
|
},
|
||||||
|
)).AddInput(compose.START)
|
||||||
|
|
||||||
|
// chunk: Split into chunks
|
||||||
|
wf.AddLambdaNode("chunk", compose.InvokableLambda(
|
||||||
|
func(ctx context.Context, docs []*schema.Document) ([]*schema.Document, error) {
|
||||||
|
var out []*schema.Document
|
||||||
|
for _, d := range docs {
|
||||||
|
out = append(out, splitIntoChunks(d.Content, 800)...)
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
},
|
||||||
|
)).AddInput("load")
|
||||||
|
|
||||||
|
// score: Parallel scoring
|
||||||
|
scorer := batch.NewBatchNode(&batch.NodeConfig[scoreTask, scoredChunk]{
|
||||||
|
Name: "ChunkScorer",
|
||||||
|
InnerTask: newScoreWorkflow(cm),
|
||||||
|
MaxConcurrency: 5,
|
||||||
|
})
|
||||||
|
|
||||||
|
wf.AddLambdaNode("score", compose.InvokableLambda(
|
||||||
|
func(ctx context.Context, in scoreIn) ([]scoredChunk, error) {
|
||||||
|
tasks := make([]scoreTask, len(in.Chunks))
|
||||||
|
for i, c := range in.Chunks {
|
||||||
|
tasks[i] = scoreTask{Text: c.Content, Question: in.Question}
|
||||||
|
}
|
||||||
|
return scorer.Invoke(ctx, tasks)
|
||||||
|
},
|
||||||
|
)).
|
||||||
|
AddInputWithOptions("chunk", []*compose.FieldMapping{compose.ToField("Chunks")}, compose.WithNoDirectDependency()).
|
||||||
|
AddInputWithOptions(compose.START, []*compose.FieldMapping{compose.MapFields("Question", "Question")}, compose.WithNoDirectDependency())
|
||||||
|
|
||||||
|
// filter: Select top-k
|
||||||
|
wf.AddLambdaNode("filter", compose.InvokableLambda(
|
||||||
|
func(ctx context.Context, scored []scoredChunk) ([]scoredChunk, error) {
|
||||||
|
sort.Slice(scored, func(i, j int) bool {
|
||||||
|
return scored[i].Score > scored[j].Score
|
||||||
|
})
|
||||||
|
// Return top-3
|
||||||
|
if len(scored) > 3 {
|
||||||
|
scored = scored[:3]
|
||||||
|
}
|
||||||
|
return scored, nil
|
||||||
|
},
|
||||||
|
)).AddInput("score")
|
||||||
|
|
||||||
|
// answer: Generate the answer
|
||||||
|
wf.AddLambdaNode("answer", compose.InvokableLambda(
|
||||||
|
func(ctx context.Context, in synthIn) (Output, error) {
|
||||||
|
return synthesize(ctx, cm, in)
|
||||||
|
},
|
||||||
|
)).
|
||||||
|
AddInputWithOptions("filter", []*compose.FieldMapping{compose.ToField("TopK")}, compose.WithNoDirectDependency()).
|
||||||
|
AddInputWithOptions(compose.START, []*compose.FieldMapping{compose.MapFields("Question", "Question")}, compose.WithNoDirectDependency())
|
||||||
|
|
||||||
|
wf.End().AddInput("answer")
|
||||||
|
|
||||||
|
return wf
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Wrap as a Tool
|
||||||
|
|
||||||
|
```go
|
||||||
|
func BuildTool(ctx context.Context, cm model.BaseChatModel) (tool.BaseTool, error) {
|
||||||
|
wf := buildWorkflow(cm)
|
||||||
|
return graphtool.NewInvokableGraphTool[Input, Output](
|
||||||
|
wf,
|
||||||
|
"answer_from_document",
|
||||||
|
"Search a large document for relevant content and synthesize an answer.",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Key code snippet (Note: this is a simplified code snippet that cannot be run directly. For the complete code, please refer to** [rag/rag.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/rag/rag.go)**)**:
|
||||||
|
|
||||||
|
```go
|
||||||
|
// Build the workflow
|
||||||
|
wf := compose.NewWorkflow[Input, Output]()
|
||||||
|
|
||||||
|
// Add nodes
|
||||||
|
wf.AddLambdaNode("load", loadFunc).AddInput(compose.START)
|
||||||
|
wf.AddLambdaNode("chunk", chunkFunc).AddInput("load")
|
||||||
|
wf.AddLambdaNode("score", scoreFunc).
|
||||||
|
AddInputWithOptions("chunk", []*compose.FieldMapping{compose.ToField("Chunks")}, compose.WithNoDirectDependency()).
|
||||||
|
AddInputWithOptions(compose.START, []*compose.FieldMapping{compose.MapFields("Question", "Question")}, compose.WithNoDirectDependency())
|
||||||
|
|
||||||
|
// Wrap as a Tool
|
||||||
|
return graphtool.NewInvokableGraphTool[Input, Output](wf, "answer_from_document", "...")
|
||||||
|
```
|
||||||
|
|
||||||
|
## Graph Tool Execution Flow
|
||||||
|
|
||||||
|
```
|
||||||
|
+------------------------------------------+
|
||||||
|
| Input: file_path, question |
|
||||||
|
+------------------------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| load: Read file |
|
||||||
|
| Output: []*Document |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| chunk: Split into |
|
||||||
|
| chunks |
|
||||||
|
| Output: []*Document |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| score: Parallel |
|
||||||
|
| scoring |
|
||||||
|
| (MaxConcurrency=5) |
|
||||||
|
| Output: []scoredChunk |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| filter: Select top-k |
|
||||||
|
| Output: []scoredChunk |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| answer: Generate |
|
||||||
|
| answer |
|
||||||
|
| Output: Output |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| Return result |
|
||||||
|
| {answer, sources} |
|
||||||
|
+------------------------+
|
||||||
|
```
|
||||||
|
|
||||||
|
## Chapter Summary
|
||||||
|
|
||||||
|
- **Graph Tool**: Wraps complex workflows as a Tool, supporting multi-step coordination
|
||||||
|
- **compose.Workflow**: The core component for building workflows
|
||||||
|
- **BatchNode**: Parallel processing of multiple tasks
|
||||||
|
- **FieldMapping**: Passing data across nodes
|
||||||
|
- **Interrupt/Resume support**: Graph Tools support the Checkpoint mechanism
|
||||||
|
|
||||||
|
## Further Thinking
|
||||||
|
|
||||||
|
**Other Graph Tool applications:**
|
||||||
|
- Multi-document RAG: Process multiple documents in parallel
|
||||||
|
- Multi-model collaboration: Different models handle different tasks
|
||||||
|
- Complex decision trees: Choose different branches based on conditions
|
||||||
|
|
||||||
|
**Performance optimization:**
|
||||||
|
- Adjust MaxConcurrency to control parallelism
|
||||||
|
- Use caching to avoid redundant computation
|
||||||
|
- Use streaming output to improve user experience
|
||||||
@ -0,0 +1,137 @@
|
|||||||
|
---
|
||||||
|
title: "Chapter 9: Skill (Console)"
|
||||||
|
---
|
||||||
|
|
||||||
|
The goal of this chapter is to build on Chapter 8 (RAG + Interrupt/Resume + Checkpoint) by introducing the `skill` middleware, enabling the Agent to discover and load a set of reusable skill documents (`SKILL.md`) and use them via tool calls when needed.
|
||||||
|
|
||||||
|
## Code Location
|
||||||
|
|
||||||
|
- Entry code: [cmd/ch09/main.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/cmd/ch09/main.go)
|
||||||
|
- Sync script: [scripts/sync_eino_ext_skills.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/scripts/sync_eino_ext_skills.go)
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
- Same as Chapter 1: you need to configure an available ChatModel (OpenAI or Ark)
|
||||||
|
- Have the skills from the `eino-ext` PR ready (`eino-guide` / `eino-component` / `eino-compose` / `eino-agent`)
|
||||||
|
|
||||||
|
Why these four?
|
||||||
|
|
||||||
|
ChatWithEino is positioned as "helping users learn the Eino framework and try writing Eino code with AI assistance". These four skills cover exactly the key knowledge areas needed for this goal:
|
||||||
|
|
||||||
|
- `eino-guide`: Learning entry point and navigation (where to start, how to get up and running quickly)
|
||||||
|
- `eino-component`: Component interfaces and various implementation references (Model/Embedding/Retriever/Tool/Callback, etc.)
|
||||||
|
- `eino-compose`: Orchestration and deterministic workflow references (Graph/Chain/Workflow, etc.)
|
||||||
|
- `eino-agent`: ADK/Agent-related references (Agent, Runner, Middleware, Filesystem, Human-in-the-loop, etc.)
|
||||||
|
|
||||||
|
Skills can come from:
|
||||||
|
|
||||||
|
- A local path to the `eino-ext` repository (the script will automatically read from `<src>/skills/...`)
|
||||||
|
- Or a directory where you've already installed the skills (the directory should contain subdirectories like the four mentioned above)
|
||||||
|
|
||||||
|
## From Graph Tool to Skill: Why We Need "Skill Documents"
|
||||||
|
|
||||||
|
Chapter 8 addressed "how to turn a complex workflow into a callable Tool" (Graph Tool). But when building an Agent for framework learning/development assistance, you'll encounter another type of problem: **How do you inject a set of stable, reusable knowledge and instructions into an Agent and let it load them on demand at runtime?**
|
||||||
|
|
||||||
|
This is the role of Skills:
|
||||||
|
|
||||||
|
- **Tools** are more like "actions/capabilities": read files, run workflows, call external systems
|
||||||
|
- **Skills** are more like "reusable knowledge/instruction packages": described by a set of markdown files (`SKILL.md` + `reference/*.md`) that explain "how to do a certain type of thing"
|
||||||
|
|
||||||
|
Simple analogy:
|
||||||
|
|
||||||
|
- **Tool** = "what it can do" (function/interface)
|
||||||
|
- **Skill** = "how to do it" (reusable manual/operations guide)
|
||||||
|
|
||||||
|
## Running
|
||||||
|
|
||||||
|
In the `quickstart/chatwitheino` directory, run:
|
||||||
|
|
||||||
|
### 1) Sync eino-ext Skills to a Local Directory
|
||||||
|
|
||||||
|
For the `skill` middleware to "discover" these skills, they need to be placed in a unified directory that follows the scanning convention:
|
||||||
|
|
||||||
|
- `EINO_EXT_SKILLS_DIR/<skillName>/SKILL.md`
|
||||||
|
|
||||||
|
Sync command (recommended):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
go run ./scripts/sync_eino_ext_skills.go -src /path/to/eino-ext -dest ./skills/eino-ext -clean
|
||||||
|
```
|
||||||
|
|
||||||
|
Notes:
|
||||||
|
|
||||||
|
- `-src` supports two forms:
|
||||||
|
- The `eino-ext` repository root directory (the script will automatically read from `<src>/skills/...`)
|
||||||
|
- A directory where you've already installed skills (should contain subdirectories like `eino-guide/`, `eino-component/`, etc.)
|
||||||
|
- `-dest` defaults to `./skills/eino-ext` (can be omitted)
|
||||||
|
|
||||||
|
### 2) Start Chapter 9
|
||||||
|
|
||||||
|
```bash
|
||||||
|
EINO_EXT_SKILLS_DIR=/absolute/path/to/chatwitheino/skills/eino-ext go run ./cmd/ch09
|
||||||
|
```
|
||||||
|
|
||||||
|
Output example (excerpt):
|
||||||
|
|
||||||
|
```text
|
||||||
|
Skills dir: /.../skills/eino-ext
|
||||||
|
Enter your message (empty line to exit):
|
||||||
|
```
|
||||||
|
|
||||||
|
## Enabling Skills in DeepAgent
|
||||||
|
|
||||||
|
The "Skills can be called" behavior in this chapter doesn't happen automatically — you need to register the `skill` middleware when building the Agent. The core process is three steps:
|
||||||
|
|
||||||
|
1. Use a local filesystem backend (this chapter uses `eino-ext/adk/backend/local`) to provide file reading/Glob capabilities
|
||||||
|
2. Use `skill.NewBackendFromFilesystem` to turn `EINO_EXT_SKILLS_DIR` into a Skill Backend
|
||||||
|
3. Use `skill.NewMiddleware` to generate the middleware and add it to DeepAgent's `Handlers`
|
||||||
|
|
||||||
|
**Key code snippet (Note: this is a simplified code snippet that cannot be run directly. For the complete code, please refer to** [cmd/ch09/main.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/cmd/ch09/main.go)**)**:
|
||||||
|
|
||||||
|
```go
|
||||||
|
backend, _ := localbk.NewBackend(ctx, &localbk.Config{})
|
||||||
|
|
||||||
|
skillBackend, _ := skill.NewBackendFromFilesystem(ctx, &skill.BackendFromFilesystemConfig{
|
||||||
|
Backend: backend,
|
||||||
|
BaseDir: skillsDir, // = $EINO_EXT_SKILLS_DIR
|
||||||
|
})
|
||||||
|
skillMiddleware, _ := skill.NewMiddleware(ctx, &skill.Config{
|
||||||
|
Backend: skillBackend,
|
||||||
|
})
|
||||||
|
|
||||||
|
agent, _ := deep.New(ctx, &deep.Config{
|
||||||
|
ChatModel: cm,
|
||||||
|
Backend: backend,
|
||||||
|
StreamingShell: backend,
|
||||||
|
Handlers: []adk.ChatModelAgentMiddleware{
|
||||||
|
skillMiddleware,
|
||||||
|
// ... other middleware, such as approval/safeTool/retry, etc.
|
||||||
|
},
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
Additional notes:
|
||||||
|
|
||||||
|
- To ensure this quickstart "works even without configured skills", the code checks for the existence of `EINO_EXT_SKILLS_DIR`: `skillMiddleware` is only registered if the directory exists; otherwise it's skipped (you can still chat and use RAG tools in that case).
|
||||||
|
- The Skill tool's input is a JSON object: `{"skill": "<skillName>"}`, for example `{"skill":"eino-guide"}`.
|
||||||
|
|
||||||
|
## Quick Verification (Recommended)
|
||||||
|
|
||||||
|
After starting, enter a command that explicitly asks the model to call the skill tool (to verify that skills have been discovered and can be loaded):
|
||||||
|
|
||||||
|
```text
|
||||||
|
Use the skill tool with skill="eino-guide" and tell me what the entry point is for getting started.
|
||||||
|
```
|
||||||
|
|
||||||
|
You should see output similar to the following in the console:
|
||||||
|
|
||||||
|
- `[tool result] Launching skill: eino-guide`
|
||||||
|
- The tool result contains `Base directory for this skill: .../eino-guide`
|
||||||
|
|
||||||
|
## What You'll See
|
||||||
|
|
||||||
|
- When the model calls the skill tool, the console will print:
|
||||||
|
- `[tool call] ...`
|
||||||
|
- `[tool result] ...` (results are truncated for display)
|
||||||
|
- Sessions are saved in `SESSION_DIR` (default `./data/sessions`), with recovery support:
|
||||||
|
- `go run ./cmd/ch09 --session <id>`
|
||||||
@ -0,0 +1,244 @@
|
|||||||
|
---
|
||||||
|
title: "Chapter 10: A2UI Protocol (Streaming UI Components)"
|
||||||
|
---
|
||||||
|
|
||||||
|
The goal of this chapter is to implement the A2UI protocol, rendering Agent output as streaming UI components.
|
||||||
|
|
||||||
|
## Important Note: The Scope of A2UI
|
||||||
|
|
||||||
|
A2UI is not part of the Eino framework itself — it is a business-layer UI protocol/rendering solution. This chapter integrates A2UI into the Agent built step by step in previous chapters, in order to provide a complete end-to-end, production-ready example: from model calls, tool calls, and workflow orchestration, all the way to presenting results in a more user-friendly UI format.
|
||||||
|
|
||||||
|
In real business scenarios, you are free to choose different UI forms based on your product needs, such as:
|
||||||
|
|
||||||
|
- Web / App: Custom components, tables, cards, charts, etc.
|
||||||
|
- IM/office suites: Message cards, interactive forms
|
||||||
|
- Command line: Plain text or TUI (Terminal UI)
|
||||||
|
|
||||||
|
Eino focuses on "composable intelligent execution and orchestration capabilities". As for "how to present it to users", that's a business-layer concern that can be freely extended.
|
||||||
|
|
||||||
|
## Code Location
|
||||||
|
|
||||||
|
- Entry code: [main.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/main.go)
|
||||||
|
- Agent construction: [agent.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/agent.go)
|
||||||
|
- Server routing: [server/server.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/server/server.go)
|
||||||
|
- A2UI subset implementation: [a2ui/types.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/a2ui/types.go)
|
||||||
|
- A2UI event stream conversion: [a2ui/streamer.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/a2ui/streamer.go)
|
||||||
|
- Frontend page: [static/index.html](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/static/index.html)
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
Same as Chapter 1: you need to configure an available ChatModel (OpenAI or Ark).
|
||||||
|
|
||||||
|
## Running
|
||||||
|
|
||||||
|
In the `quickstart/chatwitheino` directory, run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
go run .
|
||||||
|
```
|
||||||
|
|
||||||
|
Output example:
|
||||||
|
|
||||||
|
```text
|
||||||
|
starting server on http://localhost:8080
|
||||||
|
```
|
||||||
|
|
||||||
|
### (Optional) Enable Chapter 9 Skills
|
||||||
|
|
||||||
|
The final Web version uses Agent construction logic aligned with Chapter 9: when `EINO_EXT_SKILLS_DIR` points to a valid skills directory, the `skill` middleware is automatically registered, enabling the model to call the `skill` tool on demand to load `eino-guide` / `eino-component` / `eino-compose` / `eino-agent`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
go run ./scripts/sync_eino_ext_skills.go -src /path/to/eino-ext -dest ./skills/eino-ext -clean
|
||||||
|
EINO_EXT_SKILLS_DIR="$(pwd)/skills/eino-ext" go run .
|
||||||
|
```
|
||||||
|
|
||||||
|
## From Text to UI: Why We Need A2UI
|
||||||
|
|
||||||
|
In the first eight chapters, the Agent only output text, but modern AI applications need richer interactions.
|
||||||
|
|
||||||
|
**Limitations of plain text output:**
|
||||||
|
- Cannot display structured data (tables, lists, cards, etc.)
|
||||||
|
- Cannot update in real time (progress bars, status changes, etc.)
|
||||||
|
- Cannot embed interactive elements (buttons, forms, links, etc.)
|
||||||
|
- Cannot support multimedia (images, video, audio, etc.)
|
||||||
|
|
||||||
|
**The role of A2UI:**
|
||||||
|
- **A2UI is the Agent-to-UI protocol**: Defines how Agent output maps to UI components
|
||||||
|
- **A2UI supports streaming rendering**: Components can update in real time without waiting for the complete response
|
||||||
|
- **A2UI is declarative**: The Agent only needs to declare "what to display", and the UI handles rendering
|
||||||
|
|
||||||
|
**Simple analogy:**
|
||||||
|
- **Plain text output** = "terminal command line" (can only display text)
|
||||||
|
- **A2UI** = "web application" (can display any UI component)
|
||||||
|
|
||||||
|
## Key Concepts
|
||||||
|
|
||||||
|
### A2UI v0.8 Subset (Scope of This Example)
|
||||||
|
|
||||||
|
This quickstart does not implement a "complete A2UI standard library". Instead, it implements a **subset of A2UI v0.8**: the goal is to push Agent event streams to the browser as a stable, incrementally renderable UI component tree.
|
||||||
|
|
||||||
|
The currently implemented A2UI message types and component types are defined in [a2ui/types.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/a2ui/types.go).
|
||||||
|
|
||||||
|
### A2UI Messages: BeginRendering / SurfaceUpdate / DataModelUpdate / InterruptRequest
|
||||||
|
|
||||||
|
Each SSE line (`data: {...}`) carries one A2UI Message. A Message is an "envelope structure" where only one field is present at a time:
|
||||||
|
|
||||||
|
**Key code snippet (Note: this is a simplified code snippet that cannot be run directly. For the complete code, please refer to** [a2ui/types.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/a2ui/types.go)**)**:
|
||||||
|
|
||||||
|
```go
|
||||||
|
type Message struct {
|
||||||
|
BeginRendering *BeginRenderingMsg
|
||||||
|
SurfaceUpdate *SurfaceUpdateMsg
|
||||||
|
DataModelUpdate *DataModelUpdateMsg
|
||||||
|
DeleteSurface *DeleteSurfaceMsg
|
||||||
|
InterruptRequest *InterruptRequestMsg
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Where:
|
||||||
|
|
||||||
|
- `BeginRendering`: Tells the frontend to "start rendering a surface (session)" and specifies the root node ID
|
||||||
|
- `SurfaceUpdate`: Adds/updates a batch of components (components form a tree, referencing each other by `id`)
|
||||||
|
- `DataModelUpdate`: Updates data bindings (used to incrementally update streaming text to a Text component)
|
||||||
|
- `InterruptRequest`: When the Agent triggers an interrupt (e.g., approval), notifies the frontend to display an approve/reject entry
|
||||||
|
|
||||||
|
### A2UI Components: Text / Column / Card / Row
|
||||||
|
|
||||||
|
This example implements only 4 UI component types (see [a2ui/types.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/a2ui/types.go)):
|
||||||
|
|
||||||
|
- `Text`: Text rendering (supports `usageHint` to distinguish caption/body/title); when `dataKey` is present, text comes from `DataModelUpdate`
|
||||||
|
- `Column` / `Row`: Layout (children are lists of component IDs)
|
||||||
|
- `Card`: Card container (children are lists of component IDs)
|
||||||
|
|
||||||
|
## A2UI Implementation: Converting AgentEvent to A2UI SSE
|
||||||
|
|
||||||
|
The core pipeline of the final Web version is:
|
||||||
|
|
||||||
|
- The backend runs the Agent, producing `*adk.AsyncIterator[*adk.AgentEvent]`
|
||||||
|
- The event stream is converted to A2UI JSONL/SSE stream output for the browser (see [a2ui/streamer.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/a2ui/streamer.go))
|
||||||
|
- The frontend parses `data:` lines from SSE and renders the component tree (see [static/index.html](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/static/index.html))
|
||||||
|
|
||||||
|
### Server Routes (High Level)
|
||||||
|
|
||||||
|
Key endpoints related to A2UI (see [server/server.go](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/server/server.go)):
|
||||||
|
|
||||||
|
- `GET /`: Returns the frontend page `static/index.html`
|
||||||
|
- `POST /sessions/:id/chat`: Returns an SSE stream (A2UI messages), rendering Agent results to the UI as they're generated
|
||||||
|
- `GET /sessions/:id/render`: Returns JSONL (A2UI messages), used for "replaying history when selecting a session"
|
||||||
|
- `POST /sessions/:id/approve`: Handles interrupt approval/rejection and continues returning the SSE stream
|
||||||
|
|
||||||
|
### Event Stream Conversion (High Level)
|
||||||
|
|
||||||
|
The server passes the `Runner.Run(...)` event stream to `a2ui.StreamToWriter(...)`, which handles:
|
||||||
|
|
||||||
|
- Splitting user/assistant/tool output
|
||||||
|
- Rendering tool calls / tool results as "chip cards"
|
||||||
|
- Converting the assistant's streaming tokens into `DataModelUpdate`, enabling "render as you generate"
|
||||||
|
- Sending `InterruptRequest` when an interrupt is encountered, and pausing to wait for human approval
|
||||||
|
|
||||||
|
## Frontend Integration: fetch + SSE (Not WebSocket)
|
||||||
|
|
||||||
|
- The frontend initiates a request via `fetch('/sessions/:id/chat')`, then reads streaming bytes from `res.body`, splits by line, and parses the JSON from `data: {...}` lines (see [static/index.html](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/static/index.html)).
|
||||||
|
|
||||||
|
**Key code snippet (Note: this is a simplified code snippet that cannot be run directly. For the complete code, please refer to** [static/index.html](https://github.com/cloudwego/eino-examples/blob/main/quickstart/chatwitheino/static/index.html)**)**:
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
const res = await fetch(`/sessions/${id}/chat`, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: {'Content-Type': 'application/json'},
|
||||||
|
body: JSON.stringify({message}),
|
||||||
|
});
|
||||||
|
|
||||||
|
const reader = res.body.getReader();
|
||||||
|
const decoder = new TextDecoder();
|
||||||
|
let buffer = '';
|
||||||
|
while (true) {
|
||||||
|
const {done, value} = await reader.read();
|
||||||
|
if (done) break;
|
||||||
|
buffer += decoder.decode(value, {stream: true});
|
||||||
|
const lines = buffer.split('\n');
|
||||||
|
buffer = lines.pop();
|
||||||
|
for (const line of lines) {
|
||||||
|
const trimmed = line.trim();
|
||||||
|
if (trimmed.startsWith('data:')) {
|
||||||
|
const jsonStr = trimmed.slice(5).trimStart();
|
||||||
|
processA2UIMessage(JSON.parse(jsonStr));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## A2UI Streaming Rendering Flow (Overview)
|
||||||
|
|
||||||
|
```
|
||||||
|
+------------------------------------------+
|
||||||
|
| User: Analyze this file |
|
||||||
|
+------------------------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| Agent starts |
|
||||||
|
| processing |
|
||||||
|
| A2UI: AddText |
|
||||||
|
| "Analyzing..." |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| Call Tool |
|
||||||
|
| A2UI: AddProgress |
|
||||||
|
| Progress: 0% |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| Tool executing |
|
||||||
|
| A2UI: UpdateProgress |
|
||||||
|
| Progress: 50% |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| Tool complete |
|
||||||
|
| A2UI: tool result |
|
||||||
|
+------------------------+
|
||||||
|
|
|
||||||
|
+------------------------+
|
||||||
|
| Display result |
|
||||||
|
| A2UI: DataModelUpdate |
|
||||||
|
| (streaming assistant |
|
||||||
|
| update) |
|
||||||
|
+------------------------+
|
||||||
|
```
|
||||||
|
|
||||||
|
## Chapter Summary
|
||||||
|
|
||||||
|
- **A2UI**: The Agent-to-UI protocol, defining how Agent output maps to UI components
|
||||||
|
- **Subset implementation**: This example only implements Text/Column/Card/Row and data binding
|
||||||
|
- **Streaming output**: The backend pushes A2UI JSONL via SSE, and the frontend incrementally renders the component tree
|
||||||
|
- **Events to UI**: Converts `AgentEvent` into visualized output for tool calls / tool results / assistant streams
|
||||||
|
|
||||||
|
## Series Conclusion: The Complete Vision for This Quickstart Agent
|
||||||
|
|
||||||
|
By this chapter, we've used a fully runnable Agent to tie together Eino's core capabilities. You can think of it as an extensible "end-to-end Agent application skeleton":
|
||||||
|
|
||||||
|
- **Runtime**: Runner drives execution, with support for streaming output and event models
|
||||||
|
- **Tool layer**: Filesystem / Shell and other Tool capabilities, with safe tool error handling
|
||||||
|
- **Middleware**: Pluggable middleware/handlers for cross-cutting concerns like error handling, retries, approvals, etc.
|
||||||
|
- **Observability**: Callbacks/trace capabilities connect key call chains, facilitating debugging and production monitoring
|
||||||
|
- **Human-agent collaboration**: Interrupt/resume + checkpoint support for approvals, parameter completion, branch selection, and other interactive flows
|
||||||
|
- **Deterministic orchestration**: compose (graph/chain/workflow) organizes complex business flows into maintainable, reusable execution graphs
|
||||||
|
- **Business delivery**: UI integrations like A2UI are a business-layer choice, used to present Agent capabilities in the appropriate product form to users
|
||||||
|
|
||||||
|
You can gradually replace/extend any part of this skeleton — model, tools, storage, workflows, frontend rendering protocol — without starting from scratch.
|
||||||
|
|
||||||
|
## Further Thinking
|
||||||
|
|
||||||
|
**Other component types:**
|
||||||
|
- Chart components (line charts, bar charts, pie charts)
|
||||||
|
- Map components
|
||||||
|
- Timeline components
|
||||||
|
- Tree components
|
||||||
|
- Tab components
|
||||||
|
|
||||||
|
**Advanced features:**
|
||||||
|
- Component interaction (click, drag, input)
|
||||||
|
- Conditional rendering
|
||||||
|
- Component animations
|
||||||
|
- Responsive layout
|
||||||
Loading…
Reference in New Issue