Compare commits
11 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 4e592173a6 | |||
| 0cacb5c94a | |||
| 6c96409969 | |||
| 06d51c1ee9 | |||
| 2ad3d1df38 | |||
| 83c7770877 | |||
| bad14fbe7f | |||
| 578e8f55bd | |||
| ae3e432b02 | |||
| 7c14390400 | |||
| 89885c1583 |
@@ -6,7 +6,7 @@ Julia framework for building agents with tool use.
|
||||
|
||||
1. Install dependencies: `]add JSON, DataStructures, UUIDs, Dates, ...`
|
||||
2. Create a `yiemAgent` with `loadTools("src/tools")`
|
||||
3. Call `run_agent(agent, "message")` then `take_response(agent)`
|
||||
3. Call `runAgent(agent, "message")` then `takeResponse(agent)`
|
||||
|
||||
## Architecture
|
||||
|
||||
@@ -16,7 +16,7 @@ src/
|
||||
├── type.jl # Core types (messages, tools, agent state)
|
||||
├── utils.jl # Message formatting, validation
|
||||
├── agentCore.jl # Agent loop, tool execution pipeline
|
||||
├── api.jl # Public API (run_agent, take_response, etc.)
|
||||
├── api.jl # Public API (runAgent, takeResponse, etc.)
|
||||
└── tools/
|
||||
├── registry.jl # Tool registry (loadTools, registerTool, listTools)
|
||||
├── getWeather.jl # Weather lookup tool
|
||||
|
||||
@@ -151,7 +151,7 @@ result = getTime_tool.execute("call-1", Dict("city" => "Tokyo"), sig, op)
|
||||
|
||||
**Via agent loop (production):**
|
||||
```
|
||||
user message → run_agent(agent, Dict("role"=>"user", "content"=>...))
|
||||
user message → runAgent(agent, Dict("role"=>"user", "content"=>...))
|
||||
→ _agent_loop detects message → @spawn _process_message(agent)
|
||||
→ prepareContext → formatMsgForLLM → llmCall
|
||||
→ LLM returns tool_calls
|
||||
@@ -382,9 +382,9 @@ The `_agent_loop()` function runs as a background `@spawn` task, created when `y
|
||||
|
||||
```
|
||||
yiemAgent struct contains:
|
||||
- inputChannel (Channel, capacity 16) ← user sends messages here via run_agent()
|
||||
- followUpChannel (Channel, capacity 32) ← user sends follow-ups here via follow_up()
|
||||
- outputChannel (Channel, capacity 16) ← agent sends responses here via take_response()
|
||||
- inputChannel (Channel, capacity 16) ← user sends messages here via runAgent()
|
||||
- followUpChannel (Channel, capacity 32) ← user sends follow-ups here via followUp()
|
||||
- outputChannel (Channel, capacity 16) ← agent sends responses here via takeResponse()
|
||||
- _tool_store (toolStore) ← per-agent isolated tool registry
|
||||
```
|
||||
|
||||
@@ -633,7 +633,7 @@ function prepareToolCall(
|
||||
- On failure: throws `ArgumentError(error_string)`, caught by the try-catch below
|
||||
|
||||
4. **Run `beforeToolCall` hook** — if `config.beforeToolCall !== nothing`
|
||||
- Passes `assistantMsgCtx(assistantMsg, toolCall, validatedArgs, context)` and `signal`
|
||||
- Passes `beforeToolCallContext(assistantMsg, toolCall, validatedArgs, context)` and `signal`
|
||||
- Hook can return `nothing` (proceed), or `Dict(:block => true, :reason => "...")` (reject)
|
||||
- If `signal.aborted == true` → `immediateOutcome(createErrorToolResult("Operation aborted"), true)`
|
||||
- If `before.block == true` → `immediateOutcome(createErrorToolResult(get(before, :reason, "blocked")), true)`
|
||||
@@ -722,10 +722,10 @@ function finalizeExecutedToolCall(
|
||||
```
|
||||
|
||||
2. **Run `afterToolCall` hook** — if `config.afterToolCall !== nothing`:
|
||||
- Passes `afterCtx(assistantMsg, prep.toolCall, prep.args, result, isError, context)` and `signal`
|
||||
- Passes `afterToolCallContext(assistantMsg, prep.toolCall, prep.args, result, isError, context)` and `signal`
|
||||
- Hook can mutate the result:
|
||||
```julia
|
||||
after = config.afterToolCall(afterCtx(...))
|
||||
after = config.afterToolCall(afterToolCallContext(...))
|
||||
if after !== nothing
|
||||
result = merge(result, dict(
|
||||
:content => get(after, :content, result.content),
|
||||
@@ -1146,8 +1146,8 @@ The `agentEventSink` function is a user-provided callback that receives all even
|
||||
| `prepareContext` | `(state::agentState) -> agentContext` | Before each LLM call | Filter tools, inject context, modify system prompt |
|
||||
| `formatMsgForLLM` | `(ctx::agentContext) -> Dict` | After `prepareContext` | Convert to LLM-specific format |
|
||||
| `llmCall` | `(messages::Dict) -> assistantMessage` | After formatting | Actually invoke the LLM API |
|
||||
| `beforeToolCall` | `(msgCtx::assistantMsgCtx, signal) -> Union{Nothing, Dict}` | In `prepareToolCall` | Ask for user permission, block execution, abort |
|
||||
| `afterToolCall` | `(afterCtx::afterCtx, signal) -> Union{Nothing, Dict}` | In `finalizeExecutedToolCall` | Mutate result, mask data, flip `terminate` |
|
||||
| `beforeToolCall` | `(msgCtx::beforeToolCallContext, signal) -> Union{Nothing, Dict}` | In `prepareToolCall` | Ask for user permission, block execution, abort |
|
||||
| `afterToolCall` | `(afterToolCallContext::afterToolCallContext, signal) -> Union{Nothing, Dict}` | In `finalizeExecutedToolCall` | Mutate result, mask data, flip `terminate` |
|
||||
| `agentEventSink` | `(event) -> nothing` | Throughout lifecycle | Emit events for TUI, logging, monitoring |
|
||||
|
||||
### `beforeToolCall` Hook
|
||||
@@ -1157,7 +1157,7 @@ The `agentEventSink` function is a user-provided callback that receives all even
|
||||
```julia
|
||||
if config.beforeToolCall !== nothing
|
||||
before = config.beforeToolCall(
|
||||
assistantMsgCtx(assistantMsg, toolCall, validatedArgs, context), signal
|
||||
beforeToolCallContext(assistantMsg, toolCall, validatedArgs, context), signal
|
||||
)
|
||||
if signal !== nothing && signal.aborted
|
||||
return immediateOutcome(createErrorToolResult("Operation aborted"), true)
|
||||
@@ -1182,7 +1182,7 @@ end
|
||||
if config.afterToolCall !== nothing
|
||||
try
|
||||
after = config.afterToolCall(
|
||||
afterCtx(assistantMsg, prep.toolCall, prep.args, result, isError, context), signal
|
||||
afterToolCallContext(assistantMsg, prep.toolCall, prep.args, result, isError, context), signal
|
||||
)
|
||||
if after !== nothing
|
||||
result = merge(result, dict(
|
||||
@@ -1327,7 +1327,7 @@ Each `toolStore` gets its own `listTool` instance bound to that store via `listT
|
||||
|
||||
```
|
||||
USER SENDS MESSAGE
|
||||
└─> run_agent(agent, "What's the weather in Tokyo?")
|
||||
└─> runAgent(agent, "What's the weather in Tokyo?")
|
||||
└─> put!(agent.inputChannel, Dict("role" => "user", "content" => [...]))
|
||||
|
||||
|
||||
@@ -1430,7 +1430,7 @@ LOOP ITERATION 2 — LLM RETURNS FINAL TEXT RESPONSE
|
||||
|
||||
AGENT LOOP: SEND RESPONSE TO USER
|
||||
└─> put!(agent.outputChannel, final_response)
|
||||
└─> take_response(agent) → assistantMessage("The weather in Tokyo is sunny, 22°C.")
|
||||
└─> takeResponse(agent) → assistantMessage("The weather in Tokyo is sunny, 22°C.")
|
||||
```
|
||||
|
||||
---
|
||||
@@ -1606,8 +1606,8 @@ All functions in the file are scoped under `_tool_myTool`, preventing name colli
|
||||
| `agentState` | `type.jl:310` | Mutable runtime state (systemPrompt, model, tools, messages, pendingToolCalls, activeRun, errorMessage) |
|
||||
| `agentLoopConfig` | `type.jl:403` | Loop config (tools, beforeToolCall, afterToolCall, toolExecution) |
|
||||
| `abortSignal` | `type.jl:416` | Abort flag (`aborted::Bool`) |
|
||||
| `assistantMsgCtx` | `type.jl:445` | Context for beforeToolCall (message, toolCall, args, context) |
|
||||
| `afterCtx` | `type.jl:463` | Context for afterToolCall (message, toolCall, args, result, isError, context) |
|
||||
| `beforeToolCallContext` | `type.jl:445` | Context for beforeToolCall (message, toolCall, args, context) |
|
||||
| `afterToolCallContext` | `type.jl:463` | Context for afterToolCall (message, toolCall, args, result, isError, context) |
|
||||
|
||||
### Event Types
|
||||
|
||||
@@ -1,2 +1,77 @@
|
||||
# ── executeToolCalls() Julia pseudo code ──────────────────────────
|
||||
# Full call stack from runLoop → executeToolCalls → prepare → execute → finalize → emit
|
||||
i am not sure that's the case. see my NATS message log:
|
||||
<NATS debug message>
|
||||
┌ Info: debug
|
||||
└ payload = "new user msg"
|
||||
┌ Info: debug
|
||||
└ payload = "new user msg"
|
||||
┌ Info: debug
|
||||
└ payload = "new user msg"
|
||||
┌ Info: debug
|
||||
└ payload = "new user msg"
|
||||
┌ Info: debug
|
||||
└ payload = "new user msg"
|
||||
┌ Info: debug
|
||||
└ payload = "new user msg"
|
||||
┌ Info: debug
|
||||
└ payload = "new user msg"
|
||||
┌ Info: debug
|
||||
└ payload = "_process_message 3"
|
||||
┌ Info: debug
|
||||
└ payload = "_process_message 5"
|
||||
┌ Info: debug
|
||||
└ payload = "_process_message 6"
|
||||
┌ Info: debug
|
||||
└ payload = "_process_message 7"
|
||||
</NATS debug message>
|
||||
|
||||
my NATS receiver report the following for a long time
|
||||
┌ Info: debug
|
||||
└ payload = "new user msg"
|
||||
|
||||
untill I Ctrl + d so shutdown the process then i got the following report
|
||||
┌ Info: debug
|
||||
└ payload = "_process_message 3"
|
||||
┌ Info: debug
|
||||
└ payload = "_process_message 5"
|
||||
┌ Info: debug
|
||||
└ payload = "_process_message 6"
|
||||
┌ Info: debug
|
||||
└ payload = "_process_message 7"
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
my point is if _process_message() actually run then this code in _process_message()
|
||||
"raw_msg = take!(agent.inputChannel)"
|
||||
should take the new msg message out of agent.inputChannel and there should be only one debug message showing
|
||||
┌ Info: debug
|
||||
└ payload = "new user msg"
|
||||
|
||||
before reaching error("debug marker")
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
+203
-94
@@ -1,14 +1,142 @@
|
||||
module agentCore
|
||||
|
||||
export _agent_loop, OpenAiToUserMessage
|
||||
export yiemAgent, _agent_loop, OpenAiToUserMessage
|
||||
|
||||
using JSON, DataStructures, Dates, UUIDs, HTTP, Random, PrettyPrinting, Serialization,
|
||||
DataFrames, Base.Threads
|
||||
using GeneralUtils
|
||||
using ..type, ..utils
|
||||
using ..type, ..utils, ..toolRegistry
|
||||
|
||||
# ---------------------------------------------- 100 --------------------------------------------- #
|
||||
|
||||
"""
|
||||
docstring
|
||||
"""
|
||||
mutable struct yiemAgent <: agent # High-level agent wrapper
|
||||
_state::agentState # Current state (prompt, model, messages, tools, etc.)
|
||||
|
||||
# user sends prompt message to agent. if agent is idle, it process user message right away.
|
||||
# if agent is running, it process user message after the current tool call finished.
|
||||
inputChannel::Channel
|
||||
|
||||
# Buffers messages the user sends while the agent is busy. Processed after all inputChannel
|
||||
# messages are handled and the agent is idle (not using a tool call).
|
||||
followUpChannel::Channel
|
||||
|
||||
# agent sends response message to user after processing all user messages in inputChannel
|
||||
# and all followUp messages.
|
||||
outputChannel::Channel
|
||||
|
||||
_agent_loop::Union{Task, Nothing} # agent loop running in the background
|
||||
|
||||
# Preprocess/transform messages and context (modify, filter, prune, inject context from memory,
|
||||
# reorder, ...) for a single LLM call in _process_message()'s loop.
|
||||
# returns new Vector{agentMessage}
|
||||
prepareContext::Union{Function, Nothing}
|
||||
|
||||
# Convert prepareContext()'s new Vector{agentMessage} to LLM message format
|
||||
formatMsgForLLM::Function
|
||||
|
||||
# A callable struct. Actually invoke the LLM to get a completion response.
|
||||
# The LLM response comes back as an assistantMessage whose content is an array of content blocks.
|
||||
# Each block has a type — "text", "thinking", or "toolCall".
|
||||
# The code filters for type === "toolCall" blocks, then passes them to executeToolCalls().
|
||||
llmCall
|
||||
|
||||
# Callback invoked before executing a tool call (ask for user permission/confirmation/abort, etc..)
|
||||
beforeToolCall::Union{Function, Nothing}
|
||||
|
||||
# Callback invoked after executing a tool call to sanitize tools output so the output is ready
|
||||
# to be converted into toolResults message
|
||||
afterToolCall::Union{Function, Nothing}
|
||||
# prepareNextTurn::Union{Function, Nothing} # Callback to prepare the next conversation turn
|
||||
# prepareNextTurnWithContext::Union{Function, Nothing} # Same but receives context
|
||||
sessionId::Union{String, Nothing} # Optional session identifier
|
||||
maxRetryDelayMs::Union{Int64, Nothing} # Maximum delay between retries (ms)
|
||||
parallelToolExecute::Bool # Default: false
|
||||
agentEventSink # agent emits its status via this function
|
||||
end
|
||||
|
||||
"""
|
||||
Create a new yiemAgent instance with a background loop task.
|
||||
|
||||
Spawns a background `@spawn` task that runs the agent loop, listening
|
||||
on `inputChannel` and `followUpChannel` channels concurrently.
|
||||
|
||||
# Keyword Arguments
|
||||
- `systemPrompt::String`: System prompt for the agent
|
||||
- `model`: LLM model to use
|
||||
- `tools::OrderedDict{String, agentTool}`: Available tools keyed by name (default: empty)
|
||||
- `messages::Vector{agentMessage}`: Initial conversation messages (default: empty)
|
||||
- `formatMsgForLLM::Function`: Convert agent messages to LLM message format (default: `defaultformatMsgForLLM`)
|
||||
- `llmCall::Function`: Function to invoke the LLM (required)
|
||||
- `prepareContext::Union{Function, Nothing}`: Preprocess/transform messages before sending to LLM (default: `nothing`)
|
||||
- `beforeToolCall::Union{Function, Nothing}`: Callback invoked before executing a tool call (default: `nothing`)
|
||||
- `afterToolCall::Union{Function, Nothing}`: Callback invoked after executing a tool call (default: `nothing`)
|
||||
- `prepareNextTurn::Union{Function, Nothing}`: Callback to prepare the next conversation turn (default: `nothing`)
|
||||
- `prepareNextTurnWithContext::Union{Function, Nothing}`: Same but receives context (default: `nothing`)
|
||||
- `sessionId::Union{String, Nothing}`: Optional session identifier (default: `nothing`)
|
||||
- `maxRetryDelayMs::Union{Int64, Nothing}`: Maximum delay between retries in milliseconds (default: `nothing`)
|
||||
- `parallelToolExecute::Bool`: Run tool calls in parallel (default: `false`)
|
||||
- `agentEventSink::Function`: Callback to receive agent events
|
||||
|
||||
# Returns
|
||||
- A new `yiemAgent` instance with an active background task
|
||||
"""
|
||||
function yiemAgent(
|
||||
toolsFolderPath::String,
|
||||
llmCall,
|
||||
;
|
||||
systemPrompt::String="You are helpful assistant.",
|
||||
model=nothing,
|
||||
messages::Vector{agentMessage}=agentMessage[],
|
||||
prepareContext::Function=prepareContext,
|
||||
formatMsgForLLM::Function=formatMsgForLLM,
|
||||
beforeToolCall::Function=beforeToolCall,
|
||||
afterToolCall::Function=afterToolCall,
|
||||
# prepareNextTurn::Union{Function, Nothing}=nothing,
|
||||
# prepareNextTurnWithContext::Union{Function, Nothing}=nothing,
|
||||
sessionId::Union{String, Nothing}=nothing,
|
||||
maxRetryDelayMs::Union{Int64, Nothing}=nothing,
|
||||
parallelToolExecute::Bool=false,
|
||||
agentEventSink=agentEventSink,
|
||||
)
|
||||
# Create channels: input (user -> agent), followUp (async queue), output (agent -> user)
|
||||
inputChannel = Channel(16)
|
||||
followUp = Channel(32)
|
||||
outputChannel = Channel(16)
|
||||
|
||||
# load tools from toolsFolderPath
|
||||
toolStore1 = toolStore(name="myagent")
|
||||
loadTools(toolStore1, toolsFolderPath)
|
||||
|
||||
# Create struct with a placeholder task, then spawn and replace it
|
||||
agent = yiemAgent(
|
||||
agentState(systemPrompt, model, getTools(toolStore1), messages),
|
||||
inputChannel,
|
||||
followUp,
|
||||
outputChannel,
|
||||
nothing, # placeholder — replaced below
|
||||
prepareContext,
|
||||
formatMsgForLLM,
|
||||
llmCall,
|
||||
beforeToolCall,
|
||||
afterToolCall,
|
||||
# prepareNextTurn,
|
||||
# prepareNextTurnWithContext,
|
||||
sessionId,
|
||||
maxRetryDelayMs,
|
||||
parallelToolExecute,
|
||||
agentEventSink,
|
||||
)
|
||||
|
||||
# Spawn the background loop and attach it
|
||||
agent._agent_loop = @spawn _agent_loop(agent)
|
||||
|
||||
return agent
|
||||
end
|
||||
|
||||
|
||||
"""
|
||||
Private agent loop. Runs in a background `@spawn` task.
|
||||
|
||||
@@ -81,7 +209,8 @@ function _agent_loop(agent::yiemAgent)
|
||||
if isready(agent.inputChannel)
|
||||
|
||||
# message will be taken in _process_message()
|
||||
msg = fetch!(agent.inputChannel)
|
||||
msg = fetch(agent.inputChannel)
|
||||
agent.agentEventSink("new user msg")
|
||||
else
|
||||
yield()
|
||||
end
|
||||
@@ -107,9 +236,11 @@ function _agent_loop(agent::yiemAgent)
|
||||
|
||||
# start _process_message loop
|
||||
if agent._state.activeRun == false
|
||||
agent.agentEventSink("_agent_loop 2")
|
||||
# Dispatch message through the processing pipeline
|
||||
processingTask = Threads.@spawn _process_message(agent)
|
||||
processingTask = Threads.@spawn _process_message(agent)
|
||||
agent._state.activeRun = true
|
||||
agent.agentEventSink("_agent_loop 3")
|
||||
end
|
||||
|
||||
# during agent runs, check followUp message after _process_message() is done
|
||||
@@ -173,6 +304,7 @@ julia> # Currently returns a placeholder echo response
|
||||
```
|
||||
"""
|
||||
function _process_message(agent::yiemAgent)::assistantMessage
|
||||
agent.agentEventSink("_process_message 1")
|
||||
# loop until llmCall() response didn't use tool calls
|
||||
final_response = nothing
|
||||
while true
|
||||
@@ -185,21 +317,26 @@ function _process_message(agent::yiemAgent)::assistantMessage
|
||||
Dict(
|
||||
"type" => "image_url",
|
||||
"image_url" => Dict("url" => "data:mime_type;base64,image2_base64_string")
|
||||
)
|
||||
),
|
||||
]
|
||||
),
|
||||
)
|
||||
"""
|
||||
|
||||
# Drain inputChannel and convert OpenAI-format messages to userMessage type
|
||||
while isready(agent.inputChannel)
|
||||
agent.agentEventSink("_process_message 2")
|
||||
raw_msg = take!(agent.inputChannel)
|
||||
agent.agentEventSink("_process_message 3")
|
||||
if raw_msg === :shutdown
|
||||
agent.agentEventSink("_process_message 4")
|
||||
# Re-emit shutdown signal for the loop to handle
|
||||
put!(agent.inputChannel, :shutdown)
|
||||
break
|
||||
end
|
||||
agent.agentEventSink("_process_message 5")
|
||||
user_msg = OpenAiToUserMessage(raw_msg)
|
||||
push!(agent._state.messages, user_msg)
|
||||
agent.agentEventSink("_process_message 6")
|
||||
end
|
||||
|
||||
# call agent.prepareContext()
|
||||
@@ -208,10 +345,11 @@ function _process_message(agent::yiemAgent)::assistantMessage
|
||||
# Call agent.formatMsgForLLM(agent._state) to format for LLM
|
||||
formatted_messages = agent.formatMsgForLLM(preparedContext)
|
||||
|
||||
agent.agentEventSink("_process_message 7")
|
||||
# Call llmCall() (blocking — the task waits here)
|
||||
error("debug marker")
|
||||
response = agent.llmCall(formatted_messages)
|
||||
|
||||
error(5555555)
|
||||
agent.agentEventSink("_process_message 8")
|
||||
|
||||
#WORKING Check if LLM used tool calls (inspect content for tool_call blocks)
|
||||
has_tool_calls = false
|
||||
@@ -509,41 +647,42 @@ prepareToolCall(context, msg, tc, config, abortedSignal)
|
||||
```
|
||||
"""
|
||||
function prepareToolCall(
|
||||
context::agentContext,
|
||||
assistantMsg::assistantMessage,
|
||||
toolCall::agentToolCall,
|
||||
config::agentLoopConfig,
|
||||
signal::Union{Nothing, abortSignal},
|
||||
context::agentContext,
|
||||
assistantMsg::assistantMessage,
|
||||
toolCall::agentToolCall,
|
||||
config::agentLoopConfig,
|
||||
signal::Union{Nothing, abortSignal},
|
||||
)::Union{preparedToolCall,immediateOutcome}
|
||||
|
||||
tool = get(context.tools, toolCall.name, nothing)
|
||||
if tool === nothing
|
||||
return immediateOutcome(createErrorToolResult("Tool $toolCall.name not found"), true)
|
||||
tool = get(context.tools, toolCall.name, nothing)
|
||||
if tool === nothing
|
||||
return immediateOutcome(createErrorToolResult("Tool $toolCall.name not found"), true)
|
||||
end
|
||||
|
||||
try
|
||||
# 1. prepare arguments (tool-specific transform)
|
||||
prepared = prepareToolCallArguments(tool, toolCall)
|
||||
validatedArgs = validateToolArguments(tool, prepared)
|
||||
|
||||
# 2. beforeToolCall hook — can block
|
||||
if config.beforeToolCall !== nothing
|
||||
before = config.beforeToolCall(
|
||||
beforeToolCallContext(assistantMsg, toolCall, validatedArgs, context),
|
||||
signal
|
||||
)
|
||||
if signal !== nothing && signal.aborted
|
||||
return immediateOutcome(createErrorToolResult("Operation aborted"), true)
|
||||
end
|
||||
if before !== nothing && before.block
|
||||
return immediateOutcome(
|
||||
createErrorToolResult(get(before, :reason, "Tool execution was blocked")), true)
|
||||
end
|
||||
end
|
||||
|
||||
try
|
||||
# 1. prepare arguments (tool-specific transform)
|
||||
prepared = prepareToolCallArguments(tool, toolCall)
|
||||
validatedArgs = validateToolArguments(tool, prepared)
|
||||
|
||||
# 2. beforeToolCall hook — can block
|
||||
if config.beforeToolCall !== nothing
|
||||
before = config.beforeToolCall(
|
||||
assistantMsgCtx(assistantMsg, toolCall, validatedArgs, context), signal
|
||||
)
|
||||
if signal !== nothing && signal.aborted
|
||||
return immediateOutcome(createErrorToolResult("Operation aborted"), true)
|
||||
end
|
||||
if before !== nothing && before.block
|
||||
return immediateOutcome(
|
||||
createErrorToolResult(get(before, :reason, "Tool execution was blocked")), true)
|
||||
end
|
||||
end
|
||||
|
||||
return preparedToolCall(tool, toolCall, validatedArgs)
|
||||
catch err
|
||||
return immediateOutcome(createErrorToolResult(sprint(showerror, err)), true)
|
||||
end
|
||||
return preparedToolCall(tool, toolCall, validatedArgs)
|
||||
catch err
|
||||
return immediateOutcome(createErrorToolResult(sprint(showerror, err)), true)
|
||||
end
|
||||
end
|
||||
|
||||
# ── per-call execution ──────────────────────────────────────────
|
||||
@@ -681,61 +820,28 @@ function finalizeExecutedToolCall(
|
||||
signal::Union{Nothing,abortSignal},
|
||||
)::finalizedOutcome
|
||||
|
||||
result = executed.result
|
||||
isError = executed.isError
|
||||
result = executed.result
|
||||
isError = executed.isError
|
||||
|
||||
if config.afterToolCall !== nothing
|
||||
try
|
||||
after = config.afterToolCall(
|
||||
afterCtx(assistantMsg, prep.toolCall, prep.args, result, isError, context), signal
|
||||
)
|
||||
if after !== nothing
|
||||
result = merge(result, dict(:content=>get(after,:content,result.content),
|
||||
:details=>get(after,:details,result.details),
|
||||
:usage=>get(after,:usage,result.usage),
|
||||
:terminate=>get(after,:terminate,result.terminate)))
|
||||
isError = get(after, :isError, isError)
|
||||
end
|
||||
catch err
|
||||
result = createErrorToolResult(sprint(showerror, err))
|
||||
isError = true
|
||||
end
|
||||
if config.afterToolCall !== nothing
|
||||
try
|
||||
after = config.afterToolCall(
|
||||
afterToolCallContext(assistantMsg, prep.toolCall, prep.args, result, isError, context), signal
|
||||
)
|
||||
if after !== nothing
|
||||
result = merge(result, dict(:content=>get(after,:content,result.content),
|
||||
:details=>get(after,:details,result.details),
|
||||
:usage=>get(after,:usage,result.usage),
|
||||
:terminate=>get(after,:terminate,result.terminate)))
|
||||
isError = get(after, :isError, isError)
|
||||
end
|
||||
catch err
|
||||
result = createErrorToolResult(sprint(showerror, err))
|
||||
isError = true
|
||||
end
|
||||
end
|
||||
|
||||
return finalizedOutcome(prep.toolCall, result, isError)
|
||||
end
|
||||
|
||||
"""
|
||||
emitToolExecutionEnd(finalized, emit)
|
||||
|
||||
Emits the `toolExecutionEnd` event with the finalized outcome,
|
||||
signalling to listeners that the tool call has completed.
|
||||
|
||||
This event is part of the tool execution lifecycle:
|
||||
`toolExecutionStart` → (zero or more `toolExecutionUpdate` events) →
|
||||
`toolExecutionEnd`. Listeners (such as the TUI or logging systems)
|
||||
use this lifecycle to track individual tool calls. The event carries
|
||||
the final result so listeners have all the data they need without
|
||||
requiring external state lookups.
|
||||
|
||||
# Arguments
|
||||
- `finalized::finalizedOutcome`: The finalized outcome to report
|
||||
- `emit::Function`: Event emitter
|
||||
|
||||
# Notes
|
||||
- Part of a three-event lifecycle per tool call
|
||||
- Carries the complete result so listeners need no external lookups
|
||||
|
||||
# Examples
|
||||
```julia
|
||||
# Emits a single event; returns nothing
|
||||
emitToolExecutionEnd(finalized, emit)
|
||||
# (emit receives toolExecEndEvent("call_1", "search_wine", result, false))
|
||||
```
|
||||
"""
|
||||
function emitToolExecutionEnd(finalized::finalizedOutcome, emit::Function)
|
||||
emit(toolExecEndEvent(finalized.toolCall.id, finalized.toolCall.name,
|
||||
finalized.result, finalized.isError))
|
||||
return finalizedOutcome(prep.toolCall, result, isError)
|
||||
end
|
||||
|
||||
# ── sequential execution ────────────────────────────────────────
|
||||
@@ -816,7 +922,8 @@ function executeToolCallsSequential(
|
||||
finalized = finalizeExecutedToolCall(context, assistantMsg, prep, executed, config, signal)
|
||||
end
|
||||
|
||||
emitToolExecutionEnd(finalized, emit)
|
||||
emit(toolExecEndEvent(finalized.toolCall.id, finalized.toolCall.name,
|
||||
finalized.result, finalized.isError))
|
||||
push!(messages, createToolResultMessage(finalized))
|
||||
push!(finalizedCalls, finalized)
|
||||
|
||||
@@ -903,13 +1010,15 @@ function executeToolCallsParallel(
|
||||
|
||||
if prep isa immediateOutcome
|
||||
finalized = finalizedOutcome(tc, prep.result, prep.isError)
|
||||
emitToolExecutionEnd(finalized, emit)
|
||||
emit(toolExecEndEvent(finalized.toolCall.id, finalized.toolCall.name,
|
||||
finalized.result, finalized.isError))
|
||||
push!(entries, finalized)
|
||||
else
|
||||
task = task() do
|
||||
executed = executePreparedToolCall(prep, signal, emit)
|
||||
finalized = finalizeExecutedToolCall(context, assistantMsg, prep, executed, config, signal)
|
||||
emitToolExecutionEnd(finalized, emit)
|
||||
emit(toolExecEndEvent(finalized.toolCall.id, finalized.toolCall.name,
|
||||
finalized.result, finalized.isError))
|
||||
return finalized
|
||||
end
|
||||
schedule(task)
|
||||
|
||||
+14
-15
@@ -5,12 +5,11 @@ export prompt
|
||||
using JSON, DataStructures, Dates, UUIDs, HTTP, Random, PrettyPrinting, Serialization,
|
||||
DataFrames
|
||||
using GeneralUtils
|
||||
using ..type, ..utils
|
||||
using ..type, ..utils, ..agentCore, ..toolRegistry
|
||||
|
||||
# ---------------------------------------------- 100 --------------------------------------------- #
|
||||
|
||||
|
||||
|
||||
"""
|
||||
Send a message to the agent's input channel.
|
||||
|
||||
@@ -25,16 +24,16 @@ The agent processes messages from `inputChannel` in the background task.
|
||||
- The same `agent` instance for chaining
|
||||
|
||||
# Notes
|
||||
- Use `take_response(agent)` to receive the agent's response after sending a message.
|
||||
- Use `follow_up(agent, msg)` to send messages while the agent is still processing.
|
||||
- Use `takeResponse(agent)` to receive the agent's response after sending a message.
|
||||
- Use `followUp(agent, msg)` to send messages while the agent is still processing.
|
||||
|
||||
# Examples
|
||||
```jldoctest
|
||||
julia> run_agent(agent, "Hello!")
|
||||
julia> runAgent(agent, "Hello!")
|
||||
yiemAgent(...)
|
||||
```
|
||||
"""
|
||||
function run_agent(agent::yiemAgent, msg)
|
||||
function runAgent(agent::yiemAgent, msg)
|
||||
put!(agent.inputChannel, msg)
|
||||
return agent
|
||||
end
|
||||
@@ -51,15 +50,15 @@ Blocks until the agent sends a response.
|
||||
- An `assistantMessage` instance representing the agent's response
|
||||
|
||||
# Notes
|
||||
- Use `run_agent(agent, msg)` to send a message before calling this function.
|
||||
- Use `runAgent(agent, msg)` to send a message before calling this function.
|
||||
|
||||
# Examples
|
||||
```jldoctest
|
||||
julia> response = take_response(agent)
|
||||
julia> response = takeResponse(agent)
|
||||
assistantMessage(...)
|
||||
```
|
||||
"""
|
||||
function take_response(agent::yiemAgent)
|
||||
function takeResponse(agent::yiemAgent)
|
||||
return take!(agent.outputChannel)
|
||||
end
|
||||
|
||||
@@ -77,17 +76,17 @@ and before any tool call results are sent.
|
||||
- The same `agent` instance for chaining
|
||||
|
||||
# Notes
|
||||
- Use `run_agent(agent, msg)` for the primary message and `follow_up(agent, msg)` for additional
|
||||
- Use `runAgent(agent, msg)` for the primary message and `followUp(agent, msg)` for additional
|
||||
messages while the agent is processing.
|
||||
- Follow-up messages are buffered in a separate channel (capacity 32 by default).
|
||||
|
||||
# Examples
|
||||
```jldoctest
|
||||
julia> follow_up(agent, "Also consider red wines")
|
||||
julia> followUp(agent, "Also consider red wines")
|
||||
yiemAgent(...)
|
||||
```
|
||||
"""
|
||||
function follow_up(agent::yiemAgent, msg)
|
||||
function followUp(agent::yiemAgent, msg)
|
||||
put!(agent.followUpChannel, msg)
|
||||
return agent
|
||||
end
|
||||
@@ -105,16 +104,16 @@ then closes all channels (`inputChannel`, `outputChannel`, `followUpChannel`).
|
||||
- `nothing`
|
||||
|
||||
# Notes
|
||||
- After calling `stop_agent`, the agent is no longer usable. A new agent must be created
|
||||
- After calling `stopAgent`, the agent is no longer usable. A new agent must be created
|
||||
for further interaction.
|
||||
- If the background task throws a `TaskFailedException`, it is rethrown.
|
||||
|
||||
# Examples
|
||||
```jldoctest
|
||||
julia> stop_agent(agent)
|
||||
julia> stopAgent(agent)
|
||||
```
|
||||
"""
|
||||
function stop_agent(agent::yiemAgent)
|
||||
function stopAgent(agent::yiemAgent)
|
||||
put!(agent.inputChannel, :shutdown)
|
||||
try
|
||||
fetch(agent._agent_loop)
|
||||
|
||||
+26
-158
@@ -13,8 +13,8 @@
|
||||
# Context types
|
||||
agentContext, agentState, agentToolCall, prepareNextTurnContext,
|
||||
# Loop & execution types
|
||||
agentLoopConfig, abortSignal, agentToolResult,
|
||||
assistantMsgCtx, afterCtx,
|
||||
agentLoopConfig, abortSignal, agentToolResult,beforeToolCallContext,
|
||||
beforeToolCallResult, afterToolCallContext,
|
||||
# Event types
|
||||
toolExecStartEvent, toolExecUpdateEvent, toolExecEndEvent,
|
||||
# Agent
|
||||
@@ -23,7 +23,7 @@
|
||||
preparedToolCall, immediateOutcome, executedOutcome, finalizedOutcome,
|
||||
agentToolCallBatch,
|
||||
# Functions (defined elsewhere)
|
||||
run_agent, take_response, follow_up, stop_agent
|
||||
runAgent, takeResponse, followUp, stopAgent
|
||||
|
||||
|
||||
using Dates, UUIDs, DataStructures, JSON, NATS, Base.Threads
|
||||
@@ -144,7 +144,7 @@ assistantMessage("assistant", [textContent("Hello!")], "", "", "gpt-4", ..., "en
|
||||
```
|
||||
"""
|
||||
function assistantMessage(; role="assistant", content=Vector{messageContent}(),
|
||||
api="", provider="", model="", usage=llmUsage(0, 0), stopReason="end_turn",
|
||||
api="", provider="", model=nothing, usage=llmUsage(0, 0), stopReason="end_turn",
|
||||
errorMessage=nothing, timestamp=now())
|
||||
return assistantMessage(role, content, api, provider, model, usage, stopReason, errorMessage, timestamp)
|
||||
end
|
||||
@@ -309,7 +309,7 @@ end
|
||||
|
||||
mutable struct agentState # Mutable runtime state of an agent
|
||||
systemPrompt::String # System prompt for the agent
|
||||
model::llmModel # LLM model to use
|
||||
model::Union{llmModel, Nothing} # LLM model to use
|
||||
tools::OrderedDict{String, agentTool} # Available tools keyed by name, insertion-ordered
|
||||
|
||||
# messages history includes userMessage, assistantMessage, toolResultMessage. NO system prompt
|
||||
@@ -341,21 +341,21 @@ julia> state = agentState(systemPrompt="You are a helpful assistant")
|
||||
agentState("You are a helpful assistant", OrderedDict{String, agentTool}(), agentMessage[], String[], nothing)
|
||||
"""
|
||||
function agentState(
|
||||
systemPrompt::String="",
|
||||
model::llmModel=llmModel{String}("", "", "unknown", "unknown", "", false, String[],
|
||||
modelCost(0.0, 0.0, 0.0, 0.0), 0, 0),
|
||||
tools::OrderedDict{String, agentTool}=OrderedDict{String, agentTool}(),
|
||||
messages::Vector{agentMessage}=agentMessage[],
|
||||
systemPrompt::String="",
|
||||
model=llmModel("model_1", "unknown", "unknown", "", false, String[],
|
||||
modelCost(0.0, 0.0, 0.0, 0.0), 0, 0),
|
||||
tools::OrderedDict{String, agentTool}=OrderedDict{String, agentTool}(),
|
||||
messages::Vector{agentMessage}=agentMessage[],
|
||||
)
|
||||
agentState(
|
||||
systemPrompt,
|
||||
model,
|
||||
deepcopy(tools),
|
||||
deepcopy(messages),
|
||||
Vector{String}(),
|
||||
false,
|
||||
nothing,
|
||||
)
|
||||
agentState(
|
||||
systemPrompt,
|
||||
model,
|
||||
deepcopy(tools),
|
||||
deepcopy(messages),
|
||||
Vector{String}(),
|
||||
false,
|
||||
nothing,
|
||||
)
|
||||
end
|
||||
|
||||
|
||||
@@ -442,13 +442,18 @@ Context passed to the `beforeToolCall` hook.
|
||||
- `args::Dict{String,Any}`: Validated tool arguments
|
||||
- `context::agentContext`: Current conversation context
|
||||
"""
|
||||
struct assistantMsgCtx
|
||||
struct beforeToolCallContext
|
||||
message::assistantMessage
|
||||
toolCall::agentToolCall
|
||||
args::Dict{String,Any}
|
||||
context::agentContext
|
||||
end
|
||||
|
||||
struct beforeToolCallResult
|
||||
block::Bool
|
||||
reason::String
|
||||
end
|
||||
|
||||
"""
|
||||
Context passed to the `afterToolCall` hook.
|
||||
|
||||
@@ -460,7 +465,7 @@ Context passed to the `afterToolCall` hook.
|
||||
- `isError::Bool`: Whether execution resulted in an error
|
||||
- `context::agentContext`: Current conversation context
|
||||
"""
|
||||
struct afterCtx
|
||||
struct afterToolCallContext
|
||||
message::assistantMessage
|
||||
toolCall::agentToolCall
|
||||
args::Dict{String,Any}
|
||||
@@ -521,143 +526,6 @@ end
|
||||
|
||||
abstract type agent end
|
||||
|
||||
"""
|
||||
docstring
|
||||
"""
|
||||
mutable struct yiemAgent <: agent # High-level agent wrapper
|
||||
_state::agentState # Current state (prompt, model, messages, tools, etc.)
|
||||
|
||||
# user sends prompt message to agent. if agent is idle, it process user message right away.
|
||||
# if agent is running, it process user message after the current tool call finished.
|
||||
inputChannel::Channel
|
||||
|
||||
# Buffers messages the user sends while the agent is busy. Processed after all inputChannel
|
||||
# messages are handled and the agent is idle (not using a tool call).
|
||||
followUpChannel::Channel
|
||||
|
||||
# agent sends response message to user after processing all user messages in inputChannel
|
||||
# and all followUp messages.
|
||||
outputChannel::Channel
|
||||
|
||||
_agent_loop::Union{Task, Nothing} # agent loop running in the background
|
||||
|
||||
# Preprocess/transform messages and context (modify, filter, prune, inject context from memory,
|
||||
# reorder, ...) for a single LLM call in _process_message()'s loop.
|
||||
# returns new Vector{agentMessage}
|
||||
prepareContext ::Union{Function, Nothing}
|
||||
|
||||
# Convert prepareContext()'s new Vector{agentMessage} to LLM message format
|
||||
formatMsgForLLM::Function
|
||||
|
||||
# Actually invoke the LLM to get a completion response. The LLM response comes back as an
|
||||
# assistantMessage whose content is an array of content blocks.
|
||||
# Each block has a type — "text", "thinking", or "toolCall".
|
||||
# The code filters for type === "toolCall" blocks, then passes them to executeToolCalls().
|
||||
llmCall::Function
|
||||
|
||||
# Callback invoked before executing a tool call (ask for user permission/confirmation/abort, etc..)
|
||||
beforeToolCall::Union{Function, Nothing}
|
||||
|
||||
executeToolCalls::Function # execute tool calls ()
|
||||
|
||||
# Callback invoked after executing a tool call to sanitize tools output so the output is ready
|
||||
# to be converted into toolResults message
|
||||
afterToolCall::Union{Function, Nothing}
|
||||
# prepareNextTurn::Union{Function, Nothing} # Callback to prepare the next conversation turn
|
||||
# prepareNextTurnWithContext::Union{Function, Nothing} # Same but receives context
|
||||
sessionId::Union{String, Nothing} # Optional session identifier
|
||||
maxRetryDelayMs::Union{Int64, Nothing} # Maximum delay between retries (ms)
|
||||
parallelToolExecute::Bool # Default: false
|
||||
agentEventSink::Function # agent emits its status via this function
|
||||
_tool_store::Any # Reference to the toolStore for runtime registration
|
||||
end
|
||||
|
||||
"""
|
||||
Create a new yiemAgent instance with a background loop task.
|
||||
|
||||
Spawns a background `@spawn` task that runs the agent loop, listening
|
||||
on `inputChannel` and `followUpChannel` channels concurrently.
|
||||
|
||||
# Keyword Arguments
|
||||
- `systemPrompt::String`: System prompt for the agent
|
||||
- `model`: LLM model to use
|
||||
- `tools::OrderedDict{String, agentTool}`: Available tools keyed by name (default: empty)
|
||||
- `messages::Vector{agentMessage}`: Initial conversation messages (default: empty)
|
||||
- `formatMsgForLLM::Function`: Convert agent messages to LLM message format (default: `defaultformatMsgForLLM`)
|
||||
- `llmCall::Function`: Function to invoke the LLM (required)
|
||||
- `prepareContext::Union{Function, Nothing}`: Preprocess/transform messages before sending to LLM (default: `nothing`)
|
||||
- `beforeToolCall::Union{Function, Nothing}`: Callback invoked before executing a tool call (default: `nothing`)
|
||||
- `afterToolCall::Union{Function, Nothing}`: Callback invoked after executing a tool call (default: `nothing`)
|
||||
- `prepareNextTurn::Union{Function, Nothing}`: Callback to prepare the next conversation turn (default: `nothing`)
|
||||
- `prepareNextTurnWithContext::Union{Function, Nothing}`: Same but receives context (default: `nothing`)
|
||||
- `sessionId::Union{String, Nothing}`: Optional session identifier (default: `nothing`)
|
||||
- `maxRetryDelayMs::Union{Int64, Nothing}`: Maximum delay between retries in milliseconds (default: `nothing`)
|
||||
- `parallelToolExecute::Bool`: Run tool calls in parallel (default: `false`)
|
||||
- `agentEventSink::Function`: Callback to receive agent events
|
||||
- `tool_store::Union{Any, Nothing}`: toolStore for runtime tool registration (default: `nothing`)
|
||||
|
||||
# Returns
|
||||
- A new `yiemAgent` instance with an active background task
|
||||
|
||||
# Examples
|
||||
```julia
|
||||
julia> store = toolStore(name="agent1")
|
||||
julia> tools = loadTools(store, "src/tools")
|
||||
julia> agent = yiemAgent(systemPrompt="You are a helpful assistant", model=my_model, tools=tools, llmCall=..., tool_store=store)
|
||||
yiemAgent(agentState(...), Channel(...), Channel(...), Channel(...), ..., store)
|
||||
"""
|
||||
function yiemAgent(
|
||||
; systemPrompt::String="You are helpful assistant.",
|
||||
model=nothing,
|
||||
tools::OrderedDict{String, agentTool}=OrderedDict{String, agentTool}(),
|
||||
messages::Vector{agentMessage}=agentMessage[],
|
||||
prepareContext::Union{Function, Nothing}=nothing,
|
||||
formatMsgForLLM::Function=defaultformatMsgForLLM,
|
||||
llmCall::Function,
|
||||
beforeToolCall::Union{Function, Nothing}=nothing,
|
||||
afterToolCall::Union{Function, Nothing}=nothing,
|
||||
# prepareNextTurn::Union{Function, Nothing}=nothing,
|
||||
# prepareNextTurnWithContext::Union{Function, Nothing}=nothing,
|
||||
sessionId::Union{String, Nothing}=nothing,
|
||||
maxRetryDelayMs::Union{Int64, Nothing}=nothing,
|
||||
parallelToolExecute::Bool=false,
|
||||
agentEventSink::Function,
|
||||
tool_store::Union{Any, Nothing}=nothing,
|
||||
)
|
||||
# Create channels: input (user -> agent), followUp (async queue), output (agent -> user)
|
||||
inputChannel = Channel(16)
|
||||
followUp = Channel(32)
|
||||
outputChannel = Channel(16)
|
||||
|
||||
# Create struct with a placeholder task, then spawn and replace it
|
||||
agent = yiemAgent(
|
||||
agentState(systemPrompt, model, tools, messages),
|
||||
inputChannel,
|
||||
followUp,
|
||||
outputChannel,
|
||||
nothing, # placeholder — replaced below
|
||||
prepareContext,
|
||||
formatMsgForLLM,
|
||||
llmCall,
|
||||
beforeToolCall,
|
||||
afterToolCall,
|
||||
# prepareNextTurn,
|
||||
# prepareNextTurnWithContext,
|
||||
sessionId,
|
||||
maxRetryDelayMs,
|
||||
parallelToolExecute,
|
||||
agentEventSink,
|
||||
tool_store,
|
||||
)
|
||||
|
||||
# Spawn the background loop and attach it
|
||||
agent._agent_loop = @spawn _agent_loop(agent)
|
||||
|
||||
return agent
|
||||
end
|
||||
|
||||
|
||||
|
||||
"""
|
||||
preparedToolCall(tool, toolCall, args)
|
||||
|
||||
|
||||
+32
-2
@@ -1,7 +1,9 @@
|
||||
module utils
|
||||
|
||||
export clearhistory, availableWineToText, prepareContext, formatMsgForLLM, validateRequiredArgs, validateToolArguments, _userMessageToOpenAI,
|
||||
_assistantMessageToOpenAI, _toolResultMessageToOpenAI, _messageContentToBlocks
|
||||
export clearhistory, availableWineToText, prepareContext, formatMsgForLLM, validateRequiredArgs,
|
||||
validateToolArguments, _userMessageToOpenAI,
|
||||
_assistantMessageToOpenAI, _toolResultMessageToOpenAI, _messageContentToBlocks,
|
||||
beforeToolCall, afterToolCall, agentEventSink
|
||||
|
||||
using UUIDs, Dates, DataStructures, HTTP, JSON
|
||||
using GeneralUtils
|
||||
@@ -218,6 +220,34 @@ function formatMsgForLLM(ctx::agentContext)::Dict{String, Any}
|
||||
return Dict("messages" => messages)
|
||||
end
|
||||
|
||||
#TODO
|
||||
function beforeToolCall(context::beforeToolCallContext, signal::abortSignal
|
||||
)::beforeToolCallResult
|
||||
|
||||
# final context check
|
||||
|
||||
# seek user approval via UI
|
||||
|
||||
# other check
|
||||
|
||||
return beforeToolCallResult(false, "N/A")
|
||||
end
|
||||
|
||||
#TODO
|
||||
function afterToolCall(context::beforeToolCallContext, signal::abortSignal
|
||||
)::Union{agentToolResult, Nothing}
|
||||
|
||||
# modify context.result if needed and return agentToolResult
|
||||
|
||||
return nothing
|
||||
end
|
||||
|
||||
|
||||
#TODO
|
||||
function agentEventSink(x)
|
||||
|
||||
end
|
||||
|
||||
|
||||
"""
|
||||
Convert a userMessage to OpenAI message format.
|
||||
|
||||
-401
@@ -1,401 +0,0 @@
|
||||
using JSON, Dates, UUIDs, PrettyPrinting, LibPQ, Base64, DataFrames, DataStructures, HTTP, Base64,
|
||||
NATS, Base.Threads
|
||||
using YiemAgent, GeneralUtils, msghandler
|
||||
|
||||
function text2text_instruct_llm(sender_id::String, openai_msg::Dict{String, Any})
|
||||
payloads = [("msg", openai_msg, "dictionary")] # List of tuples
|
||||
_, msg_envelope_json_str = msghandler.smartpack(
|
||||
config["externalservice"]["servicesloadbalancer"]["nats"],
|
||||
payloads;
|
||||
sender_id=sender_id,
|
||||
msg_purpose="text2text",
|
||||
broker_url=config["nats_server_info"]["url"],
|
||||
fileserver_url=config["externalservice"]["fileserver"]["url"])
|
||||
|
||||
reply = NATS.request(agent_conn,
|
||||
config["externalservice"]["servicesloadbalancer"]["nats"],
|
||||
msg_envelope_json_str, timeout=120)
|
||||
|
||||
incoming_env_json_str = String(reply.payload)
|
||||
incoming_env = msghandler.smartunpack(incoming_env_json_str)
|
||||
_llm_response = incoming_env["payloads"][1][2]
|
||||
llm_response = _llm_response["choices"][1]["message"]["content"]
|
||||
return llm_response
|
||||
end
|
||||
|
||||
""" get a single text embedding from a LLM service
|
||||
Example
|
||||
text = ["hello"]
|
||||
embedding = get_embedding(text)
|
||||
"""
|
||||
function get_embedding(text::AbstractArray{String})
|
||||
documents_dict = Dict("documents" => text)
|
||||
payloads = [("documents", documents_dict, "dictionary")]
|
||||
_, msg_envelope_json_str = msghandler.smartpack(
|
||||
config["externalservice"]["servicesloadbalancer"]["nats"],
|
||||
payloads;
|
||||
msg_purpose="embedding",
|
||||
broker_url=config["nats_server_info"]["url"],
|
||||
fileserver_url=config["externalservice"]["fileserver"]["url"])
|
||||
|
||||
reply = NATS.request(agent_conn,
|
||||
config["externalservice"]["servicesloadbalancer"]["nats"],
|
||||
msg_envelope_json_str, timeout=120)
|
||||
incoming_env_json_str = String(reply.payload)
|
||||
incoming_env = msghandler.smartunpack(incoming_env_json_str)
|
||||
embedding_response = incoming_env["payloads"][1][2]
|
||||
|
||||
return embedding_response
|
||||
end
|
||||
|
||||
""" sql = "SELECT * FROM wine;"
|
||||
result = execute_sql_winedb(sql)
|
||||
"""
|
||||
function execute_sql_winedb(sql::T) where {T<:AbstractString}
|
||||
host_url, _port = split(config["externalservice"]["sommpanion_db"]["url"], ':')
|
||||
port = parse(Int, _port)
|
||||
dbname = "winedb"
|
||||
user = config["externalservice"]["sommpanion_db"]["user"]
|
||||
password = config["externalservice"]["sommpanion_db"]["password"]
|
||||
db_connection = LibPQ.Connection("host=$host_url port=$port dbname=$dbname user=$user password=$password")
|
||||
result = nothing
|
||||
try
|
||||
result = LibPQ.execute(db_connection, sql)
|
||||
catch e
|
||||
LibPQ.close(db_connection)
|
||||
end
|
||||
|
||||
LibPQ.close(db_connection)
|
||||
return result
|
||||
end
|
||||
|
||||
""" find similar sql from vector database
|
||||
sql = "SELECT * FROM wine;"
|
||||
result, distance = similar_sql_vectordb(sql)
|
||||
"""
|
||||
function similar_sql_vectordb(sql::T; maxdistance::Number=0.2) where {T<:AbstractString}
|
||||
tablename = "sqlllm_decision_repository"
|
||||
# get embedding of the query
|
||||
df = find_similar_text_from_vectordb(sql, tablename,
|
||||
"function_input_embedding", execute_sql_vectordb)
|
||||
# println(df[1, [:id, :function_output]])
|
||||
row, col = size(df)
|
||||
distance = row == 0 ? Inf : df[1, :distance]
|
||||
if row != 0 && distance < maxdistance
|
||||
# if there is usable SQL, return it.
|
||||
output_b64 = df[1, :function_output_base64] # pick the closest match
|
||||
output_str = String(base64decode(output_b64))
|
||||
rowid = df[1, :id]
|
||||
println("\n--| similar sql found. row id $rowid, distance $distance ", @__FILE__, ":", @__LINE__, " $(Dates.now())")
|
||||
pprintln(output_str)
|
||||
return (result=output_str, distance=distance)
|
||||
else
|
||||
println("\n--| similar sql not found, max distance $maxdistance ", @__FILE__, ":", @__LINE__, " $(Dates.now())")
|
||||
return (result=nothing, distance=nothing)
|
||||
end
|
||||
end
|
||||
|
||||
""" insert query and sql into vector database
|
||||
query = "get all wines from wine table"
|
||||
sql = "SELECT * FROM wine;"
|
||||
insert_sql_vectordb(query, sql)
|
||||
"""
|
||||
function insert_sql_vectordb(query::T1, SQL::T2; maxdistance::Number=3
|
||||
) where {T1<:AbstractString, T2<:AbstractString}
|
||||
|
||||
tablename = "sqlllm_decision_repository"
|
||||
# get embedding of the query
|
||||
# query = state[:thoughtHistory][:question]
|
||||
df = find_similar_text_from_vectordb(query, tablename,
|
||||
"function_input_embedding", execute_sql_vectordb)
|
||||
row, col = size(df)
|
||||
distance = row == 0 ? Inf : df[1, :distance]
|
||||
if row == 0 || distance > maxdistance # no close enough SQL stored in the database
|
||||
_query_embedding = get_embedding([query])
|
||||
_query_embedding = GeneralUtils.dictify(_query_embedding)
|
||||
# println("\n--- _query_embedding() ", @__FILE__, ":", @__LINE__, " $(Dates.now())")
|
||||
# println(_query_embedding)
|
||||
# println("---\n")
|
||||
query_embedding = _query_embedding["data"][1]["embedding"]
|
||||
query = replace(query, "'" => "")
|
||||
sql_base64 = base64encode(SQL)
|
||||
sql_ = replace(SQL, "'" => "")
|
||||
|
||||
sql =
|
||||
"""
|
||||
INSERT INTO $tablename (function_input, function_output, function_output_base64, function_input_embedding) VALUES ('$query', '$sql_', '$sql_base64', '$query_embedding');
|
||||
"""
|
||||
# println("\n--| added new decision to vectorDB ", @__FILE__, ":", @__LINE__, " $(Dates.now())")
|
||||
# println(sql)
|
||||
_ = execute_sql_vectordb(sql)
|
||||
end
|
||||
end
|
||||
|
||||
""" execute sql against vectordb
|
||||
sql = "SELECT * FROM wine;"
|
||||
result = execute_sql_vectordb(sql)
|
||||
"""
|
||||
function execute_sql_vectordb(sql::T) where {T<:AbstractString}
|
||||
host_url, _port = split(config["externalservice"]["sommpanion_vectordb"]["url"], ':')
|
||||
port = parse(Int, _port)
|
||||
dbname = config["externalservice"]["sommpanion_vectordb"]["dbname"]
|
||||
user = config["externalservice"]["sommpanion_vectordb"]["user"]
|
||||
password = config["externalservice"]["sommpanion_vectordb"]["password"]
|
||||
DBconnection = LibPQ.Connection("host=$host_url port=$port dbname=$dbname user=$user password=$password")
|
||||
result = LibPQ.execute(DBconnection, sql)
|
||||
close(DBconnection)
|
||||
return result
|
||||
end
|
||||
|
||||
""" search similar decision llm made from vectordb
|
||||
"""
|
||||
function similar_sommelier_decision(recentevents::T1; maxdistance::Integer=3
|
||||
)::Union{AbstractDict, Nothing} where {T1<:AbstractString}
|
||||
|
||||
tablename = "sommelier_decision_repository"
|
||||
# find similar
|
||||
df = find_similar_text_from_vectordb(recentevents, tablename,
|
||||
"function_input_embedding", execute_sql_vectordb)
|
||||
row, col = size(df)
|
||||
distance = row == 0 ? Inf : df[1, :distance]
|
||||
if row != 0 && distance < maxdistance
|
||||
# if there is usable decision, return it.
|
||||
rowid = df[1, :id]
|
||||
println("\n--| found similar decision. row id $rowid, distance $distance ", @__FILE__, " ", @__LINE__)
|
||||
output_b64 = df[1, :function_output_base64] # pick the closest match
|
||||
_output_str = String(base64decode(output_b64))
|
||||
output = copy(JSON.read(_output_str))
|
||||
return output
|
||||
else
|
||||
println("\n--| similar decision not found, max distance $maxdistance ", @__FILE__, " ", @__LINE__)
|
||||
return nothing
|
||||
end
|
||||
end
|
||||
|
||||
""" search similar text from vectordb
|
||||
"""
|
||||
function find_similar_text_from_vectordb(text::T1, tablename::T2, embeddingColumnName::T3,
|
||||
vectorDB::Function; limit::Integer=1
|
||||
)::DataFrame where {T1<:AbstractString, T2<:AbstractString, T3<:AbstractString}
|
||||
# get embedding from LLM service
|
||||
_embedding = get_embedding([text])
|
||||
_embedding = _embedding["data"][1]["embedding"]
|
||||
_embedding = "$_embedding"
|
||||
|
||||
embedding = _embedding[4:end]
|
||||
|
||||
# check whether there is close enough vector already store in vectorDB. if no, add, else skip
|
||||
sql = """
|
||||
SELECT *, $embeddingColumnName <-> '$embedding' as distance
|
||||
FROM $tablename
|
||||
ORDER BY distance LIMIT $limit;
|
||||
"""
|
||||
response = vectorDB(sql)
|
||||
df = DataFrame(response)
|
||||
|
||||
return df
|
||||
end
|
||||
|
||||
""" insert decision llm made to vectordb
|
||||
"""
|
||||
function insert_sommelier_decision(recentevents::T1, decision::T2; maxdistance::Integer=5
|
||||
) where {T1<:AbstractString, T2<:AbstractDict}
|
||||
tablename = "sommelier_decision_repository"
|
||||
# find similar
|
||||
df = find_similar_text_from_vectordb(recentevents, tablename,
|
||||
"function_input_embedding", execute_sql_vectordb)
|
||||
row, col = size(df)
|
||||
distance = row == 0 ? Inf : df[1, :distance]
|
||||
if row == 0 || distance > maxdistance # no close enough SQL stored in the database
|
||||
_embedding = get_embedding([recentevents])[1]
|
||||
recentevents_embedding = _embedding["data"][1]["embedding"]
|
||||
recentevents = replace(recentevents, "'" => "")
|
||||
decision_json = JSON.json(decision)
|
||||
decision_base64 = base64encode(decision_json)
|
||||
decision = replace(decision_json, "'" => "")
|
||||
|
||||
sql =
|
||||
"""
|
||||
INSERT INTO $tablename (function_input, function_output, function_output_base64, function_input_embedding) VALUES ('$recentevents', '$decision', '$decision_base64', '$recentevents_embedding');
|
||||
"""
|
||||
println("\n--| added new decision to vectorDB ", @__FILE__, " ", @__LINE__)
|
||||
println(sql)
|
||||
_ = execute_sql_vectordb(sql)
|
||||
else
|
||||
println("--| similar decision previously cached, distance $distance ", @__FILE__, " ", @__LINE__)
|
||||
end
|
||||
end
|
||||
|
||||
config = JSON.parsefile("./appconfig.json")
|
||||
sessionId = "0"
|
||||
backend_session_topic = "sommpanion.testsubject"
|
||||
agent_ch = Channel(8)
|
||||
agent_conn = NATS.connect(config["nats_server_info"]["url"])
|
||||
|
||||
sub2 = NATS.subscribe(agent_conn, backend_session_topic) do msg
|
||||
put!(agent_ch, msg)
|
||||
end
|
||||
|
||||
agent_context = YiemAgent.agentcontext(
|
||||
text2text_instruct_llm,
|
||||
get_embedding,
|
||||
execute_sql_winedb,
|
||||
similar_sql_vectordb,
|
||||
insert_sql_vectordb,
|
||||
similar_sommelier_decision,
|
||||
insert_sommelier_decision
|
||||
)
|
||||
|
||||
# can't instantiate
|
||||
agent = YiemAgent.sommelier(
|
||||
agent_context;
|
||||
name="Janie",
|
||||
id=sessionId, # agent instance id
|
||||
retailername="Yiem Wine Ltd.",
|
||||
llmFormatName=""
|
||||
)
|
||||
|
||||
|
||||
image1_path = "test/large_image.png"
|
||||
image1_bytes = read(image1_path)
|
||||
image1_base64_string = base64encode(image1_bytes)
|
||||
mime_type = "image/png"
|
||||
data1_uri = "data:$(mime_type);base64,$(image1_base64_string)"
|
||||
|
||||
# 1. Read local file and encode to base64 string
|
||||
image2_path = "test/small_image.png"
|
||||
image2_bytes = read(image2_path)
|
||||
image2_base64_string = base64encode(image2_bytes)
|
||||
mime_type = "image/png"
|
||||
data2_uri = "data:$(mime_type);base64,$(image2_base64_string)"
|
||||
|
||||
# 3. Construct payload with the Data URI
|
||||
message = Dict(
|
||||
"role" => "user",
|
||||
"content" => [
|
||||
Dict("type" => "text", "text" => "Do you know type of wine in the image?"),
|
||||
Dict(
|
||||
"type" => "image_url",
|
||||
"image_url" => Dict("url" => data1_uri)
|
||||
)
|
||||
]
|
||||
)
|
||||
|
||||
result = YiemAgent.conversation(agent; userinput=message)
|
||||
println("\n$result")
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
# message = Dict(
|
||||
# "role" => "user",
|
||||
# "content" => [
|
||||
# Dict("type" => "text", "text" =>
|
||||
# "
|
||||
# เป็นงานเลี้ยงทั่วไป
|
||||
# "),
|
||||
# ]
|
||||
# )
|
||||
|
||||
# result = YiemAgent.conversation(agent; userinput=message)
|
||||
# println("\n$result")
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
# message = Dict(
|
||||
# "role" => "user",
|
||||
# "content" => [
|
||||
# Dict("type" => "text", "text" => "no thanks. that's all"),
|
||||
# ]
|
||||
# )
|
||||
|
||||
# result = YiemAgent.conversation(agent; userinput=message)
|
||||
# println("\n$result")
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
# message = Dict(
|
||||
# "role" => "user",
|
||||
# "content" => [
|
||||
# Dict("type" => "text", "text" => "What about this wine?"),
|
||||
# Dict(
|
||||
# "type" => "image_url",
|
||||
# "image_url" => Dict("url" => data2_uri)
|
||||
# )
|
||||
# ]
|
||||
# )
|
||||
|
||||
# result = YiemAgent.conversation(agent; userinput=message)
|
||||
# println("\n$result")
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user