Compare commits

..

11 Commits

Author SHA1 Message Date
ton 4e592173a6 update 2026-08-12 14:37:11 +07:00
ton 0cacb5c94a update 2026-08-12 04:35:34 +07:00
ton 6c96409969 update 2026-08-12 04:33:19 +07:00
ton 06d51c1ee9 update 2026-08-12 04:00:09 +07:00
ton 2ad3d1df38 update 2026-08-11 19:10:37 +07:00
ton 83c7770877 update 2026-08-11 18:57:53 +07:00
ton bad14fbe7f update 2026-08-11 18:42:34 +07:00
ton 578e8f55bd update 2026-08-11 18:28:03 +07:00
ton ae3e432b02 update 2026-08-11 17:35:56 +07:00
ton 7c14390400 update 2026-08-11 17:28:25 +07:00
ton 89885c1583 update 2026-08-11 16:43:48 +07:00
8 changed files with 369 additions and 689 deletions
+2 -2
View File
@@ -6,7 +6,7 @@ Julia framework for building agents with tool use.
1. Install dependencies: `]add JSON, DataStructures, UUIDs, Dates, ...`
2. Create a `yiemAgent` with `loadTools("src/tools")`
3. Call `run_agent(agent, "message")` then `take_response(agent)`
3. Call `runAgent(agent, "message")` then `takeResponse(agent)`
## Architecture
@@ -16,7 +16,7 @@ src/
├── type.jl # Core types (messages, tools, agent state)
├── utils.jl # Message formatting, validation
├── agentCore.jl # Agent loop, tool execution pipeline
├── api.jl # Public API (run_agent, take_response, etc.)
├── api.jl # Public API (runAgent, takeResponse, etc.)
└── tools/
├── registry.jl # Tool registry (loadTools, registerTool, listTools)
├── getWeather.jl # Weather lookup tool
+15 -15
View File
@@ -151,7 +151,7 @@ result = getTime_tool.execute("call-1", Dict("city" => "Tokyo"), sig, op)
**Via agent loop (production):**
```
user message → run_agent(agent, Dict("role"=>"user", "content"=>...))
user message → runAgent(agent, Dict("role"=>"user", "content"=>...))
→ _agent_loop detects message → @spawn _process_message(agent)
→ prepareContext → formatMsgForLLM → llmCall
→ LLM returns tool_calls
@@ -382,9 +382,9 @@ The `_agent_loop()` function runs as a background `@spawn` task, created when `y
```
yiemAgent struct contains:
- inputChannel (Channel, capacity 16) ← user sends messages here via run_agent()
- followUpChannel (Channel, capacity 32) ← user sends follow-ups here via follow_up()
- outputChannel (Channel, capacity 16) ← agent sends responses here via take_response()
- inputChannel (Channel, capacity 16) ← user sends messages here via runAgent()
- followUpChannel (Channel, capacity 32) ← user sends follow-ups here via followUp()
- outputChannel (Channel, capacity 16) ← agent sends responses here via takeResponse()
- _tool_store (toolStore) ← per-agent isolated tool registry
```
@@ -633,7 +633,7 @@ function prepareToolCall(
- On failure: throws `ArgumentError(error_string)`, caught by the try-catch below
4. **Run `beforeToolCall` hook** — if `config.beforeToolCall !== nothing`
- Passes `assistantMsgCtx(assistantMsg, toolCall, validatedArgs, context)` and `signal`
- Passes `beforeToolCallContext(assistantMsg, toolCall, validatedArgs, context)` and `signal`
- Hook can return `nothing` (proceed), or `Dict(:block => true, :reason => "...")` (reject)
- If `signal.aborted == true``immediateOutcome(createErrorToolResult("Operation aborted"), true)`
- If `before.block == true``immediateOutcome(createErrorToolResult(get(before, :reason, "blocked")), true)`
@@ -722,10 +722,10 @@ function finalizeExecutedToolCall(
```
2. **Run `afterToolCall` hook** — if `config.afterToolCall !== nothing`:
- Passes `afterCtx(assistantMsg, prep.toolCall, prep.args, result, isError, context)` and `signal`
- Passes `afterToolCallContext(assistantMsg, prep.toolCall, prep.args, result, isError, context)` and `signal`
- Hook can mutate the result:
```julia
after = config.afterToolCall(afterCtx(...))
after = config.afterToolCall(afterToolCallContext(...))
if after !== nothing
result = merge(result, dict(
:content => get(after, :content, result.content),
@@ -1146,8 +1146,8 @@ The `agentEventSink` function is a user-provided callback that receives all even
| `prepareContext` | `(state::agentState) -> agentContext` | Before each LLM call | Filter tools, inject context, modify system prompt |
| `formatMsgForLLM` | `(ctx::agentContext) -> Dict` | After `prepareContext` | Convert to LLM-specific format |
| `llmCall` | `(messages::Dict) -> assistantMessage` | After formatting | Actually invoke the LLM API |
| `beforeToolCall` | `(msgCtx::assistantMsgCtx, signal) -> Union{Nothing, Dict}` | In `prepareToolCall` | Ask for user permission, block execution, abort |
| `afterToolCall` | `(afterCtx::afterCtx, signal) -> Union{Nothing, Dict}` | In `finalizeExecutedToolCall` | Mutate result, mask data, flip `terminate` |
| `beforeToolCall` | `(msgCtx::beforeToolCallContext, signal) -> Union{Nothing, Dict}` | In `prepareToolCall` | Ask for user permission, block execution, abort |
| `afterToolCall` | `(afterToolCallContext::afterToolCallContext, signal) -> Union{Nothing, Dict}` | In `finalizeExecutedToolCall` | Mutate result, mask data, flip `terminate` |
| `agentEventSink` | `(event) -> nothing` | Throughout lifecycle | Emit events for TUI, logging, monitoring |
### `beforeToolCall` Hook
@@ -1157,7 +1157,7 @@ The `agentEventSink` function is a user-provided callback that receives all even
```julia
if config.beforeToolCall !== nothing
before = config.beforeToolCall(
assistantMsgCtx(assistantMsg, toolCall, validatedArgs, context), signal
beforeToolCallContext(assistantMsg, toolCall, validatedArgs, context), signal
)
if signal !== nothing && signal.aborted
return immediateOutcome(createErrorToolResult("Operation aborted"), true)
@@ -1182,7 +1182,7 @@ end
if config.afterToolCall !== nothing
try
after = config.afterToolCall(
afterCtx(assistantMsg, prep.toolCall, prep.args, result, isError, context), signal
afterToolCallContext(assistantMsg, prep.toolCall, prep.args, result, isError, context), signal
)
if after !== nothing
result = merge(result, dict(
@@ -1327,7 +1327,7 @@ Each `toolStore` gets its own `listTool` instance bound to that store via `listT
```
USER SENDS MESSAGE
└─> run_agent(agent, "What's the weather in Tokyo?")
└─> runAgent(agent, "What's the weather in Tokyo?")
└─> put!(agent.inputChannel, Dict("role" => "user", "content" => [...]))
@@ -1430,7 +1430,7 @@ LOOP ITERATION 2 — LLM RETURNS FINAL TEXT RESPONSE
AGENT LOOP: SEND RESPONSE TO USER
└─> put!(agent.outputChannel, final_response)
└─> take_response(agent) → assistantMessage("The weather in Tokyo is sunny, 22°C.")
└─> takeResponse(agent) → assistantMessage("The weather in Tokyo is sunny, 22°C.")
```
---
@@ -1606,8 +1606,8 @@ All functions in the file are scoped under `_tool_myTool`, preventing name colli
| `agentState` | `type.jl:310` | Mutable runtime state (systemPrompt, model, tools, messages, pendingToolCalls, activeRun, errorMessage) |
| `agentLoopConfig` | `type.jl:403` | Loop config (tools, beforeToolCall, afterToolCall, toolExecution) |
| `abortSignal` | `type.jl:416` | Abort flag (`aborted::Bool`) |
| `assistantMsgCtx` | `type.jl:445` | Context for beforeToolCall (message, toolCall, args, context) |
| `afterCtx` | `type.jl:463` | Context for afterToolCall (message, toolCall, args, result, isError, context) |
| `beforeToolCallContext` | `type.jl:445` | Context for beforeToolCall (message, toolCall, args, context) |
| `afterToolCallContext` | `type.jl:463` | Context for afterToolCall (message, toolCall, args, result, isError, context) |
### Event Types
+77 -2
View File
@@ -1,2 +1,77 @@
# ── executeToolCalls() Julia pseudo code ──────────────────────────
# Full call stack from runLoop → executeToolCalls → prepare → execute → finalize → emit
i am not sure that's the case. see my NATS message log:
<NATS debug message>
Info: debug
payload = "new user msg"
Info: debug
payload = "new user msg"
Info: debug
payload = "new user msg"
Info: debug
payload = "new user msg"
Info: debug
payload = "new user msg"
Info: debug
payload = "new user msg"
Info: debug
payload = "new user msg"
Info: debug
payload = "_process_message 3"
Info: debug
payload = "_process_message 5"
Info: debug
payload = "_process_message 6"
Info: debug
payload = "_process_message 7"
</NATS debug message>
my NATS receiver report the following for a long time
Info: debug
payload = "new user msg"
untill I Ctrl + d so shutdown the process then i got the following report
Info: debug
payload = "_process_message 3"
Info: debug
payload = "_process_message 5"
Info: debug
payload = "_process_message 6"
Info: debug
payload = "_process_message 7"
my point is if _process_message() actually run then this code in _process_message()
"raw_msg = take!(agent.inputChannel)"
should take the new msg message out of agent.inputChannel and there should be only one debug message showing
Info: debug
payload = "new user msg"
before reaching error("debug marker")
+203 -94
View File
@@ -1,14 +1,142 @@
module agentCore
export _agent_loop, OpenAiToUserMessage
export yiemAgent, _agent_loop, OpenAiToUserMessage
using JSON, DataStructures, Dates, UUIDs, HTTP, Random, PrettyPrinting, Serialization,
DataFrames, Base.Threads
using GeneralUtils
using ..type, ..utils
using ..type, ..utils, ..toolRegistry
# ---------------------------------------------- 100 --------------------------------------------- #
"""
docstring
"""
mutable struct yiemAgent <: agent # High-level agent wrapper
_state::agentState # Current state (prompt, model, messages, tools, etc.)
# user sends prompt message to agent. if agent is idle, it process user message right away.
# if agent is running, it process user message after the current tool call finished.
inputChannel::Channel
# Buffers messages the user sends while the agent is busy. Processed after all inputChannel
# messages are handled and the agent is idle (not using a tool call).
followUpChannel::Channel
# agent sends response message to user after processing all user messages in inputChannel
# and all followUp messages.
outputChannel::Channel
_agent_loop::Union{Task, Nothing} # agent loop running in the background
# Preprocess/transform messages and context (modify, filter, prune, inject context from memory,
# reorder, ...) for a single LLM call in _process_message()'s loop.
# returns new Vector{agentMessage}
prepareContext::Union{Function, Nothing}
# Convert prepareContext()'s new Vector{agentMessage} to LLM message format
formatMsgForLLM::Function
# A callable struct. Actually invoke the LLM to get a completion response.
# The LLM response comes back as an assistantMessage whose content is an array of content blocks.
# Each block has a type — "text", "thinking", or "toolCall".
# The code filters for type === "toolCall" blocks, then passes them to executeToolCalls().
llmCall
# Callback invoked before executing a tool call (ask for user permission/confirmation/abort, etc..)
beforeToolCall::Union{Function, Nothing}
# Callback invoked after executing a tool call to sanitize tools output so the output is ready
# to be converted into toolResults message
afterToolCall::Union{Function, Nothing}
# prepareNextTurn::Union{Function, Nothing} # Callback to prepare the next conversation turn
# prepareNextTurnWithContext::Union{Function, Nothing} # Same but receives context
sessionId::Union{String, Nothing} # Optional session identifier
maxRetryDelayMs::Union{Int64, Nothing} # Maximum delay between retries (ms)
parallelToolExecute::Bool # Default: false
agentEventSink # agent emits its status via this function
end
"""
Create a new yiemAgent instance with a background loop task.
Spawns a background `@spawn` task that runs the agent loop, listening
on `inputChannel` and `followUpChannel` channels concurrently.
# Keyword Arguments
- `systemPrompt::String`: System prompt for the agent
- `model`: LLM model to use
- `tools::OrderedDict{String, agentTool}`: Available tools keyed by name (default: empty)
- `messages::Vector{agentMessage}`: Initial conversation messages (default: empty)
- `formatMsgForLLM::Function`: Convert agent messages to LLM message format (default: `defaultformatMsgForLLM`)
- `llmCall::Function`: Function to invoke the LLM (required)
- `prepareContext::Union{Function, Nothing}`: Preprocess/transform messages before sending to LLM (default: `nothing`)
- `beforeToolCall::Union{Function, Nothing}`: Callback invoked before executing a tool call (default: `nothing`)
- `afterToolCall::Union{Function, Nothing}`: Callback invoked after executing a tool call (default: `nothing`)
- `prepareNextTurn::Union{Function, Nothing}`: Callback to prepare the next conversation turn (default: `nothing`)
- `prepareNextTurnWithContext::Union{Function, Nothing}`: Same but receives context (default: `nothing`)
- `sessionId::Union{String, Nothing}`: Optional session identifier (default: `nothing`)
- `maxRetryDelayMs::Union{Int64, Nothing}`: Maximum delay between retries in milliseconds (default: `nothing`)
- `parallelToolExecute::Bool`: Run tool calls in parallel (default: `false`)
- `agentEventSink::Function`: Callback to receive agent events
# Returns
- A new `yiemAgent` instance with an active background task
"""
function yiemAgent(
toolsFolderPath::String,
llmCall,
;
systemPrompt::String="You are helpful assistant.",
model=nothing,
messages::Vector{agentMessage}=agentMessage[],
prepareContext::Function=prepareContext,
formatMsgForLLM::Function=formatMsgForLLM,
beforeToolCall::Function=beforeToolCall,
afterToolCall::Function=afterToolCall,
# prepareNextTurn::Union{Function, Nothing}=nothing,
# prepareNextTurnWithContext::Union{Function, Nothing}=nothing,
sessionId::Union{String, Nothing}=nothing,
maxRetryDelayMs::Union{Int64, Nothing}=nothing,
parallelToolExecute::Bool=false,
agentEventSink=agentEventSink,
)
# Create channels: input (user -> agent), followUp (async queue), output (agent -> user)
inputChannel = Channel(16)
followUp = Channel(32)
outputChannel = Channel(16)
# load tools from toolsFolderPath
toolStore1 = toolStore(name="myagent")
loadTools(toolStore1, toolsFolderPath)
# Create struct with a placeholder task, then spawn and replace it
agent = yiemAgent(
agentState(systemPrompt, model, getTools(toolStore1), messages),
inputChannel,
followUp,
outputChannel,
nothing, # placeholder — replaced below
prepareContext,
formatMsgForLLM,
llmCall,
beforeToolCall,
afterToolCall,
# prepareNextTurn,
# prepareNextTurnWithContext,
sessionId,
maxRetryDelayMs,
parallelToolExecute,
agentEventSink,
)
# Spawn the background loop and attach it
agent._agent_loop = @spawn _agent_loop(agent)
return agent
end
"""
Private agent loop. Runs in a background `@spawn` task.
@@ -81,7 +209,8 @@ function _agent_loop(agent::yiemAgent)
if isready(agent.inputChannel)
# message will be taken in _process_message()
msg = fetch!(agent.inputChannel)
msg = fetch(agent.inputChannel)
agent.agentEventSink("new user msg")
else
yield()
end
@@ -107,9 +236,11 @@ function _agent_loop(agent::yiemAgent)
# start _process_message loop
if agent._state.activeRun == false
agent.agentEventSink("_agent_loop 2")
# Dispatch message through the processing pipeline
processingTask = Threads.@spawn _process_message(agent)
processingTask = Threads.@spawn _process_message(agent)
agent._state.activeRun = true
agent.agentEventSink("_agent_loop 3")
end
# during agent runs, check followUp message after _process_message() is done
@@ -173,6 +304,7 @@ julia> # Currently returns a placeholder echo response
```
"""
function _process_message(agent::yiemAgent)::assistantMessage
agent.agentEventSink("_process_message 1")
# loop until llmCall() response didn't use tool calls
final_response = nothing
while true
@@ -185,21 +317,26 @@ function _process_message(agent::yiemAgent)::assistantMessage
Dict(
"type" => "image_url",
"image_url" => Dict("url" => "data:mime_type;base64,image2_base64_string")
)
),
]
),
)
"""
# Drain inputChannel and convert OpenAI-format messages to userMessage type
while isready(agent.inputChannel)
agent.agentEventSink("_process_message 2")
raw_msg = take!(agent.inputChannel)
agent.agentEventSink("_process_message 3")
if raw_msg === :shutdown
agent.agentEventSink("_process_message 4")
# Re-emit shutdown signal for the loop to handle
put!(agent.inputChannel, :shutdown)
break
end
agent.agentEventSink("_process_message 5")
user_msg = OpenAiToUserMessage(raw_msg)
push!(agent._state.messages, user_msg)
agent.agentEventSink("_process_message 6")
end
# call agent.prepareContext()
@@ -208,10 +345,11 @@ function _process_message(agent::yiemAgent)::assistantMessage
# Call agent.formatMsgForLLM(agent._state) to format for LLM
formatted_messages = agent.formatMsgForLLM(preparedContext)
agent.agentEventSink("_process_message 7")
# Call llmCall() (blocking — the task waits here)
error("debug marker")
response = agent.llmCall(formatted_messages)
error(5555555)
agent.agentEventSink("_process_message 8")
#WORKING Check if LLM used tool calls (inspect content for tool_call blocks)
has_tool_calls = false
@@ -509,41 +647,42 @@ prepareToolCall(context, msg, tc, config, abortedSignal)
```
"""
function prepareToolCall(
context::agentContext,
assistantMsg::assistantMessage,
toolCall::agentToolCall,
config::agentLoopConfig,
signal::Union{Nothing, abortSignal},
context::agentContext,
assistantMsg::assistantMessage,
toolCall::agentToolCall,
config::agentLoopConfig,
signal::Union{Nothing, abortSignal},
)::Union{preparedToolCall,immediateOutcome}
tool = get(context.tools, toolCall.name, nothing)
if tool === nothing
return immediateOutcome(createErrorToolResult("Tool $toolCall.name not found"), true)
tool = get(context.tools, toolCall.name, nothing)
if tool === nothing
return immediateOutcome(createErrorToolResult("Tool $toolCall.name not found"), true)
end
try
# 1. prepare arguments (tool-specific transform)
prepared = prepareToolCallArguments(tool, toolCall)
validatedArgs = validateToolArguments(tool, prepared)
# 2. beforeToolCall hook — can block
if config.beforeToolCall !== nothing
before = config.beforeToolCall(
beforeToolCallContext(assistantMsg, toolCall, validatedArgs, context),
signal
)
if signal !== nothing && signal.aborted
return immediateOutcome(createErrorToolResult("Operation aborted"), true)
end
if before !== nothing && before.block
return immediateOutcome(
createErrorToolResult(get(before, :reason, "Tool execution was blocked")), true)
end
end
try
# 1. prepare arguments (tool-specific transform)
prepared = prepareToolCallArguments(tool, toolCall)
validatedArgs = validateToolArguments(tool, prepared)
# 2. beforeToolCall hook — can block
if config.beforeToolCall !== nothing
before = config.beforeToolCall(
assistantMsgCtx(assistantMsg, toolCall, validatedArgs, context), signal
)
if signal !== nothing && signal.aborted
return immediateOutcome(createErrorToolResult("Operation aborted"), true)
end
if before !== nothing && before.block
return immediateOutcome(
createErrorToolResult(get(before, :reason, "Tool execution was blocked")), true)
end
end
return preparedToolCall(tool, toolCall, validatedArgs)
catch err
return immediateOutcome(createErrorToolResult(sprint(showerror, err)), true)
end
return preparedToolCall(tool, toolCall, validatedArgs)
catch err
return immediateOutcome(createErrorToolResult(sprint(showerror, err)), true)
end
end
# ── per-call execution ──────────────────────────────────────────
@@ -681,61 +820,28 @@ function finalizeExecutedToolCall(
signal::Union{Nothing,abortSignal},
)::finalizedOutcome
result = executed.result
isError = executed.isError
result = executed.result
isError = executed.isError
if config.afterToolCall !== nothing
try
after = config.afterToolCall(
afterCtx(assistantMsg, prep.toolCall, prep.args, result, isError, context), signal
)
if after !== nothing
result = merge(result, dict(:content=>get(after,:content,result.content),
:details=>get(after,:details,result.details),
:usage=>get(after,:usage,result.usage),
:terminate=>get(after,:terminate,result.terminate)))
isError = get(after, :isError, isError)
end
catch err
result = createErrorToolResult(sprint(showerror, err))
isError = true
end
if config.afterToolCall !== nothing
try
after = config.afterToolCall(
afterToolCallContext(assistantMsg, prep.toolCall, prep.args, result, isError, context), signal
)
if after !== nothing
result = merge(result, dict(:content=>get(after,:content,result.content),
:details=>get(after,:details,result.details),
:usage=>get(after,:usage,result.usage),
:terminate=>get(after,:terminate,result.terminate)))
isError = get(after, :isError, isError)
end
catch err
result = createErrorToolResult(sprint(showerror, err))
isError = true
end
end
return finalizedOutcome(prep.toolCall, result, isError)
end
"""
emitToolExecutionEnd(finalized, emit)
Emits the `toolExecutionEnd` event with the finalized outcome,
signalling to listeners that the tool call has completed.
This event is part of the tool execution lifecycle:
`toolExecutionStart` → (zero or more `toolExecutionUpdate` events) →
`toolExecutionEnd`. Listeners (such as the TUI or logging systems)
use this lifecycle to track individual tool calls. The event carries
the final result so listeners have all the data they need without
requiring external state lookups.
# Arguments
- `finalized::finalizedOutcome`: The finalized outcome to report
- `emit::Function`: Event emitter
# Notes
- Part of a three-event lifecycle per tool call
- Carries the complete result so listeners need no external lookups
# Examples
```julia
# Emits a single event; returns nothing
emitToolExecutionEnd(finalized, emit)
# (emit receives toolExecEndEvent("call_1", "search_wine", result, false))
```
"""
function emitToolExecutionEnd(finalized::finalizedOutcome, emit::Function)
emit(toolExecEndEvent(finalized.toolCall.id, finalized.toolCall.name,
finalized.result, finalized.isError))
return finalizedOutcome(prep.toolCall, result, isError)
end
# ── sequential execution ────────────────────────────────────────
@@ -816,7 +922,8 @@ function executeToolCallsSequential(
finalized = finalizeExecutedToolCall(context, assistantMsg, prep, executed, config, signal)
end
emitToolExecutionEnd(finalized, emit)
emit(toolExecEndEvent(finalized.toolCall.id, finalized.toolCall.name,
finalized.result, finalized.isError))
push!(messages, createToolResultMessage(finalized))
push!(finalizedCalls, finalized)
@@ -903,13 +1010,15 @@ function executeToolCallsParallel(
if prep isa immediateOutcome
finalized = finalizedOutcome(tc, prep.result, prep.isError)
emitToolExecutionEnd(finalized, emit)
emit(toolExecEndEvent(finalized.toolCall.id, finalized.toolCall.name,
finalized.result, finalized.isError))
push!(entries, finalized)
else
task = task() do
executed = executePreparedToolCall(prep, signal, emit)
finalized = finalizeExecutedToolCall(context, assistantMsg, prep, executed, config, signal)
emitToolExecutionEnd(finalized, emit)
emit(toolExecEndEvent(finalized.toolCall.id, finalized.toolCall.name,
finalized.result, finalized.isError))
return finalized
end
schedule(task)
+14 -15
View File
@@ -5,12 +5,11 @@ export prompt
using JSON, DataStructures, Dates, UUIDs, HTTP, Random, PrettyPrinting, Serialization,
DataFrames
using GeneralUtils
using ..type, ..utils
using ..type, ..utils, ..agentCore, ..toolRegistry
# ---------------------------------------------- 100 --------------------------------------------- #
"""
Send a message to the agent's input channel.
@@ -25,16 +24,16 @@ The agent processes messages from `inputChannel` in the background task.
- The same `agent` instance for chaining
# Notes
- Use `take_response(agent)` to receive the agent's response after sending a message.
- Use `follow_up(agent, msg)` to send messages while the agent is still processing.
- Use `takeResponse(agent)` to receive the agent's response after sending a message.
- Use `followUp(agent, msg)` to send messages while the agent is still processing.
# Examples
```jldoctest
julia> run_agent(agent, "Hello!")
julia> runAgent(agent, "Hello!")
yiemAgent(...)
```
"""
function run_agent(agent::yiemAgent, msg)
function runAgent(agent::yiemAgent, msg)
put!(agent.inputChannel, msg)
return agent
end
@@ -51,15 +50,15 @@ Blocks until the agent sends a response.
- An `assistantMessage` instance representing the agent's response
# Notes
- Use `run_agent(agent, msg)` to send a message before calling this function.
- Use `runAgent(agent, msg)` to send a message before calling this function.
# Examples
```jldoctest
julia> response = take_response(agent)
julia> response = takeResponse(agent)
assistantMessage(...)
```
"""
function take_response(agent::yiemAgent)
function takeResponse(agent::yiemAgent)
return take!(agent.outputChannel)
end
@@ -77,17 +76,17 @@ and before any tool call results are sent.
- The same `agent` instance for chaining
# Notes
- Use `run_agent(agent, msg)` for the primary message and `follow_up(agent, msg)` for additional
- Use `runAgent(agent, msg)` for the primary message and `followUp(agent, msg)` for additional
messages while the agent is processing.
- Follow-up messages are buffered in a separate channel (capacity 32 by default).
# Examples
```jldoctest
julia> follow_up(agent, "Also consider red wines")
julia> followUp(agent, "Also consider red wines")
yiemAgent(...)
```
"""
function follow_up(agent::yiemAgent, msg)
function followUp(agent::yiemAgent, msg)
put!(agent.followUpChannel, msg)
return agent
end
@@ -105,16 +104,16 @@ then closes all channels (`inputChannel`, `outputChannel`, `followUpChannel`).
- `nothing`
# Notes
- After calling `stop_agent`, the agent is no longer usable. A new agent must be created
- After calling `stopAgent`, the agent is no longer usable. A new agent must be created
for further interaction.
- If the background task throws a `TaskFailedException`, it is rethrown.
# Examples
```jldoctest
julia> stop_agent(agent)
julia> stopAgent(agent)
```
"""
function stop_agent(agent::yiemAgent)
function stopAgent(agent::yiemAgent)
put!(agent.inputChannel, :shutdown)
try
fetch(agent._agent_loop)
+26 -158
View File
@@ -13,8 +13,8 @@
# Context types
agentContext, agentState, agentToolCall, prepareNextTurnContext,
# Loop & execution types
agentLoopConfig, abortSignal, agentToolResult,
assistantMsgCtx, afterCtx,
agentLoopConfig, abortSignal, agentToolResult,beforeToolCallContext,
beforeToolCallResult, afterToolCallContext,
# Event types
toolExecStartEvent, toolExecUpdateEvent, toolExecEndEvent,
# Agent
@@ -23,7 +23,7 @@
preparedToolCall, immediateOutcome, executedOutcome, finalizedOutcome,
agentToolCallBatch,
# Functions (defined elsewhere)
run_agent, take_response, follow_up, stop_agent
runAgent, takeResponse, followUp, stopAgent
using Dates, UUIDs, DataStructures, JSON, NATS, Base.Threads
@@ -144,7 +144,7 @@ assistantMessage("assistant", [textContent("Hello!")], "", "", "gpt-4", ..., "en
```
"""
function assistantMessage(; role="assistant", content=Vector{messageContent}(),
api="", provider="", model="", usage=llmUsage(0, 0), stopReason="end_turn",
api="", provider="", model=nothing, usage=llmUsage(0, 0), stopReason="end_turn",
errorMessage=nothing, timestamp=now())
return assistantMessage(role, content, api, provider, model, usage, stopReason, errorMessage, timestamp)
end
@@ -309,7 +309,7 @@ end
mutable struct agentState # Mutable runtime state of an agent
systemPrompt::String # System prompt for the agent
model::llmModel # LLM model to use
model::Union{llmModel, Nothing} # LLM model to use
tools::OrderedDict{String, agentTool} # Available tools keyed by name, insertion-ordered
# messages history includes userMessage, assistantMessage, toolResultMessage. NO system prompt
@@ -341,21 +341,21 @@ julia> state = agentState(systemPrompt="You are a helpful assistant")
agentState("You are a helpful assistant", OrderedDict{String, agentTool}(), agentMessage[], String[], nothing)
"""
function agentState(
systemPrompt::String="",
model::llmModel=llmModel{String}("", "", "unknown", "unknown", "", false, String[],
modelCost(0.0, 0.0, 0.0, 0.0), 0, 0),
tools::OrderedDict{String, agentTool}=OrderedDict{String, agentTool}(),
messages::Vector{agentMessage}=agentMessage[],
systemPrompt::String="",
model=llmModel("model_1", "unknown", "unknown", "", false, String[],
modelCost(0.0, 0.0, 0.0, 0.0), 0, 0),
tools::OrderedDict{String, agentTool}=OrderedDict{String, agentTool}(),
messages::Vector{agentMessage}=agentMessage[],
)
agentState(
systemPrompt,
model,
deepcopy(tools),
deepcopy(messages),
Vector{String}(),
false,
nothing,
)
agentState(
systemPrompt,
model,
deepcopy(tools),
deepcopy(messages),
Vector{String}(),
false,
nothing,
)
end
@@ -442,13 +442,18 @@ Context passed to the `beforeToolCall` hook.
- `args::Dict{String,Any}`: Validated tool arguments
- `context::agentContext`: Current conversation context
"""
struct assistantMsgCtx
struct beforeToolCallContext
message::assistantMessage
toolCall::agentToolCall
args::Dict{String,Any}
context::agentContext
end
struct beforeToolCallResult
block::Bool
reason::String
end
"""
Context passed to the `afterToolCall` hook.
@@ -460,7 +465,7 @@ Context passed to the `afterToolCall` hook.
- `isError::Bool`: Whether execution resulted in an error
- `context::agentContext`: Current conversation context
"""
struct afterCtx
struct afterToolCallContext
message::assistantMessage
toolCall::agentToolCall
args::Dict{String,Any}
@@ -521,143 +526,6 @@ end
abstract type agent end
"""
docstring
"""
mutable struct yiemAgent <: agent # High-level agent wrapper
_state::agentState # Current state (prompt, model, messages, tools, etc.)
# user sends prompt message to agent. if agent is idle, it process user message right away.
# if agent is running, it process user message after the current tool call finished.
inputChannel::Channel
# Buffers messages the user sends while the agent is busy. Processed after all inputChannel
# messages are handled and the agent is idle (not using a tool call).
followUpChannel::Channel
# agent sends response message to user after processing all user messages in inputChannel
# and all followUp messages.
outputChannel::Channel
_agent_loop::Union{Task, Nothing} # agent loop running in the background
# Preprocess/transform messages and context (modify, filter, prune, inject context from memory,
# reorder, ...) for a single LLM call in _process_message()'s loop.
# returns new Vector{agentMessage}
prepareContext ::Union{Function, Nothing}
# Convert prepareContext()'s new Vector{agentMessage} to LLM message format
formatMsgForLLM::Function
# Actually invoke the LLM to get a completion response. The LLM response comes back as an
# assistantMessage whose content is an array of content blocks.
# Each block has a type — "text", "thinking", or "toolCall".
# The code filters for type === "toolCall" blocks, then passes them to executeToolCalls().
llmCall::Function
# Callback invoked before executing a tool call (ask for user permission/confirmation/abort, etc..)
beforeToolCall::Union{Function, Nothing}
executeToolCalls::Function # execute tool calls ()
# Callback invoked after executing a tool call to sanitize tools output so the output is ready
# to be converted into toolResults message
afterToolCall::Union{Function, Nothing}
# prepareNextTurn::Union{Function, Nothing} # Callback to prepare the next conversation turn
# prepareNextTurnWithContext::Union{Function, Nothing} # Same but receives context
sessionId::Union{String, Nothing} # Optional session identifier
maxRetryDelayMs::Union{Int64, Nothing} # Maximum delay between retries (ms)
parallelToolExecute::Bool # Default: false
agentEventSink::Function # agent emits its status via this function
_tool_store::Any # Reference to the toolStore for runtime registration
end
"""
Create a new yiemAgent instance with a background loop task.
Spawns a background `@spawn` task that runs the agent loop, listening
on `inputChannel` and `followUpChannel` channels concurrently.
# Keyword Arguments
- `systemPrompt::String`: System prompt for the agent
- `model`: LLM model to use
- `tools::OrderedDict{String, agentTool}`: Available tools keyed by name (default: empty)
- `messages::Vector{agentMessage}`: Initial conversation messages (default: empty)
- `formatMsgForLLM::Function`: Convert agent messages to LLM message format (default: `defaultformatMsgForLLM`)
- `llmCall::Function`: Function to invoke the LLM (required)
- `prepareContext::Union{Function, Nothing}`: Preprocess/transform messages before sending to LLM (default: `nothing`)
- `beforeToolCall::Union{Function, Nothing}`: Callback invoked before executing a tool call (default: `nothing`)
- `afterToolCall::Union{Function, Nothing}`: Callback invoked after executing a tool call (default: `nothing`)
- `prepareNextTurn::Union{Function, Nothing}`: Callback to prepare the next conversation turn (default: `nothing`)
- `prepareNextTurnWithContext::Union{Function, Nothing}`: Same but receives context (default: `nothing`)
- `sessionId::Union{String, Nothing}`: Optional session identifier (default: `nothing`)
- `maxRetryDelayMs::Union{Int64, Nothing}`: Maximum delay between retries in milliseconds (default: `nothing`)
- `parallelToolExecute::Bool`: Run tool calls in parallel (default: `false`)
- `agentEventSink::Function`: Callback to receive agent events
- `tool_store::Union{Any, Nothing}`: toolStore for runtime tool registration (default: `nothing`)
# Returns
- A new `yiemAgent` instance with an active background task
# Examples
```julia
julia> store = toolStore(name="agent1")
julia> tools = loadTools(store, "src/tools")
julia> agent = yiemAgent(systemPrompt="You are a helpful assistant", model=my_model, tools=tools, llmCall=..., tool_store=store)
yiemAgent(agentState(...), Channel(...), Channel(...), Channel(...), ..., store)
"""
function yiemAgent(
; systemPrompt::String="You are helpful assistant.",
model=nothing,
tools::OrderedDict{String, agentTool}=OrderedDict{String, agentTool}(),
messages::Vector{agentMessage}=agentMessage[],
prepareContext::Union{Function, Nothing}=nothing,
formatMsgForLLM::Function=defaultformatMsgForLLM,
llmCall::Function,
beforeToolCall::Union{Function, Nothing}=nothing,
afterToolCall::Union{Function, Nothing}=nothing,
# prepareNextTurn::Union{Function, Nothing}=nothing,
# prepareNextTurnWithContext::Union{Function, Nothing}=nothing,
sessionId::Union{String, Nothing}=nothing,
maxRetryDelayMs::Union{Int64, Nothing}=nothing,
parallelToolExecute::Bool=false,
agentEventSink::Function,
tool_store::Union{Any, Nothing}=nothing,
)
# Create channels: input (user -> agent), followUp (async queue), output (agent -> user)
inputChannel = Channel(16)
followUp = Channel(32)
outputChannel = Channel(16)
# Create struct with a placeholder task, then spawn and replace it
agent = yiemAgent(
agentState(systemPrompt, model, tools, messages),
inputChannel,
followUp,
outputChannel,
nothing, # placeholder — replaced below
prepareContext,
formatMsgForLLM,
llmCall,
beforeToolCall,
afterToolCall,
# prepareNextTurn,
# prepareNextTurnWithContext,
sessionId,
maxRetryDelayMs,
parallelToolExecute,
agentEventSink,
tool_store,
)
# Spawn the background loop and attach it
agent._agent_loop = @spawn _agent_loop(agent)
return agent
end
"""
preparedToolCall(tool, toolCall, args)
+32 -2
View File
@@ -1,7 +1,9 @@
module utils
export clearhistory, availableWineToText, prepareContext, formatMsgForLLM, validateRequiredArgs, validateToolArguments, _userMessageToOpenAI,
_assistantMessageToOpenAI, _toolResultMessageToOpenAI, _messageContentToBlocks
export clearhistory, availableWineToText, prepareContext, formatMsgForLLM, validateRequiredArgs,
validateToolArguments, _userMessageToOpenAI,
_assistantMessageToOpenAI, _toolResultMessageToOpenAI, _messageContentToBlocks,
beforeToolCall, afterToolCall, agentEventSink
using UUIDs, Dates, DataStructures, HTTP, JSON
using GeneralUtils
@@ -218,6 +220,34 @@ function formatMsgForLLM(ctx::agentContext)::Dict{String, Any}
return Dict("messages" => messages)
end
#TODO
function beforeToolCall(context::beforeToolCallContext, signal::abortSignal
)::beforeToolCallResult
# final context check
# seek user approval via UI
# other check
return beforeToolCallResult(false, "N/A")
end
#TODO
function afterToolCall(context::beforeToolCallContext, signal::abortSignal
)::Union{agentToolResult, Nothing}
# modify context.result if needed and return agentToolResult
return nothing
end
#TODO
function agentEventSink(x)
end
"""
Convert a userMessage to OpenAI message format.
-401
View File
@@ -1,401 +0,0 @@
using JSON, Dates, UUIDs, PrettyPrinting, LibPQ, Base64, DataFrames, DataStructures, HTTP, Base64,
NATS, Base.Threads
using YiemAgent, GeneralUtils, msghandler
function text2text_instruct_llm(sender_id::String, openai_msg::Dict{String, Any})
payloads = [("msg", openai_msg, "dictionary")] # List of tuples
_, msg_envelope_json_str = msghandler.smartpack(
config["externalservice"]["servicesloadbalancer"]["nats"],
payloads;
sender_id=sender_id,
msg_purpose="text2text",
broker_url=config["nats_server_info"]["url"],
fileserver_url=config["externalservice"]["fileserver"]["url"])
reply = NATS.request(agent_conn,
config["externalservice"]["servicesloadbalancer"]["nats"],
msg_envelope_json_str, timeout=120)
incoming_env_json_str = String(reply.payload)
incoming_env = msghandler.smartunpack(incoming_env_json_str)
_llm_response = incoming_env["payloads"][1][2]
llm_response = _llm_response["choices"][1]["message"]["content"]
return llm_response
end
""" get a single text embedding from a LLM service
Example
text = ["hello"]
embedding = get_embedding(text)
"""
function get_embedding(text::AbstractArray{String})
documents_dict = Dict("documents" => text)
payloads = [("documents", documents_dict, "dictionary")]
_, msg_envelope_json_str = msghandler.smartpack(
config["externalservice"]["servicesloadbalancer"]["nats"],
payloads;
msg_purpose="embedding",
broker_url=config["nats_server_info"]["url"],
fileserver_url=config["externalservice"]["fileserver"]["url"])
reply = NATS.request(agent_conn,
config["externalservice"]["servicesloadbalancer"]["nats"],
msg_envelope_json_str, timeout=120)
incoming_env_json_str = String(reply.payload)
incoming_env = msghandler.smartunpack(incoming_env_json_str)
embedding_response = incoming_env["payloads"][1][2]
return embedding_response
end
""" sql = "SELECT * FROM wine;"
result = execute_sql_winedb(sql)
"""
function execute_sql_winedb(sql::T) where {T<:AbstractString}
host_url, _port = split(config["externalservice"]["sommpanion_db"]["url"], ':')
port = parse(Int, _port)
dbname = "winedb"
user = config["externalservice"]["sommpanion_db"]["user"]
password = config["externalservice"]["sommpanion_db"]["password"]
db_connection = LibPQ.Connection("host=$host_url port=$port dbname=$dbname user=$user password=$password")
result = nothing
try
result = LibPQ.execute(db_connection, sql)
catch e
LibPQ.close(db_connection)
end
LibPQ.close(db_connection)
return result
end
""" find similar sql from vector database
sql = "SELECT * FROM wine;"
result, distance = similar_sql_vectordb(sql)
"""
function similar_sql_vectordb(sql::T; maxdistance::Number=0.2) where {T<:AbstractString}
tablename = "sqlllm_decision_repository"
# get embedding of the query
df = find_similar_text_from_vectordb(sql, tablename,
"function_input_embedding", execute_sql_vectordb)
# println(df[1, [:id, :function_output]])
row, col = size(df)
distance = row == 0 ? Inf : df[1, :distance]
if row != 0 && distance < maxdistance
# if there is usable SQL, return it.
output_b64 = df[1, :function_output_base64] # pick the closest match
output_str = String(base64decode(output_b64))
rowid = df[1, :id]
println("\n--| similar sql found. row id $rowid, distance $distance ", @__FILE__, ":", @__LINE__, " $(Dates.now())")
pprintln(output_str)
return (result=output_str, distance=distance)
else
println("\n--| similar sql not found, max distance $maxdistance ", @__FILE__, ":", @__LINE__, " $(Dates.now())")
return (result=nothing, distance=nothing)
end
end
""" insert query and sql into vector database
query = "get all wines from wine table"
sql = "SELECT * FROM wine;"
insert_sql_vectordb(query, sql)
"""
function insert_sql_vectordb(query::T1, SQL::T2; maxdistance::Number=3
) where {T1<:AbstractString, T2<:AbstractString}
tablename = "sqlllm_decision_repository"
# get embedding of the query
# query = state[:thoughtHistory][:question]
df = find_similar_text_from_vectordb(query, tablename,
"function_input_embedding", execute_sql_vectordb)
row, col = size(df)
distance = row == 0 ? Inf : df[1, :distance]
if row == 0 || distance > maxdistance # no close enough SQL stored in the database
_query_embedding = get_embedding([query])
_query_embedding = GeneralUtils.dictify(_query_embedding)
# println("\n--- _query_embedding() ", @__FILE__, ":", @__LINE__, " $(Dates.now())")
# println(_query_embedding)
# println("---\n")
query_embedding = _query_embedding["data"][1]["embedding"]
query = replace(query, "'" => "")
sql_base64 = base64encode(SQL)
sql_ = replace(SQL, "'" => "")
sql =
"""
INSERT INTO $tablename (function_input, function_output, function_output_base64, function_input_embedding) VALUES ('$query', '$sql_', '$sql_base64', '$query_embedding');
"""
# println("\n--| added new decision to vectorDB ", @__FILE__, ":", @__LINE__, " $(Dates.now())")
# println(sql)
_ = execute_sql_vectordb(sql)
end
end
""" execute sql against vectordb
sql = "SELECT * FROM wine;"
result = execute_sql_vectordb(sql)
"""
function execute_sql_vectordb(sql::T) where {T<:AbstractString}
host_url, _port = split(config["externalservice"]["sommpanion_vectordb"]["url"], ':')
port = parse(Int, _port)
dbname = config["externalservice"]["sommpanion_vectordb"]["dbname"]
user = config["externalservice"]["sommpanion_vectordb"]["user"]
password = config["externalservice"]["sommpanion_vectordb"]["password"]
DBconnection = LibPQ.Connection("host=$host_url port=$port dbname=$dbname user=$user password=$password")
result = LibPQ.execute(DBconnection, sql)
close(DBconnection)
return result
end
""" search similar decision llm made from vectordb
"""
function similar_sommelier_decision(recentevents::T1; maxdistance::Integer=3
)::Union{AbstractDict, Nothing} where {T1<:AbstractString}
tablename = "sommelier_decision_repository"
# find similar
df = find_similar_text_from_vectordb(recentevents, tablename,
"function_input_embedding", execute_sql_vectordb)
row, col = size(df)
distance = row == 0 ? Inf : df[1, :distance]
if row != 0 && distance < maxdistance
# if there is usable decision, return it.
rowid = df[1, :id]
println("\n--| found similar decision. row id $rowid, distance $distance ", @__FILE__, " ", @__LINE__)
output_b64 = df[1, :function_output_base64] # pick the closest match
_output_str = String(base64decode(output_b64))
output = copy(JSON.read(_output_str))
return output
else
println("\n--| similar decision not found, max distance $maxdistance ", @__FILE__, " ", @__LINE__)
return nothing
end
end
""" search similar text from vectordb
"""
function find_similar_text_from_vectordb(text::T1, tablename::T2, embeddingColumnName::T3,
vectorDB::Function; limit::Integer=1
)::DataFrame where {T1<:AbstractString, T2<:AbstractString, T3<:AbstractString}
# get embedding from LLM service
_embedding = get_embedding([text])
_embedding = _embedding["data"][1]["embedding"]
_embedding = "$_embedding"
embedding = _embedding[4:end]
# check whether there is close enough vector already store in vectorDB. if no, add, else skip
sql = """
SELECT *, $embeddingColumnName <-> '$embedding' as distance
FROM $tablename
ORDER BY distance LIMIT $limit;
"""
response = vectorDB(sql)
df = DataFrame(response)
return df
end
""" insert decision llm made to vectordb
"""
function insert_sommelier_decision(recentevents::T1, decision::T2; maxdistance::Integer=5
) where {T1<:AbstractString, T2<:AbstractDict}
tablename = "sommelier_decision_repository"
# find similar
df = find_similar_text_from_vectordb(recentevents, tablename,
"function_input_embedding", execute_sql_vectordb)
row, col = size(df)
distance = row == 0 ? Inf : df[1, :distance]
if row == 0 || distance > maxdistance # no close enough SQL stored in the database
_embedding = get_embedding([recentevents])[1]
recentevents_embedding = _embedding["data"][1]["embedding"]
recentevents = replace(recentevents, "'" => "")
decision_json = JSON.json(decision)
decision_base64 = base64encode(decision_json)
decision = replace(decision_json, "'" => "")
sql =
"""
INSERT INTO $tablename (function_input, function_output, function_output_base64, function_input_embedding) VALUES ('$recentevents', '$decision', '$decision_base64', '$recentevents_embedding');
"""
println("\n--| added new decision to vectorDB ", @__FILE__, " ", @__LINE__)
println(sql)
_ = execute_sql_vectordb(sql)
else
println("--| similar decision previously cached, distance $distance ", @__FILE__, " ", @__LINE__)
end
end
config = JSON.parsefile("./appconfig.json")
sessionId = "0"
backend_session_topic = "sommpanion.testsubject"
agent_ch = Channel(8)
agent_conn = NATS.connect(config["nats_server_info"]["url"])
sub2 = NATS.subscribe(agent_conn, backend_session_topic) do msg
put!(agent_ch, msg)
end
agent_context = YiemAgent.agentcontext(
text2text_instruct_llm,
get_embedding,
execute_sql_winedb,
similar_sql_vectordb,
insert_sql_vectordb,
similar_sommelier_decision,
insert_sommelier_decision
)
# can't instantiate
agent = YiemAgent.sommelier(
agent_context;
name="Janie",
id=sessionId, # agent instance id
retailername="Yiem Wine Ltd.",
llmFormatName=""
)
image1_path = "test/large_image.png"
image1_bytes = read(image1_path)
image1_base64_string = base64encode(image1_bytes)
mime_type = "image/png"
data1_uri = "data:$(mime_type);base64,$(image1_base64_string)"
# 1. Read local file and encode to base64 string
image2_path = "test/small_image.png"
image2_bytes = read(image2_path)
image2_base64_string = base64encode(image2_bytes)
mime_type = "image/png"
data2_uri = "data:$(mime_type);base64,$(image2_base64_string)"
# 3. Construct payload with the Data URI
message = Dict(
"role" => "user",
"content" => [
Dict("type" => "text", "text" => "Do you know type of wine in the image?"),
Dict(
"type" => "image_url",
"image_url" => Dict("url" => data1_uri)
)
]
)
result = YiemAgent.conversation(agent; userinput=message)
println("\n$result")
# message = Dict(
# "role" => "user",
# "content" => [
# Dict("type" => "text", "text" =>
# "
# เป็นงานเลี้ยงทั่วไป
# "),
# ]
# )
# result = YiemAgent.conversation(agent; userinput=message)
# println("\n$result")
# message = Dict(
# "role" => "user",
# "content" => [
# Dict("type" => "text", "text" => "no thanks. that's all"),
# ]
# )
# result = YiemAgent.conversation(agent; userinput=message)
# println("\n$result")
# message = Dict(
# "role" => "user",
# "content" => [
# Dict("type" => "text", "text" => "What about this wine?"),
# Dict(
# "type" => "image_url",
# "image_url" => Dict("url" => data2_uri)
# )
# ]
# )
# result = YiemAgent.conversation(agent; userinput=message)
# println("\n$result")