update
This commit is contained in:
@@ -68,10 +68,22 @@ MCTSNode(
|
|||||||
isterminal::Bool,
|
isterminal::Bool,
|
||||||
parent::Union{MCTSNode, Nothing},
|
parent::Union{MCTSNode, Nothing},
|
||||||
children::Dict{String, MCTSNode},
|
children::Dict{String, MCTSNode},
|
||||||
etc::Dict{String, Any}
|
etc::Dict{Symbol, Any}
|
||||||
)
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
|
**Fields:**
|
||||||
|
- `nodekey::String` — Unique identifier for the node
|
||||||
|
- `state::Dict` — Current state represented as a dictionary
|
||||||
|
- `visits::Integer` — Number of times this node has been visited
|
||||||
|
- `progressvalue::Number` — LLM's estimate of state quality
|
||||||
|
- `statevalue::Number` — Average cumulative reward from simulations
|
||||||
|
- `reward::Number` — Immediate reward at this node
|
||||||
|
- `isterminal::Bool` — Whether this node represents a terminal state
|
||||||
|
- `parent::Union{MCTSNode, Nothing}` — Parent node reference (nothing for root)
|
||||||
|
- `children::Dict{String, MCTSNode}` — Mapping of child nodes
|
||||||
|
- `etc::Dict{Symbol, Any}` — Additional arbitrary data storage (uses Symbol keys)
|
||||||
|
|
||||||
### Understanding `progressvalue`, `statevalue`, and `reward`
|
### Understanding `progressvalue`, `statevalue`, and `reward`
|
||||||
|
|
||||||
| Field | Source | Purpose |
|
| Field | Source | Purpose |
|
||||||
@@ -212,11 +224,33 @@ Run simulation from a node and backpropagate the reward. Returns `nothing`.
|
|||||||
- `multithread::Bool=false` — Enable multithreading
|
- `multithread::Bool=false` — Enable multithreading
|
||||||
- `highValueState` — Channel to store high-value states
|
- `highValueState` — Channel to store high-value states
|
||||||
|
|
||||||
|
#### `backpropagate(node, simTrajectoryReward; kwargs...)`
|
||||||
|
|
||||||
|
Backpropagate reward along the simulation chain. Updates visit counts and state values for all nodes along the path to the root. Returns `nothing`.
|
||||||
|
|
||||||
|
**Arguments:**
|
||||||
|
- `node::MCTSNode` — The leaf node from which to start backpropagation
|
||||||
|
- `simTrajectoryReward::Number` — The total reward from the trajectory simulation
|
||||||
|
|
||||||
|
**Keyword Arguments:**
|
||||||
|
- `discountRewardCoeff::AbstractFloat=0.9` — Discount coefficient applied to future rewards
|
||||||
|
|
||||||
### Utility Functions
|
### Utility Functions
|
||||||
|
|
||||||
- `UCTselect(node, w)` — Select node using UCT score
|
- `UCTselect(node, w)` — Select node using UCT score
|
||||||
- `dictify(x; keytype=Any)` — Convert JSON.Object/OrderedDict to plain Dict
|
- `dictify(x; keytype=Any)` — Convert JSON.Object/OrderedDict to plain Dict
|
||||||
|
|
||||||
|
### MCTS Utility Functions
|
||||||
|
|
||||||
|
- `selectBestNextNode(node)` — Select best child node based on value metric
|
||||||
|
- `selectBestTrajectoryNode(node)` — Select best node along optimal trajectory
|
||||||
|
- `backpropagate(node, simTrajectoryReward; kwargs...)` — Backpropagate reward up the tree
|
||||||
|
- `isleaf(node)` — Check if node is a leaf (has no children)
|
||||||
|
- `isroot(node)` — Check if node is the root node
|
||||||
|
- `selectChildNode(node)` — Select child with highest `progressvalue + reward`
|
||||||
|
- `expand(node, transition, transitionargs; kwargs...)` — Generate child nodes
|
||||||
|
- `simulate(node, transition, transitionargs; kwargs...)` — Perform rollout simulation
|
||||||
|
|
||||||
### MCTS Node Structure
|
### MCTS Node Structure
|
||||||
|
|
||||||
```julia
|
```julia
|
||||||
@@ -230,7 +264,7 @@ MCTSNode(
|
|||||||
isterminal::Bool,
|
isterminal::Bool,
|
||||||
parent::Union{MCTSNode, Nothing},
|
parent::Union{MCTSNode, Nothing},
|
||||||
children::Dict{String, MCTSNode},
|
children::Dict{String, MCTSNode},
|
||||||
etc::Dict{String, Any}
|
etc::Dict{Symbol, Any}
|
||||||
)
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
@@ -1,87 +0,0 @@
|
|||||||
""" Recursively convert dictionary-like variable (e.g. JSON.Object) into a dictionary.
|
|
||||||
The function walks any nested structure composed of `AbstractDict` (e.g., `JSON.Object`,
|
|
||||||
`Dict`, `OrderedDict`) and `AbstractArray` and produces a new tree where
|
|
||||||
every dictionary-like node is a plain `Dict` and every array-like node is a
|
|
||||||
`Vector{Any}`. Scalar values (numbers, strings, booleans, `nothing`, etc.)
|
|
||||||
are returned unchanged.
|
|
||||||
Does **not** mutate the input; it always allocates new containers.
|
|
||||||
|
|
||||||
# Arguments
|
|
||||||
- `x`
|
|
||||||
Any Julia value. If `x` is an `AbstractDict` it will be converted to a `Dict`;
|
|
||||||
if it is an `AbstractArray` its elements will be processed recursively.
|
|
||||||
|
|
||||||
# Keyword Arguments
|
|
||||||
- `keytype::Type=Any`
|
|
||||||
The key type for the output Dict. Use `String` for `Dict{String,Any}`, `Symbol` for `Dict{Symbol,Any}`, or `Any` to preserve original key types.
|
|
||||||
- `stringkey::Bool=false`
|
|
||||||
If `true`, every dictionary key is converted to `String` via `string(k)`. This parameter is ignored when `keytype` is explicitly set.
|
|
||||||
|
|
||||||
# Return
|
|
||||||
- A newly allocated nested structure composed of `Dict{keytype,Any}` and
|
|
||||||
`Vector{Any}` that mirrors the input shape but uses plain Julia containers.
|
|
||||||
|
|
||||||
# Notes
|
|
||||||
- The function treats any `AbstractDict` as a mapping source, so it works with
|
|
||||||
`JSON.Object`, `Dict`, `OrderedDict`, etc.
|
|
||||||
- Arrays are returned as `Vector{Any}` with their elements processed
|
|
||||||
recursively.
|
|
||||||
|
|
||||||
# Examples
|
|
||||||
```jldoctest
|
|
||||||
julia> using JSON
|
|
||||||
julia> d = Dict(
|
|
||||||
"a" => 4,
|
|
||||||
"b" => 6,
|
|
||||||
"c" => Dict(
|
|
||||||
"d"=>7,
|
|
||||||
:e=>Dict(
|
|
||||||
"f"=>"hey",
|
|
||||||
"g"=>Dict(
|
|
||||||
"world"=>[1, "2", 3, Dict(:dd=>4.7)]
|
|
||||||
)
|
|
||||||
)
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
julia jsonstring = JSON.json(d)
|
|
||||||
julia> A1 = JSON.parse(jsonstring) # A1 type is JSON.Object
|
|
||||||
julia> A2 = dictify(A1; keytype=String)
|
|
||||||
Dict{String,Any} with 3 entries:
|
|
||||||
"a" => 4
|
|
||||||
"b" => 6
|
|
||||||
"c" => Dict("d"=>7, "e"=>Dict("f"=>"hey", "g"=>Dict("world"=>[1, "2", 3, 4.7])))
|
|
||||||
|
|
||||||
julia> A3 = dictify(A1; keytype=Symbol)
|
|
||||||
Dict{Symbol,Any} with 3 entries:
|
|
||||||
:a => 4
|
|
||||||
:b => 6
|
|
||||||
:c => Dict(:d=>7, :e=>Dict("f"=>"hey", "g"=>Dict("world"=>[1, "2", 3, 4.7])))
|
|
||||||
|
|
||||||
julia> B1 = dictify(d; keytype=String)
|
|
||||||
Dict{String, Any} with 3 entries:
|
|
||||||
"""
|
|
||||||
function dictify(x; keytype::Type=Any)
|
|
||||||
# Dict-like objects
|
|
||||||
if x isa AbstractDict
|
|
||||||
# choose output key type container
|
|
||||||
out = Dict{keytype,Any}()
|
|
||||||
for (k,v) in x
|
|
||||||
if keytype === String
|
|
||||||
newk = string(k)
|
|
||||||
elseif keytype === Symbol
|
|
||||||
newk = Symbol(string(k))
|
|
||||||
else
|
|
||||||
newk = k
|
|
||||||
end
|
|
||||||
out[newk] = dictify(v; keytype=keytype)
|
|
||||||
end
|
|
||||||
return out
|
|
||||||
# Arrays / vectors: map elements recursively and return a Vector{Any}
|
|
||||||
elseif x isa AbstractArray
|
|
||||||
return [dictify(element; keytype=keytype) for element in x]
|
|
||||||
# everything else: return as-is (primitives, numbers, strings, etc.)
|
|
||||||
else
|
|
||||||
return x
|
|
||||||
end
|
|
||||||
end
|
|
||||||
Reference in New Issue
Block a user