update
This commit is contained in:
@@ -68,10 +68,22 @@ MCTSNode(
|
||||
isterminal::Bool,
|
||||
parent::Union{MCTSNode, Nothing},
|
||||
children::Dict{String, MCTSNode},
|
||||
etc::Dict{String, Any}
|
||||
etc::Dict{Symbol, Any}
|
||||
)
|
||||
```
|
||||
|
||||
**Fields:**
|
||||
- `nodekey::String` — Unique identifier for the node
|
||||
- `state::Dict` — Current state represented as a dictionary
|
||||
- `visits::Integer` — Number of times this node has been visited
|
||||
- `progressvalue::Number` — LLM's estimate of state quality
|
||||
- `statevalue::Number` — Average cumulative reward from simulations
|
||||
- `reward::Number` — Immediate reward at this node
|
||||
- `isterminal::Bool` — Whether this node represents a terminal state
|
||||
- `parent::Union{MCTSNode, Nothing}` — Parent node reference (nothing for root)
|
||||
- `children::Dict{String, MCTSNode}` — Mapping of child nodes
|
||||
- `etc::Dict{Symbol, Any}` — Additional arbitrary data storage (uses Symbol keys)
|
||||
|
||||
### Understanding `progressvalue`, `statevalue`, and `reward`
|
||||
|
||||
| Field | Source | Purpose |
|
||||
@@ -212,11 +224,33 @@ Run simulation from a node and backpropagate the reward. Returns `nothing`.
|
||||
- `multithread::Bool=false` — Enable multithreading
|
||||
- `highValueState` — Channel to store high-value states
|
||||
|
||||
#### `backpropagate(node, simTrajectoryReward; kwargs...)`
|
||||
|
||||
Backpropagate reward along the simulation chain. Updates visit counts and state values for all nodes along the path to the root. Returns `nothing`.
|
||||
|
||||
**Arguments:**
|
||||
- `node::MCTSNode` — The leaf node from which to start backpropagation
|
||||
- `simTrajectoryReward::Number` — The total reward from the trajectory simulation
|
||||
|
||||
**Keyword Arguments:**
|
||||
- `discountRewardCoeff::AbstractFloat=0.9` — Discount coefficient applied to future rewards
|
||||
|
||||
### Utility Functions
|
||||
|
||||
- `UCTselect(node, w)` — Select node using UCT score
|
||||
- `dictify(x; keytype=Any)` — Convert JSON.Object/OrderedDict to plain Dict
|
||||
|
||||
### MCTS Utility Functions
|
||||
|
||||
- `selectBestNextNode(node)` — Select best child node based on value metric
|
||||
- `selectBestTrajectoryNode(node)` — Select best node along optimal trajectory
|
||||
- `backpropagate(node, simTrajectoryReward; kwargs...)` — Backpropagate reward up the tree
|
||||
- `isleaf(node)` — Check if node is a leaf (has no children)
|
||||
- `isroot(node)` — Check if node is the root node
|
||||
- `selectChildNode(node)` — Select child with highest `progressvalue + reward`
|
||||
- `expand(node, transition, transitionargs; kwargs...)` — Generate child nodes
|
||||
- `simulate(node, transition, transitionargs; kwargs...)` — Perform rollout simulation
|
||||
|
||||
### MCTS Node Structure
|
||||
|
||||
```julia
|
||||
@@ -230,7 +264,7 @@ MCTSNode(
|
||||
isterminal::Bool,
|
||||
parent::Union{MCTSNode, Nothing},
|
||||
children::Dict{String, MCTSNode},
|
||||
etc::Dict{String, Any}
|
||||
etc::Dict{Symbol, Any}
|
||||
)
|
||||
```
|
||||
|
||||
|
||||
@@ -1,87 +0,0 @@
|
||||
""" Recursively convert dictionary-like variable (e.g. JSON.Object) into a dictionary.
|
||||
The function walks any nested structure composed of `AbstractDict` (e.g., `JSON.Object`,
|
||||
`Dict`, `OrderedDict`) and `AbstractArray` and produces a new tree where
|
||||
every dictionary-like node is a plain `Dict` and every array-like node is a
|
||||
`Vector{Any}`. Scalar values (numbers, strings, booleans, `nothing`, etc.)
|
||||
are returned unchanged.
|
||||
Does **not** mutate the input; it always allocates new containers.
|
||||
|
||||
# Arguments
|
||||
- `x`
|
||||
Any Julia value. If `x` is an `AbstractDict` it will be converted to a `Dict`;
|
||||
if it is an `AbstractArray` its elements will be processed recursively.
|
||||
|
||||
# Keyword Arguments
|
||||
- `keytype::Type=Any`
|
||||
The key type for the output Dict. Use `String` for `Dict{String,Any}`, `Symbol` for `Dict{Symbol,Any}`, or `Any` to preserve original key types.
|
||||
- `stringkey::Bool=false`
|
||||
If `true`, every dictionary key is converted to `String` via `string(k)`. This parameter is ignored when `keytype` is explicitly set.
|
||||
|
||||
# Return
|
||||
- A newly allocated nested structure composed of `Dict{keytype,Any}` and
|
||||
`Vector{Any}` that mirrors the input shape but uses plain Julia containers.
|
||||
|
||||
# Notes
|
||||
- The function treats any `AbstractDict` as a mapping source, so it works with
|
||||
`JSON.Object`, `Dict`, `OrderedDict`, etc.
|
||||
- Arrays are returned as `Vector{Any}` with their elements processed
|
||||
recursively.
|
||||
|
||||
# Examples
|
||||
```jldoctest
|
||||
julia> using JSON
|
||||
julia> d = Dict(
|
||||
"a" => 4,
|
||||
"b" => 6,
|
||||
"c" => Dict(
|
||||
"d"=>7,
|
||||
:e=>Dict(
|
||||
"f"=>"hey",
|
||||
"g"=>Dict(
|
||||
"world"=>[1, "2", 3, Dict(:dd=>4.7)]
|
||||
)
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
julia jsonstring = JSON.json(d)
|
||||
julia> A1 = JSON.parse(jsonstring) # A1 type is JSON.Object
|
||||
julia> A2 = dictify(A1; keytype=String)
|
||||
Dict{String,Any} with 3 entries:
|
||||
"a" => 4
|
||||
"b" => 6
|
||||
"c" => Dict("d"=>7, "e"=>Dict("f"=>"hey", "g"=>Dict("world"=>[1, "2", 3, 4.7])))
|
||||
|
||||
julia> A3 = dictify(A1; keytype=Symbol)
|
||||
Dict{Symbol,Any} with 3 entries:
|
||||
:a => 4
|
||||
:b => 6
|
||||
:c => Dict(:d=>7, :e=>Dict("f"=>"hey", "g"=>Dict("world"=>[1, "2", 3, 4.7])))
|
||||
|
||||
julia> B1 = dictify(d; keytype=String)
|
||||
Dict{String, Any} with 3 entries:
|
||||
"""
|
||||
function dictify(x; keytype::Type=Any)
|
||||
# Dict-like objects
|
||||
if x isa AbstractDict
|
||||
# choose output key type container
|
||||
out = Dict{keytype,Any}()
|
||||
for (k,v) in x
|
||||
if keytype === String
|
||||
newk = string(k)
|
||||
elseif keytype === Symbol
|
||||
newk = Symbol(string(k))
|
||||
else
|
||||
newk = k
|
||||
end
|
||||
out[newk] = dictify(v; keytype=keytype)
|
||||
end
|
||||
return out
|
||||
# Arrays / vectors: map elements recursively and return a Vector{Any}
|
||||
elseif x isa AbstractArray
|
||||
return [dictify(element; keytype=keytype) for element in x]
|
||||
# everything else: return as-is (primitives, numbers, strings, etc.)
|
||||
else
|
||||
return x
|
||||
end
|
||||
end
|
||||
Reference in New Issue
Block a user