This commit is contained in:
2026-06-30 12:20:18 +07:00
parent 73f769d13b
commit 0ae28b28c0
2 changed files with 36 additions and 89 deletions
+36 -2
View File
@@ -68,10 +68,22 @@ MCTSNode(
isterminal::Bool,
parent::Union{MCTSNode, Nothing},
children::Dict{String, MCTSNode},
etc::Dict{String, Any}
etc::Dict{Symbol, Any}
)
```
**Fields:**
- `nodekey::String` — Unique identifier for the node
- `state::Dict` — Current state represented as a dictionary
- `visits::Integer` — Number of times this node has been visited
- `progressvalue::Number` — LLM's estimate of state quality
- `statevalue::Number` — Average cumulative reward from simulations
- `reward::Number` — Immediate reward at this node
- `isterminal::Bool` — Whether this node represents a terminal state
- `parent::Union{MCTSNode, Nothing}` — Parent node reference (nothing for root)
- `children::Dict{String, MCTSNode}` — Mapping of child nodes
- `etc::Dict{Symbol, Any}` — Additional arbitrary data storage (uses Symbol keys)
### Understanding `progressvalue`, `statevalue`, and `reward`
| Field | Source | Purpose |
@@ -212,11 +224,33 @@ Run simulation from a node and backpropagate the reward. Returns `nothing`.
- `multithread::Bool=false` — Enable multithreading
- `highValueState` — Channel to store high-value states
#### `backpropagate(node, simTrajectoryReward; kwargs...)`
Backpropagate reward along the simulation chain. Updates visit counts and state values for all nodes along the path to the root. Returns `nothing`.
**Arguments:**
- `node::MCTSNode` — The leaf node from which to start backpropagation
- `simTrajectoryReward::Number` — The total reward from the trajectory simulation
**Keyword Arguments:**
- `discountRewardCoeff::AbstractFloat=0.9` — Discount coefficient applied to future rewards
### Utility Functions
- `UCTselect(node, w)` — Select node using UCT score
- `dictify(x; keytype=Any)` — Convert JSON.Object/OrderedDict to plain Dict
### MCTS Utility Functions
- `selectBestNextNode(node)` — Select best child node based on value metric
- `selectBestTrajectoryNode(node)` — Select best node along optimal trajectory
- `backpropagate(node, simTrajectoryReward; kwargs...)` — Backpropagate reward up the tree
- `isleaf(node)` — Check if node is a leaf (has no children)
- `isroot(node)` — Check if node is the root node
- `selectChildNode(node)` — Select child with highest `progressvalue + reward`
- `expand(node, transition, transitionargs; kwargs...)` — Generate child nodes
- `simulate(node, transition, transitionargs; kwargs...)` — Perform rollout simulation
### MCTS Node Structure
```julia
@@ -230,7 +264,7 @@ MCTSNode(
isterminal::Bool,
parent::Union{MCTSNode, Nothing},
children::Dict{String, MCTSNode},
etc::Dict{String, Any}
etc::Dict{Symbol, Any}
)
```