update
This commit is contained in:
@@ -68,10 +68,22 @@ MCTSNode(
|
||||
isterminal::Bool,
|
||||
parent::Union{MCTSNode, Nothing},
|
||||
children::Dict{String, MCTSNode},
|
||||
etc::Dict{String, Any}
|
||||
etc::Dict{Symbol, Any}
|
||||
)
|
||||
```
|
||||
|
||||
**Fields:**
|
||||
- `nodekey::String` — Unique identifier for the node
|
||||
- `state::Dict` — Current state represented as a dictionary
|
||||
- `visits::Integer` — Number of times this node has been visited
|
||||
- `progressvalue::Number` — LLM's estimate of state quality
|
||||
- `statevalue::Number` — Average cumulative reward from simulations
|
||||
- `reward::Number` — Immediate reward at this node
|
||||
- `isterminal::Bool` — Whether this node represents a terminal state
|
||||
- `parent::Union{MCTSNode, Nothing}` — Parent node reference (nothing for root)
|
||||
- `children::Dict{String, MCTSNode}` — Mapping of child nodes
|
||||
- `etc::Dict{Symbol, Any}` — Additional arbitrary data storage (uses Symbol keys)
|
||||
|
||||
### Understanding `progressvalue`, `statevalue`, and `reward`
|
||||
|
||||
| Field | Source | Purpose |
|
||||
@@ -212,11 +224,33 @@ Run simulation from a node and backpropagate the reward. Returns `nothing`.
|
||||
- `multithread::Bool=false` — Enable multithreading
|
||||
- `highValueState` — Channel to store high-value states
|
||||
|
||||
#### `backpropagate(node, simTrajectoryReward; kwargs...)`
|
||||
|
||||
Backpropagate reward along the simulation chain. Updates visit counts and state values for all nodes along the path to the root. Returns `nothing`.
|
||||
|
||||
**Arguments:**
|
||||
- `node::MCTSNode` — The leaf node from which to start backpropagation
|
||||
- `simTrajectoryReward::Number` — The total reward from the trajectory simulation
|
||||
|
||||
**Keyword Arguments:**
|
||||
- `discountRewardCoeff::AbstractFloat=0.9` — Discount coefficient applied to future rewards
|
||||
|
||||
### Utility Functions
|
||||
|
||||
- `UCTselect(node, w)` — Select node using UCT score
|
||||
- `dictify(x; keytype=Any)` — Convert JSON.Object/OrderedDict to plain Dict
|
||||
|
||||
### MCTS Utility Functions
|
||||
|
||||
- `selectBestNextNode(node)` — Select best child node based on value metric
|
||||
- `selectBestTrajectoryNode(node)` — Select best node along optimal trajectory
|
||||
- `backpropagate(node, simTrajectoryReward; kwargs...)` — Backpropagate reward up the tree
|
||||
- `isleaf(node)` — Check if node is a leaf (has no children)
|
||||
- `isroot(node)` — Check if node is the root node
|
||||
- `selectChildNode(node)` — Select child with highest `progressvalue + reward`
|
||||
- `expand(node, transition, transitionargs; kwargs...)` — Generate child nodes
|
||||
- `simulate(node, transition, transitionargs; kwargs...)` — Perform rollout simulation
|
||||
|
||||
### MCTS Node Structure
|
||||
|
||||
```julia
|
||||
@@ -230,7 +264,7 @@ MCTSNode(
|
||||
isterminal::Bool,
|
||||
parent::Union{MCTSNode, Nothing},
|
||||
children::Dict{String, MCTSNode},
|
||||
etc::Dict{String, Any}
|
||||
etc::Dict{Symbol, Any}
|
||||
)
|
||||
```
|
||||
|
||||
|
||||
Reference in New Issue
Block a user