Watson's Daily-Double Wagering Strategy
The phrase current game state matters: the same wager amount can have different consequences in different states, so Watson evaluated a wager in the context where it was available.
From Point Amount to Game Outcome
A Daily-Double wager can be judged in two different ways. One approach asks only how many points the wager might add. Watson used a broader question: if this particular legal wager is selected from the current game state, how favorable is the resulting chance of winning the game? That change in viewpoint turns wagering into an action-selection problem.
An action value is an estimate of the probability of winning from the current game state if a particular Daily-Double wager is selected.
Action Value as a Decision
Watson treated each legal wager as an action. For a candidate action, it estimated the value of the game from the current state if that action were taken. The resulting action value therefore refers to a possible decision, not merely to a number of points attached to the bet.
The phrase current game state matters: the same wager amount can have different consequences in different states, so Watson evaluated a wager in the context where it was available.
What the Afterstate Captures
To evaluate one candidate wager, begin with the current state s and select a legal round-dollar bet. That selection creates an afterstate: the game state resulting from choosing the action. The afterstate gives the evaluation a concrete situation to assess instead of leaving the wager as an isolated point amount.
The afterstate is not the final game outcome. It is the resulting game situation used during evaluation. The state value function estimates how favorable that situation is. The complete next game state also depends on which square is selected next, so the more precise description is an expected next afterstate value.
Two Estimates Work Together
Watson needed two kinds of information. First, its learned state value function, written as v-hat with parameters theta, estimated the probability of winning from a game state. During a betting decision, this function estimated the values of the afterstates associated with the legal wagers.
Second, Watson used in-category Daily-Double confidence, written p DD. This estimated the likelihood that Watson would respond correctly to the Daily-Double clue, which had not yet been revealed. The action-value evaluation therefore considered both the value of the resulting situations and the likelihood of a correct response.
State value answers, “How favorable is the resulting situation?” In-category confidence answers, “How likely is a correct response to this unrevealed clue?” The action value brings these considerations together for the chosen wager.
One Candidate Wager
Following one wager through the evaluation
Suppose Watson is in a current game state and is considering one legal round-dollar Daily-Double wager. How is that wager evaluated?
- 1Start at the current state
The evaluation begins with the current game state s, not with the wager amount considered in isolation.
- 2Select the candidate action
Watson treats the legal wager as an action that may be taken from state s.
- 3Construct the afterstate
Selecting the wager creates the afterstate: the game situation resulting from that action.
- 4Estimate the resulting value
The learned state value function estimates how favorable the resulting afterstate is, using the more precise idea of an expected next afterstate value because the next square also matters.
- 5Account for response confidence
Because the Daily-Double clue has not yet been revealed, in-category Daily-Double confidence represents the estimated likelihood of a correct response.
- 6Assign an action value
These estimates produce an action value for the candidate wager: an estimated probability of winning from the current state if that wager is selected.
The candidate is evaluated as a possible action with possible response outcomes, rather than only as a possible point increase.
This sequence explains why the afterstate and confidence estimate belong in the same evaluation. A bet changes the situation, and the unrevealed clue introduces uncertainty about the response. Watson's action value represents the estimated winning chance after taking both features into account.
Comparing the Legal Bets
Watson repeated the evaluation for the legal bets available in the current state. Each candidate received an estimated action value. The basic selection rule was to choose the legal bet with the greatest action value.
A Generated Comparison
| Candidate | Evaluation focus | Decision meaning |
|---|---|---|
| Wager A | Its resulting afterstate and response confidence are evaluated | One possible action value |
| Wager B | Its resulting afterstate and response confidence are evaluated | A second possible action value |
| Wager C | Its resulting afterstate and response confidence are evaluated | A third possible action value |
Generated illustration of the repeated comparison process; these are not reported Watson wager values.
Choosing the best estimated action
Imagine three legal wagers are available. How should a learner describe Watson's comparison without reducing the decision to the largest point amount?
- 1Evaluate Wager A
Consider the afterstate created by Wager A and use the learned state value estimate together with in-category Daily-Double confidence to obtain its action value.
- 2Evaluate Wager B
Perform the same evaluation for Wager B. Its action value depends on its own resulting situation and the same kind of response-confidence estimate.
- 3Evaluate Wager C
Perform the same evaluation for Wager C rather than assuming that its point amount determines its value.
- 4Compare the results
Compare the three estimated action values, because each represents an estimated chance of winning from the current state under that candidate action.
- 5Select the candidate
Choose the legal wager with the greatest action value unless a risk-abatement measure changes the basic choice.
The decision is an action-value comparison across legal bets, followed by selection of the largest estimate subject to possible risk abatement.
Mistakes in Reading the Strategy
Common mistake Treating the action value as the number of points Watson expects to gain.
What it looks likeDescribing a wager as valuable only because it could add many points.
Why it failsThe action value is an estimate of the probability of winning from the current state if the wager is selected.
Do this insteadDescribe the wager as an action whose effect on the chance of winning is being estimated.
Common mistake Ignoring the afterstate.
What it looks likeJumping directly from a legal bet to a decision without describing the resulting game situation.
Why it failsThe state value function is used to estimate the values of the afterstates produced by candidate wagers.
Do this insteadFollow each candidate bet forward to the afterstate it creates before discussing its estimated value.
Common mistake Using only the learned state value function.
What it looks likeAssuming the resulting situation is enough to evaluate the wager even though the clue has not been revealed.
Why it failsThe evaluation also uses in-category Daily-Double confidence to estimate the likelihood of a correct response.
Do this insteadMention both the value of the resulting situations and the confidence of responding correctly.
Common mistake Assuming the largest wager must be selected.
What it looks likeSelecting the candidate with the greatest point amount without comparing action values.
Why it failsWatson's basic choice was the legal bet with the greatest estimated action value, with possible changes from risk-abatement measures.
Do this insteadCompare the estimated action values for the legal bets.
Practice the Evaluation Pipeline
Explain how Watson would evaluate two legal Daily-Double wagers from the same current game state. Your explanation should use the terms current state, afterstate, learned state value function, in-category Daily-Double confidence, action value, and greatest estimated action value.
Hints
- Begin by identifying the legal wagers available from the current state.
- Explain that each wager creates its own afterstate.
- Describe how the state value function and response confidence contribute to each candidate's evaluation.
- End by stating the basic selection rule and its risk-abatement qualification.
What do you think happens?
If one legal wager has a larger point amount but another has a greater estimated action value, which wager does the basic strategy select?
Reveal answer
The wager with the greatest estimated action value.
Watson evaluated wagers as actions and used the basic choice rule of selecting the legal bet with the greatest action value, although risk-abatement measures could alter that choice.
The Decision in Brief
- 1An action value estimates the probability of winning from the current game state if a particular Daily-Double wager is selected.
- 2A legal wager creates an afterstate, which is the resulting game situation evaluated by the learned state value function.
- 3In-category Daily-Double confidence represents the estimated likelihood of responding correctly to the unrevealed clue.
- 4Watson computed an action value for each legal wager, compared those values, and basically selected the wager with the greatest estimate.
- 5Risk-abatement measures could modify the basic greatest-action-value choice.
Key Takeaways
- Watson treated a Daily-Double wager as an action whose value concerns the chance of winning, not merely the number of points at stake.
- The learned state value function evaluated the afterstates created by possible wagers.
- In-category Daily-Double confidence accounted for the likelihood of a correct response to the unrevealed clue.
- Watson compared the action values of legal wagers and basically selected the greatest one, subject to possible risk-abatement measures.