Yahtzee

Building an On-Policy RL Algorithm to play Yahtzee

Setup

install requirements

pip install -r requirements.txt

set up weights and biases account using the quickstart page:

https://docs.wandb.ai/quickstart/

Core Functionality

Training a Model

To train a model, run:

python trainer.py

Running the App

To run the interactive Yahtzee app, use:

python app.py

Optimal Results Model Configuration:

python trainer.py \
    --batch_size 8192 \
    --num_steps 10000 \
    --policy_loss_coefficient 100.0 \
    --value_loss_coefficient 0.01 \
    --entropy_loss_coefficient 1.0 \
    --use_learned_value

Example Runs

Here are example runs using the optimal configuration:

• Run 1 - Example training run with optimal parameters
• Run 2 - Alternative training run with optimal parameters

All experiments can be compared on the Weights & Biases project page.

TODO

• Clean up the State class to group features into dicts
• Implement UI for calculation mode
• Model saving / Loading
• Model Store via hugging face
• Experiment Management / Comparison Improvements

References

• Reinforcement Learning for Yahtzee - Explores using Deep Q-Learning and Policy Gradient methods to train an AI agent to play Yahtzee
• Optimal Play in Yahtzee - Mathematical analysis of optimal Yahtzee strategies and expected values
• Yahtzee Q-Learning Implementation - Example implementation of Q-learning applied to Yahtzee