Skip to content
Nathan Zhou

The CartPole lab

CartPole is where I started with reinforcement learning: a cart on a track, a pole hinged on top, and a push left or right fifty times a second to keep it up. It runs here in your browser with the same physics as Gymnasium’s CartPole-v1. Watch the agents I trained, try it yourself, or train a new one and see it learn.

Agent

The network dqn_train.py saved after 500 episodes, with its real weights. It nearly always lasts all 500 steps; once in a while the cart drifts off the end of the track just before.

Step 0 of 500

Episode 1

Push the cart yourself with the buttons, or click the picture and use ← →. One push is 20 newtons for 0.06 seconds: the DQN rides out one, but two close together can topple it.

What it sees

Cart position
0.01 m
Cart speed
-0.05 m/s
Pole angle
0.2°
Pole turning
3°/s

What it expects from each push

Push left
55.9its choice
Push right
55.9

The points it expects to follow; it takes the higher.

Recent scores: –

How it works

The physics are Gymnasium’s, step for step, and checked against it. The two agents are the networks I saved, run with the same arithmetic as PyTorch. Training uses my scripts’ settings, and nothing leaves your browser.

← Back to Learning RL by Playing Games