pip install -r requirements.txt
Put your OpenAI API key from https://platform.openai.com/account/api-keys and the organization id from https://platform.openai.com/account/org-settings
to a file named apikey.json containing following:
{
"apikey": "yourapikeyfromopenai",
"org": "yourorganizationid",
"model": "gpt-3.5-turbo"
}
Newton chatbot achieves a performance that is not extremely good, but is significantly better than random walk.
Mean scores:
This is expected, as zero-shot CartPole is an unsolved problem, it is a difficult task for humans as well, and chatbots tend to be bad at mechanical tasks.
Example result for one partial episode:
[
{
"content" : "You are Sir Isaac Newton. Your task is to balance a cartpole.\nYou are effecting forces on the cart to make it accelerate either left or right.\nThere is a pole set up on top of the cart so that it can fall freely either left or right.\nIf the pole is falling right, you need to push the cart right to let the pole rise to an upright position again, and vice versa.\nYour task is to make the pole stay upright by pushing the cart left and right.\nYou must always push the cart left or right, those are your only options.\nThe cart is controlled by commands \"left\" and \"right\". Answer only left or right. Before giving a command, the description of the state\nof the cart is given as follows:\ncart-position: [left|center|right]\ncart-velocity: [leftwards|stopped|rightwards]\npole-angular-velocity: [leftwards|zero|rightwards]\npole-angle: [left|upright|right]\nHere is the first state of the cart, answer only left or right:\n",
"role" : "system"
},
{
"content" : "cart-position: center\ncart-velocity: stopped\npole-angular-velocity: zero\npole-angle: right",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: stopped\npole-angular-velocity: leftwards\npole-angle: right",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: rightwards\npole-angular-velocity: leftwards\npole-angle: right",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: rightwards\npole-angular-velocity: leftwards\npole-angle: right",
"role" : "user"
},
{
"content" : "left",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: rightwards\npole-angular-velocity: leftwards\npole-angle: upright",
"role" : "user"
},
{
"content" : "left",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: stopped\npole-angular-velocity: leftwards\npole-angle: upright",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: rightwards\npole-angular-velocity: leftwards\npole-angle: upright",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
},
{
"content" : " right",
"role" : "assistant"
},
{
"content" : "cart-position: left\ncart-velocity: stopped\npole-angular-velocity: rightwards\npole-angle: right",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
},
{
"content" : "You failed, the pole fell too far. Try again:",
"role" : "user"
},
{
"content" : "cart-position: center\ncart-velocity: stopped\npole-angular-velocity: zero\npole-angle: right",
"role" : "user"
},
{
"content" : "left",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: leftwards\npole-angular-velocity: rightwards\npole-angle: right",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: stopped\npole-angular-velocity: zero\npole-angle: right",
"role" : "user"
},
{
"content" : "left",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: leftwards\npole-angular-velocity: rightwards\npole-angle: right",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: stopped\npole-angular-velocity: zero\npole-angle: right",
"role" : "user"
},
{
"content" : "left",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: leftwards\npole-angular-velocity: rightwards\npole-angle: right",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: stopped\npole-angular-velocity: rightwards\npole-angle: right",
"role" : "user"
},
{
"content" : "left",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: leftwards\npole-angular-velocity: rightwards\npole-angle: right",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: stopped\npole-angular-velocity: rightwards\npole-angle: right",
"role" : "user"
},
{
"content" : "left",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: leftwards\npole-angular-velocity: rightwards\npole-angle: right",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
},
{
"content" : "You failed, the pole fell too far. Try again:",
"role" : "user"
},
{
"content" : "cart-position: center\ncart-velocity: stopped\npole-angular-velocity: zero\npole-angle: left",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: rightwards\npole-angular-velocity: leftwards\npole-angle: left",
"role" : "user"
},
{
"content" : "left",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: stopped\npole-angular-velocity: zero\npole-angle: left",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: rightwards\npole-angular-velocity: leftwards\npole-angle: left",
"role" : "user"
},
{
"content" : "left",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: stopped\npole-angular-velocity: zero\npole-angle: left",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: rightwards\npole-angular-velocity: leftwards\npole-angle: left",
"role" : "user"
},
{
"content" : "left",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: stopped\npole-angular-velocity: leftwards\npole-angle: left",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: rightwards\npole-angular-velocity: leftwards\npole-angle: left",
"role" : "user"
},
{
"content" : "left",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: stopped\npole-angular-velocity: leftwards\npole-angle: left",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
},
{
"content" : "You failed, the pole fell too far. Try again:",
"role" : "user"
},
{
"content" : "cart-position: center\ncart-velocity: stopped\npole-angular-velocity: zero\npole-angle: upright",
"role" : "user"
},
{
"content" : "left",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: leftwards\npole-angular-velocity: rightwards\npole-angle: right",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: stopped\npole-angular-velocity: zero\npole-angle: right",
"role" : "user"
},
{
"content" : "left",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: leftwards\npole-angular-velocity: rightwards\npole-angle: right",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: stopped\npole-angular-velocity: zero\npole-angle: right",
"role" : "user"
},
{
"content" : "left",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: leftwards\npole-angular-velocity: rightwards\npole-angle: right",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: stopped\npole-angular-velocity: zero\npole-angle: right",
"role" : "user"
},
{
"content" : "left",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: leftwards\npole-angular-velocity: rightwards\npole-angle: right",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: stopped\npole-angular-velocity: rightwards\npole-angle: right",
"role" : "user"
},
{
"content" : "left",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: leftwards\npole-angular-velocity: rightwards\npole-angle: right",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: stopped\npole-angular-velocity: rightwards\npole-angle: right",
"role" : "user"
},
{
"content" : "left",
"role" : "assistant"
},
{
"content" : "You failed, the pole fell too far. Try again:",
"role" : "user"
},
{
"content" : "cart-position: center\ncart-velocity: stopped\npole-angular-velocity: zero\npole-angle: upright",
"role" : "user"
},
{
"content" : "left",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: leftwards\npole-angular-velocity: rightwards\npole-angle: right",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: stopped\npole-angular-velocity: zero\npole-angle: right",
"role" : "user"
},
{
"content" : "left",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: leftwards\npole-angular-velocity: rightwards\npole-angle: right",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: stopped\npole-angular-velocity: zero\npole-angle: right",
"role" : "user"
},
{
"content" : "left",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: leftwards\npole-angular-velocity: rightwards\npole-angle: right",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: stopped\npole-angular-velocity: zero\npole-angle: right",
"role" : "user"
},
{
"content" : "left",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: leftwards\npole-angular-velocity: rightwards\npole-angle: right",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: stopped\npole-angular-velocity: rightwards\npole-angle: right",
"role" : "user"
},
{
"content" : "left",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: leftwards\npole-angular-velocity: rightwards\npole-angle: right",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: stopped\npole-angular-velocity: rightwards\npole-angle: right",
"role" : "user"
},
{
"content" : "left",
"role" : "assistant"
},
{
"content" : "cart-position: center\ncart-velocity: leftwards\npole-angular-velocity: rightwards\npole-angle: right",
"role" : "user"
},
{
"content" : "right",
"role" : "assistant"
}
]