rmalde/minecraft-agent

rmalde’s Minecraft agent splits planning and acting across two different models. GPT-6 Astra (or GPT-5.6 Sol) sets the plan, and Jev selects the actual in-game action at each step, connecting to the official vanilla Minecraft Java 1.16.5 server through Mineflayer, a JavaScript library for controlling Minecraft players, so every selected action goes through the normal player protocol. A read-only Java sensor reports the dragon head position without changing game state.

Each time the agent needs to act, the code builds a state from the game observation and a list of available actions, each with a text description, and asks Jev to choose the one that best advances Astra’s current objective, using the choice primitive. The prompt itself carries detailed play instructions, such as escaping a breath-attack cloud before doing anything else, or preferring a boat for long water crossings.

In the project’s latest verified run, the agent went from an empty inventory to defeating the ender dragon in 8 minutes 43 seconds, 40% faster than the previous 14 minute 32 second run, with the End combat stage dropping from 332 seconds to 152. The run used 131 Jev decisions against 35 Astra planning calls, with no deaths, restarts, or manual intervention, and passed all 17 automated run checks plus 29 local tests. Camera movement is capped at 240 degrees per second, and the full run is recorded on video with an overlay showing the model names, elapsed time, coordinates, and each selected action.

How it uses Jev

state=compactObservation(state);
const criteria=Object.fromEntries(options.map((o,i)=>['a'+i,o.description]));
const body={model:'typesafe/jev-1.13',state:JSON.stringify(state),questions:{action:{type:'choice',instructions:'Control the Minecraft player. Choose one available action that best advances the current planner objective. FIRST choose an offered escape action when a breath cloud threatens the player. Clouds marked safeAtCurrentHeight are vertically separated on verified ground; distance alone does not make them dangerous. Never eat or wait inside a breath cloud. After reaching safety, if health is below 16, eat available food until the food bar is full so health can regenerate. Survival has priority over item reserve targets. Prefer a boat for long water crossings. Craft one before departure, place and board it at water, then row. Dismount and recover it at land. Swim only for short approaches; use side detours when blocked. Never pathfind along the sea floor. Check current inventory; do not keep crafting or collecting after the target is met. Avoid failed actions and needless waiting. Movement and mining options include standard pathfinding, but you own the choice.',criteria}}};
const r=await request('/api/alpha/decisions',body);const answer=r.data.answers?.action;
const selected=options[Number(answer?.choice?.slice(1))];
if(!selected||!Object.hasOwn(criteria,answer.choice))throw new Error('Invalid JEV action');
return {...r,selected,body};

View in models.mjs

Open on GitHub