The trials and tribulations of training a physical robot with reinforcement learning

This is a follow up to my article “Training a real robot to play Puckworld with reinforcement learning”. In that one, to make it a little punchier, I showed the overview and end results of the project, but left out the insane number of little hurdles and decisions I had to figure out.

So this article will be about those details instead, partly for me to justify the pain, but maybe more charitably to show that for any project with a neat (hopefully?) presentation, there’s probably a harrowing saga of hair-pulling roadblocks behind it. Here, it’s roughly in the order I encountered things. There are lots I’m leaving out too. read more

In case hexapods weren’t creepy enough: the centipede robot!

Similar to…most of? my ideas, I don’t remember why I thought of this. I think after I made the reinforcement learning robot, I was on a robot kick, and came up with this. Hexapods are of course a robot classic, but I don’t think I had ever seen a centipede robot.

Why a centipede? Well… I can make up a few “practical” reasons: because of its length, it could potentially bridge gaps, or bend “upwards” to have height, or possibly even climb. But the real reason is because they haven’t been done that much and I thought it would be cool, funny, and creepy. read more

The Mostly Printed CNC (MPCNC), mostly

I’m not sure when I got the urge to make a CNC… or maybe it was always there. I did a summer job in a machine shop when I was 19, where I was given a minimum of training by the 88 year old machinist: “keep the pink things [wiggling his fingers] away from the sharp things [pointing to the milling machine’s cutting edge], boy.”

(I’m not joking. He did show me more things later, but only because I asked. He would mostly just motion for me to come over so he could tell me filthy jokes from the silent film era and cackle to himself.) read more

First project with the new 3D printer: a TOF sensor mount

I’m pretty late to hop on the 3D printing bandwagon, but I heard its siren song and couldn’t stay away much longer!

After the briefest search online and asking a few friends, I decided to go with the Monoprice Delta Mini. My main reasons were that I didn’t want to have to tinker and build much (at least to start), I wanted to be able to get decent quality, I didn’t want to spend a ton, and I don’t especially need a large print size. The MPDM matched all of this (supposedly works out of box, can do 0.05 mm layer height, 160 bucks, 4″ height by 3″ diameter print volume), so I went for it. read more