Training a real robot to play Puckworld with reinforcement learning

After I trained an agent to play “puckworld” using Q-learning, I thought “hey, maybe I should make a real robot that learns this. It can’t be that hard, right?”

Hooooooooo boy. I did not appreciate how much harder problems in the physical world can be. Examples of amateurs doing Reinforcement Learning (RL) projects are all over the place on the internet, and robotics are certainly touted as one of the main applications for RL, but in my experience, I’ve only found a few examples of someone actually using RL to train a robot. Here’s a (very abridged!) overview of my adventure getting a robot to learn to play a game called puckworld. read more

The Red Lama (Red Llama clone)

After making the worst fuzz pedal ever (that’s for another post) and Orange Ya Glad (which was fine, but didn’t add quite as much fuzz as I wanted and adds a weird buzz even when you’re not playing on some speakers), I just wanted a normal fuzz pedal. After doing a bit of reading, I found that the Red Llama overdrive pedal (by Way Huge) is a classic, and after watching a few YouTube demos, it seemed good (to be honest, people are crazy about the “different” sounds of various fuzz/distortion/overdrive that various antique/obscure transistors or configurations will give you, but they all sound pretty similar to me, and I suspect people think they’re hearing differences more often than there actually are).

Anyway, I wanted to tribute the original Red Llama circuit I was cloning, so I went for… read more

A spooOOOOoooky project!


This is a fun one.

It’s also a testament to how nifty and easy it is to quickly whip up a project with Arduinos, provided you have enough of a “critical mass”, as I’ve called it before, of other stuff that you might end up needing. read more

Low power Arduinos, part 1


As part of an ongoing project, I wanted to see how low I could get the power consumption of Arduinos to go. The reason is as follows. When getting back into Arduinos a few months ago, I wanted to try a telemetry project of some sort, collecting data remotely and sending it back. Ideally, the idea would be to collect data from different places and analyze the aggregate in some cool way, but that’s a story for another post.

The point I was going for, though, is that I wanted to put these Arduinos in places that wouldn’t have constant access to power, so that already means using a battery. Using a battery to power an Arduino isn’t a big deal (plenty of people do it for portable projects), but once you’re looking at long term powering without recharging, it’s a different story. read more

Servo controller box


Woowee, this is a long one.

This is actually something I did for my job. Here’s the deal: I have this electrochemical bath that has a sample in it; one electrode is a piece of graphite, the other electrode is the sample itself. Don’t worry about what the bath does for now, but it’s important that the longer the sample is in the bath, the more the effect of the bath is on the sample. It’s pretty much linear with time. read more