Podcast: Play in new window | Download (Duration: 4:43 — 2.9MB) | Embed
Neural Turing machines Traditional programming methods are very good at solving problems that have simple rules to apply. They’re not so good when there are no simple rules that can be used, or when the rules are unknown. Neural networks are very good at problems that have complex or poorly defined rules, but not so …
Podcast: Play in new window | Download (Duration: 6:34 — 4.0MB) | Embed
Long short-term memory In episode 117, I expressed some concern. It seemed that neural network implementations lacked a way of holding onto information over time. It turns out that the problem has been addressed by recurrent neural networks. Recurrent networks remember, though not very well. Today, we look at a modification of recurrent networks that …
Podcast: Play in new window | Download (Duration: 4:32 — 2.7MB) | Embed
Robotic dreaming When you are awake, the world comes in at you through your senses. When you are asleep and dreaming, you create a world from within. An algorithm for deep learning, called “the wake sleep algorithm,” seems to capture this behavior. I referenced the previous episode in this one, so you may as well …
I need to make a couple of notes before I forget. I went through and put in code to set b to 1 or -1 on the methods that needed it. I was going to have write and paste, both inner and outer return a -1 if they were called while the buffer was empty. …
It’s Saturday night, and I’ve got some coding to do. I’d really like to get a figure to self-reproduce tonight, but I don’t know if I can pull it off or not. Last time, in the middle of creating the realm, I realized that there was and is some stuff that needs doing with the …
Podcast: Play in new window | Download (Duration: 5:47 — 3.6MB) | Embed
Sleep and dreams There are two types of sleep: rapid eye movement or REM sleep, and non-rapid eye movement, or non-REM. Dreams happen during both types of sleep, and there is a well-established link between the amount and quality of sleep you get, and how well you recall and/or learn. Today, we take a little …
Podcast: Play in new window | Download (Duration: 6:00 — 3.7MB) | Embed
Sleep, reset and brain wash While you are sleeping, your brain performs a reset of sorts. Synaptic weights that increased over the course of the day decrease while you are sleeping. At the same time, the fluid your brain floats in, rushes through your brain tissue, clearing out wastes that couldn’t be removed over the …
Have the output of a test run. memory={-6, -14, -1, -1, -1, -1, -1, -1, -1, -1, -1} size=11 Size=8 Buffer size=3 memory={-1, -1, -1, -1, -1, -1, -1, -1} buffer memory={-6, -14, -1}
Podcast: Play in new window | Download (Duration: 5:34 — 3.5MB) | Embed
Bit seat drivers Deep learning algorithms, and neural networks in general, require much more training than humans do. They are unable to generalize well enough to handle situations not covered in the training data, and can be thrown off by things that a human wouldn’t even notice. Today we look at these challenges by examining …
Podcast: Play in new window | Download (Duration: 3:50 — 2.5MB) | Embed
do we need something else, or just more? Though deep learning has had some promising results, there are still some things that it simply doesn’t do well at. There are other algorithms that do as well or better at certain tasks. On the other hand, we’ve only been able to implement comparatively small neural networks. …