Podcast: Play in new window | Download (Duration: 5:47 — 3.6MB) | Embed

Subscribe: Email | RSS | More

Sleep and dreams

There are two types of sleep: rapid eye movement or REM sleep, and non-rapid eye movement, or non-REM. Dreams happen during both types of sleep, and there is a well-established link between the amount and quality of sleep you get, and how well you recall and/or learn. Today, we take a little peak at what happens in the brain while you sleep and dream.

Here’s a link to a panel discussion on sleep and dreams. The part I talk about in this episode starts at roughly 22 minutes and 22 seconds in.

The Mind After Midnight: Where Do You Go When You Go to Sleep?

Here are a couple of articles about the studies done with rats and their dreams.

Rats May Dream, It Seems, Of Their Days at the Mazes

Rats dream about their tasks during slow wave sleep

Here’s a link to an article about memory, and the types of dreams that occur during REM and non-REM sleep.

Memory, Sleep and Dreaming: Experiencing Consolidation

Podcast: Play in new window | Download (Duration: 6:00 — 3.7MB) | Embed

Subscribe: Email | RSS | More

Sleep, reset and brain wash

While you are sleeping, your brain performs a reset of sorts. Synaptic weights that increased over the course of the day decrease while you are sleeping. At the same time, the fluid your brain floats in, rushes through your brain tissue, clearing out wastes that couldn’t be removed over the course of the day.

Here’s a video and an article about how wastes are cleared away during your sleep.

One more reason to get a good night’s sleep

How Sleep Clears the Brain

Here’s an article on the link between synapse size and synaptic weight—the strength of the signal that comes from a given synapse.

The Secret to the Brain’s Memory Capacity May Be Synapse Size

And here’s an article about how the size of synapses shrink during sleep.

How Sleep Resets the Brain

Podcast: Play in new window | Download (Duration: 5:34 — 3.5MB) | Embed

Subscribe: Email | RSS | More

Bit seat drivers

Deep learning algorithms, and neural networks in general, require much more training than humans do. They are unable to generalize well enough to handle situations not covered in the training data, and can be thrown off by things that a human wouldn’t even notice. Today we look at these challenges by examining what it takes to train a neural network to drive a car.

Here are a couple of links about training self-driving vehicles.

Edge case training and discovery are keeping self-driving cars from gaining full autonomy

Training AI for Self-Driving Vehicles: the Challenge of Scale

Here’s a short video demo and an article about how AI image recognition can be fooled by things that wouldn’t fool many animals.

Adversarial Patch

Google ‘optical illusion’ stickers make AI hallucinate

Podcast: Play in new window | Download (Duration: 3:50 — 2.5MB) | Embed

Subscribe: Email | RSS | More

do we need something else, or just more?

Though deep learning has had some promising results, there are still some things that it simply doesn’t do well at. There are other algorithms that do as well or better at certain tasks. On the other hand, we’ve only been able to implement comparatively small neural networks. Perhaps, if we could simulate larger networks, deep learning or an algorithm like it could do what it currently cannot.

Here’s a link to a paper by Gary Marcus, providing a critical review of deep learning and suggesting that it may have to be combined with other approaches to create a general intelligence.

Deep Learning: A Critical Appraisal

Podcast: Play in new window | Download (Duration: 6:42 — 4.0MB) | Embed

Subscribe: Email | RSS | More

Use of graphics cards for neural networks

Your graphics card holds a GPU—graphics processing unit. Unlike your CPU, which does one operation at a time, the GPU does many simple operations on many numbers at once. This allows your computer to run your favorite game, and it can be used to run artificial neural networks, and implement deep learning.

Here’s an amusing short video that demonstrates the difference between the sequential CPU approach, and the parallel processing of a GPU.

Mythbusters Demo GPU versus CPU

Podcast: Play in new window | Download (Duration: 6:14 — 3.9MB) | Embed

Subscribe: Email | RSS | More

Diving into deep learning

Neural networks have had their ups and downs. However, in the last decade or so, they’ve really taken off. There were a couple of breakthroughs that made all the difference. Today we look at one of them, called “Deep Learning.”

Here’s a 4-post long blog series on deep learning concepts and history.

A ‘Brief’ History of Neural Nets and Deep Learning

Here are a couple of talks on deep learning, and some of the things it has been able to do.

The wonderful and terrifying implications of computers that can learn

The Deep End of Deep Learning