It’s called, the Tetris effect.

People who have played the game Tetris often find that they will start to see those falling blocks as they doze off, as they are just waking up, or even when they blink from time to time as they go about their lives. Though named after one rather addictive video game, the effect isn’t limited to that game. In fact, just about any activity can leave echoes playing in your mind. If you spend the day playing on the beach, you can find yourself feeling as though your body is being pushed and pulled by waves as you fall asleep that night, an echo off the sun and fun you had while playing in the shallows and serf. In similar fashion, a day on a small boat can leave the solid ground seemingly shifting and rolling under your feet, or if you play drums all damn day, just about every sound you hear starts to seem like it’s part of a persistent and complex rhythm.

Here’s a two-and-a-half minute YouTube video that both covered the subject, and made me smile.

The Tetris Effect

This morning, I was working with the vOICe, yet again. As I mentioned in the first post on the subject, I was getting some of those visual qualia again. This time, sometimes when I saw the target, where I saw it was rather far away from where the little white cube actually was. Both the occurrence of visual qualia and the accuracy of said qualia correlated with how well I was able to trust the back of my head. That is to say, the harder I tried, the worse it went.

After spending roughly 30 minutes with it, I moved on with my life. Later, I was on my computer, looking up something entirely different. That’s when I noticed that in the back of my head, I was hearing the sounds of the vOICe, including the beep like sound that corresponds to the presence of the cube. In fact, once, as I leaned back in my chair, puffing my air out through my lips, letting my mind rest for a moment after reading whatever the hell I’d just been reading, I noticed that I both heard it, and sort of saw it, in the same rather vague shadow like way that I see the target when I get visual qualia. For a moment, the “sight” of the cube moved up and down and side to side, and the sound of the cube did the same within the sound scape. It’s much like having a song stuck in your head, or seeing scenes from a movie you’ve watched recently behind your eyes.

I suspect this is my brain, replaying the sensory memory in order to train my neurons to be able to model the real-world phenomenon. A quick Google search just before I wrote this and I find that most researchers seem to think the same thing.

Go brain go!

I was chatting on Twitter back and forth with @seeingwithsound about the vOICe. My Twitter handle is @lobbyandlab if you’d like to join the conversation. I mentioned that my echo location seemed to have improved since I began working with the vOICe. @seeingwithsound replied that though it was only anecdotal evidence, he’d noticed that his sensitivity to subtle sounds seems to have increased.

Wherein it is still anecdotal evidence for you and the rest of the internet, it is direct evidence for me.

Quiet things seem to be subjectively louder. I’m not certain that’s an accurate way of phrasing it–what I do know is that I’ve been mildly surprised by a soft sound from time to time. It happened this morning when I rolled over in bed, and heard something that made me stop and go, what was that? It turned out just to be the sound of one blanket rubbing against the other, a sound that normally doesn’t intrude its way into my awareness. Another time I was unconsciously tapping my toe, and again, my focus was involuntarily drawn to the surprisingly loud seeming sound of my sock-clad foot moving around in my shoe.

Presumably, there has been an increase in the firing rate of some of my auditory nerves, or those that receive auditory nerve signals. I may also have formed more connections, so that more neurons are firing in concert for a given mild sonic stimulus.

I wonder if anyone has figured out how the brain judges the relative volume of sounds. I’ll have to look it up.

Over the weekend, I did a good deal of research. I was trying to track down which parts of the brain may interfere with certain skills. But I wanted more than just which bit does what; I wanted to know the subjective feel. How does it feel when that part of the brain is active, and can I consciously inhibit that part of the brain in order to improve performance?

I’ve been working with something called the “vOICe.” For background, you can checkout episode 19. Based on what I’ve been learning, I decided to slightly modify an exercise from the vOICe training manual—reach and grab.

If you consider mnemonic techniques, the approaches involve attaching things that are difficult for your mind to recall, to things that are easy. you tie the new information to structures within your brain/mind that already are in place. It occurred to me that I might be able to do that with the soundscapes. Perhaps, I could use methods that are already in place within my brain to help me keep track of where objects are in relation to my body while absorbing the new source of information, so that the new form of processing could piggyback on methods that are already in place.

Thus, I would set the target I was reaching for, “A small cube of white modeling clay,” down on the surface of my bed. Then I would use the vOICe to listen to where it was, when I already knew where it was. I did this multiple times, placing the target in different places, listening with the camera at different angles, and reaching out to touch the target with either hand.

When I’d toss the cube so that it would bounce slightly and land somewhere I wasn’t aware of, I’d find it in the soundscape, and then reach out for it. If I missed, even by just a little, instead of picking it up, I’d step back, look at it with the vOICe again, and reach out to touch it several times. It’s much the same as setting down the target somewhere on purpose, only it starts without knowing exactly where the target is, until you’ve already touched it.

Going by the notion that my forebrain might interfere with the sensory processing that usually takes place mostly in the back brain, I changed the way I listen to the soundscapes. I don’t try and hear where it is. I don’t focus too strongly on the soundscape at all. Instead, I’m simply aware of the soundscape, and I avoid trying to understand how and why it begins to work.

I was able to have some visual qualia. Visual qualia is the way that things look like they look; the steady seeming sensation of actually seeing something, and it turns out to be independent of the way the information reaches your mind.

I’d be listening to the voice, and after a time, like seeing a little spark or glint, I’d “see” the little cube appear in my view. Interestingly, I often missed by a tiny bit, even when I’d “seen” the target, by about the same amount I’d tend to miss without the visual qualia. When that happened, sometimes the little image would jump slightly as I noticed where it really was, but more often, it would simply vanish.

There were other times when the subjective “sight” of the target was completely correct, and I could just reach out and grab it.

Unfortunately, I haven’t been keeping track of my error rate, and I have nobody around to hold the clipboard, as it were.

I’m putting terms like “look,” and “see,” in quotes, because my eyes no longer work well enough to provide the “sight” of the target. However, as I had enough vision in the past to know what looking at something looks like, I can say that it’s the same. I see it, like seeing it looks.

As I’ve been totally blind for around 6 or 7 years now, I’m likely to be using the visual cortex to help me model the world around me without vision. Looking with the vOICe when I already know where it is should help integrate the newer source of visual information into that process.

I imagine the mental balancing act of listening without straining is something that most users of the vOICe figure out instinctively, but I thought I might be able to accelerate the process by doing it deliberately. I believe it to be a mistake to try and imagine what it might look like; that strikes me as building a model with the wrong part of the brain, and I suspect doing so interferes with building the model in the parts of your brain that do this sort of thing automatically.

Sure, let’s separate mental processing into deliberate and automatic activity, based on how conscious you are of the process. I submit that deliberate mental activity can block or interfere with automatic mental activity.

I’m slowly putting together some idea on how to better allocate mental resources for a given task, and this is merely the first experiment. I have no control, and no way to avoid observer biases, and this is only the very first try. Stay tuned.

Podcast: Play in new window | Download (Duration: 3:47 — 2.6MB) | Embed

Subscribe: Email | RSS | More

Learning and doing

If I hadn’t been digging around, attempting to learn about something else entirely, I may never have learned of the vOICe. And what fun is that? The more you learn, the more you can do. The more you can do, the more you can learn. I’m glad you decided to join me.

Here’s a link to the vOICe training manual

Learn to see – The vOICe Training Manual

And here’s a link to episode 19, if you’d like to hear what the exercise I’m doing sounds like.

Ep 19: Can you hear what I see?

Podcast: Play in new window | Download (Duration: 4:00 — 2.7MB) | Embed

Subscribe: Email | RSS | More

where to zap and why

If you wanted to use brain stimulation to increase the usefulness of, or decrease the training time for, sensory substitution; where would you stimulate the brain? According to the model developed in today’s reference, it would be more effective to turn up the sound, rather than to stimulate and cause random activity within the area of the brain associated with vision.

Here’s the paper.

The Emergence of Synaesthesia in a Neuronal Network Model via Changes in Perceptual Sensitivity and Plasticity

Podcast: Play in new window | Download (Duration: 4:24 — 5.0MB) | Embed

Subscribe: Email | RSS | More

Can you hear what I see?

Continuing the subject of sensory substitution, we have the vOICe. In this episode, I explain what it is, how it works, and let you listen in as I do an exercise from the vOICe manual.

If you’d like to know more about the vOICe and/or download a free copy of the software for your very own, you can visit the “seeing with sound” website.

seeingwithsound.com