For Christmas last year, my little brother built me a single speed / fixed gear bike. He was kind enough to add both front and rear brakes, so I could get up to speed with riding it without, uh, dying. I started riding single speed -- it felt like I always had the wrong gear. Too slow, too fast. I was bored.
Then I started riding fixed-gear. Its true what they say: You feel more in touch with the road and the bike. But I still had front and rear brakes -- and I used them quite a lot, even though I didn't need to. I still hadn't broken with my non-fixie habits.
Today, I removed the rear brake. I took off the whole mechanism -- cable, calipers, everything. (I kept the front brake just to be safe.) The bike looks a LOT cleaner. But that's not interesting. What matters: It changed my entire cycling experience. I'm right handed, and the rear brake handle was on the right side of the handlebar -- so now that it was gone, the urge to brake went away. I regulated my speed according to my surroundings. I didn't brake. I way more free to just roll naturally, as I had one less knob or control to worry about. It was liberating.
When it comes to software and products of all kinds -- think about what removing a rear brake might do. There are so many needless dialogs, radio buttons, menus, alerts, gradients, drop shadows, mouseovers, text, icons, lines, boxes, and so on. Its absurd. Every single element in a UI exerts some cognitive load -- some weight on the brain. Its slowing you down. You're trying to get to a destination, and all the inessential UI is just screaming for more of your precious brain power.
Get rid of the things you don't need. Keep the things you do. Yes, you can add to the experience by subtracting.