Previous month:
December 2011
Next month:
February 2012

January 2012

The Television's Natural User Interface

Below is a post I had started writing about three years ago, but never got around to finishing and publishing.  I updated this post periodically, so there is some more recent news sprinkled throughout. Given the incredible buzz around Smart TV at CES 2012, I thought this post could be enlightening.

Smart tvTV's Challenge

TVs are unnatural by design.  Hundreds of channels, half a dozen remotes and boxes and we still can't find anything worth watching.  Menus sit within menus which are buried within menus.  Content comes in from everywhere - our gaming consoles, media extenders, streaming boxes, DVRs, Cable boxes, over the air - and there is no way to sort through it all.  The TV  is screaming for the simplicity of great design, but there isn't a simple answer.  TVs aren't within reach, so direct touch interfaces are out of the question.  TVs create lots of background noise, so voice communications can be tricky, and voice processing times can create obstructive lag times that sully a smooth user experience.  

Microsoft has taken another step forward by releasing Xbox 360 / Kinect integration with many video providers.  However, talking or gesturing to or at our TVs is not yet a strong natural user interface for a number of reasons:

  1. Most people watching large TVs are not close enough to the TVs for natural vocal conversations.  Users will likely feel the need to speak loudly to their TVs, which will introduce vocal strain.  One may raise their voice to speak to a friend across a large dinner table, but in this case, the friend would also provide real-time visual and audio feedback, creating a sense of connective proximity.  This brings us to our next point.
  2. At present, our TVs do not yet talk back or otherwise show meaninful visual cues of engagment (see above), creating yet another layer of tension that depletes the natural elements of the exchange.  Apple's Siri speaks back, creating a level of intimacy through a feedback loop.  
  3. Coverflow as an interface isn't the answer to everything.  Coverflow in Apple products made a lot of sense beasue we were familiar with the visual analogy of flipping through CDs, audio tapes, pictures, video cassets and records.  However, we have never flipped through pages and pages of channels on a TV.  The physical full-arm flipping motion, particularly when performed with one's entire arm waving through the air at a distant screen, is not the "natural" way that one would surf through programming.  The interface metaphor doesn't translate especially well (as of now), and for the couch potatoes who loathed getting up to change the channel, waving an arm requires far more movement than pushing a button on a remote.
  4. TVs are dumb.  We have to adapt our behavior to meet their dynamic.  For example, most TVs don't have a robust set of metadata about programming that will allow me to say, discover great new programming based on my previous behavior, favorite actors or directors, or those of my social circle.  
  5. Voice navigation isn't smart.  Saying "Xbox, Bing, Blazing Saddles" is memorized voice navigation scheme.  Xbox initiates the Xbox program, Bing initiated the search... this is like navigating through a traditional interface with one's voice.  This isn't natural.

The Optimal TV Navigation Solution

The TV user experience faces three core tension points: (a) confusing navigation schemes (b) data and (c) reliance on analog navigation/difficult discovery.

Fixing Navigation

The natural television interface is likely going to have a remote control, but one that is flexible and intimate.  This will be a slim and simple touch screen with voice input (a mic) and a speaker - such as a phone or a tablet.  As a matter of fact, it often will be a phone or tablet.  A fantastically designed program guide will serve as a base layer that will also deliver cross-device content syncing and enable streaming (to the TV and to a remote device).  This platform will not only display information in a beautiful and simple manner, but it will provide on device and on-tv visual and auditory feedback for user inputs.  This in-hand, intimate interplay will solve the proximity challenge typically challenging set top boxes.   This solution should use a unique active noise cancellation solution - one that removes the noise generated by the TV from the noise picked up from the in-remote microphone.  

There will also likely be a new crop of more capable dedicated remotes.  For example, a clamshell design could put a simple, traditional numerical remote on the top, and a full keyboard for thumb-typing on the inside or bottom.   

Possibly most importantly, this navigation scheme must be universal.  I believe that we are very close to an era of navigation APIs where the central box interfaces with the other entertainment devices, portals and options to display all media in a singular interface.  This is about more than the PS3 and Xbox 360 dashboards, it's full integration with the cable box, DVR, Sonos system etc.  Think iTunes for TV, if iTunes didn't suck.  This interface would not only pull in all media sources, but would provide multiple tablet or phone operating systems with an app for controlling the TV.

But that isn't all.


This platform will have to rule search.  As television content becomes increasing tagged with additional metadata, the navigation solution will find the right content for the viewer from a variety of sources.  This will be Netflix meets Amazon meets Pandora meets Tivo meets the social graph meets ad targeting.  Getting all this video data into a single integrated search experience isn't going to be easy.  But that overload of available content data, coupled with individual user data, overlayed with social data, could create an incredible personalized suggestion and discovery engine.

Data will not be limited to the television, but will extend onto the remote devices through rich third party app integrations, such as GetGlue for social television viewing.  These third party integrations (apps plus devices) will offer enhanced functionality on top of the existing broadcast layer, offering a robust layer of monetization opportunities.  The TV and "remote" become the platform and hub.

What Powers This Interplay?

Most early Smart TVs will feature proprietary operating systems with limited apps.  Over time, this space will settle into one, two or at most three operating systems.  While some early TV manufacturers tried to sell companion tablets and TVs, we will being to see bundled packages.  The Samsung TV and Samsung Tablet bundle is coming to a Best Buy near you.  However, with OS and API standardization most manufacturers will develop a suite of remote-control like apps apps for popular mobile operating systems. 

TV is the next frontier in the mobile OS war.  Microsoft, Google and Apple are going to go head to head.  And believe it or not, I think the TV is going to be a far more lucrative channel than the mobile.  Here come the TV wars.

Keeping Up With The News

Most of my friends in the space regularly ask how stay on top of the news.  What am I reading?  Where do I find the time?  The real answer is that I don't read as much as they think and I find the media that meets my time demands.  

As a rule, I have found that social and mobile media are driven by a number of trends and trend makers.  Following these trends requires far less work that one would expect.  If you want to know about the next big thing, don't spend too much time focusing on advertising industry or the clueless pundits - focus on the converging trends and follow that momentum.

For example, consumer and enterprise technology has a huge impact on user and marketer behavior.  This doesn't mean that I need to read every post on Engadget.  Many people also believe that it's important to have a wide perspective on in-market activity from others in their space, and read Mashable religiously.  This also isn't necessary.  In fact, it is rarely necessary to read every article on any blog.  Find the bigger story and track the trends creating that future vision.

My Approach

SCHEDULE: The right mix of media is the mix that works for your schedule.  I commute just over an hour in the car, each way.  If we add in the random errands on the way to and from work (typically dry cleaning, groceries and diapers), I have about two and a half hours a day for audio content consumption.

PODCASTS: My audio content varies based on my mood and focus on a given week.  My favorite "social media and communications" podcast include Mitch Joel's Six Pixels of Separation and Shel Holtz and Neville Hobson's For Immediate Release.  I also listen to This Week In Tech and This Week In Google for tech industry perspective.  Finally, The Vergecast is pretty solid for consumer tech/tech industry perspective and pure geekery.  Rarely do I listen to all of these podcasts.  But they are all entertaining and informative.  And they make folding laundry far more interesting.

AUDIOBOOKS: Audiobooks are the knowledge worker's secret weapon.  Audible makes my world go round.  For less than the cost of a single business book, I get two audiobooks a month.  Audible's selection is outstanding, and if you look around you will find a coupon code for a couple of free books when you sign up for your first account.  I usually get one tech or business book and one general knowledge/for the fun of it book.  As most books are between 10 and 25 hours, I usually get through two books in about 3 weeks.  As a bonus, the higher level accounts (such as the platinum account which I subscribe to) offer a free audio version of the daily NY Times or Wall Street Journal (top articles only).  This is a nice bonus, but I found limited value in getting my news in this format.

BLOGS: There are very few must-read blogs.  While I was once an RSS addict, I have nuked my Google Reader.  I now rely on social refferals and aggregators for most of my blog content.  I read TechMeme religiously - usually checking in once or twice a day.  I read almost every article on TechMeme.  When I have the time, I also read MediaGazer.  TechMeme and MedaiGazer present the day's top stories and link to a variety of blogs and news outlets covering each story.  I tend to look to my favorite sources for their perspectives - but through TechMeme, not through an RSS subscription.  For example, I regularly enjoy All Things D (particularly Kara Swisher) for tech industry perspective and ReadWriteWeb is usually pretty good perspective for data news.  But the beauty of these sites is that you have the freedom to chose the source by the story, not the story by the source.  

The only blogger that I still read religiously is David Armano - but as he writes for multiple sites I usually pick up his stuff through his social feeds.  David is unique in the clarity he brings to his posts and perspective, I highly recommend checking him out.

MAINSTREAM NEWS:  Marketing innovation doesn't happen in a vacuum.  Reading through, or at least glancing through maintstream news outlets is very important.  I prefer web access, but print works well for many as well.

MAGAZINES:  For the past few years I have read Time, Wired, Fast Company and Inc.  Now that I have relocated abroad, I still get Wired by mail (international subscription isn't cheap) and I'm trying to keep up with my magazine reading on my Kindle Fire.  I can't say that the magazine experience is equally enjoyable.

SOCIAL:  There are some great tools out there for social discovery.  For example, Percolate will deliver a daily email of the top tweeted articles from among your followers.  LinkedIn Today offers a similar service with articles sorted by topic or industry.  While these tools are great, I have found that my social feeds - Google+, Facebook and Twitter - are great sources for referring articles.  Remember that the web is a cocktail party, not a textbook.  You don't need to be part of every conversation or read every article.  Mingle, float from place to place and only stay and engage in areas that interest you.

Finding The Time

It really isn't that hard to find the time to keep up with the industry.  I listen to podcasts and audiobooks while on the go, while running errands and occasionally while doing chores.  I read magazines over the weekend.  I browse through a handful of websites daily, scanning through much of the content and handpicking a few articles for thorough reading.  I usually spend about 20-30 minutes to reading industry content over the course of a day.  And on busy days, I miss a day.  The world doesn't end when you miss a big story.  If the news was that big, it will still be discussed tomorrow.