Product Kaizen: Observing how users interact with products to drive future enhancements

I discovered two new things today. Not big things, mind you, but very helpful.

1. Typing '@' in a Facebook comment stream immediately brings up your friends list and starts filtering your text as you type

2. Selecting text from an error message in Adobe Flex and hitting right click brings up a menu that has 'search in google' as the first option.

Both are great examples of observing user behaviour after the product has shipped and using those observations to drive improvements to the product. Most user testing occurs during the design cycle (in agile) or after the code has been written (lets face it - in waterfall development) - but rarely post production.

In the Facebook example, people have been typing '@' to direct a comment towards a specific friend in a discussion thread. The use of '@' in this way carried over from Twitter, at least that's why I started doing it.
In the Adobe Flex example, people getting error messages would select the text from the error message, paste the message in a Google search, where they would most likely find a) other people who had the problem and b) solutions to the problem they encountered. (that's how I found the feature). It's not surprising, as most product documentation falls short on "problem diagnosis with corrective actions", so the user community fills the gap.

Both cases sound obvious at first, but you will be amazed at how these things get overlooked by traditional software companies. Web analytics, deployed to understand user behaviour in Enterprise Applications is still a novelty.
In these use cases, at some point, the product organisations observed a repeatable pattern occurring with the use of the product; they looked at the pattern and determined there was a way to optimise the product as a result.
But how did they recognise the pattern in the first place? Where did the pattern start?

We start by recognising events. An event is an occurrence within a system or domain. In our case, it is something that has happened within the application. Many events are beyond our scope of interest and do not require any reaction. The events that are of interest to us are called 'situations'. A situation is an event occurrence that we determine requires a reaction. Through observation we see that there are certain conditions in which the situation arises. It may take some time to determine what the conditions are, (some could be co-incidental) - so iterative testing is required to ensure that the conditions we will watch for are appropriate.
Finally we must determine what the appropriate behaviour, that is, the reaction, of the application to the situation; these are the actions. The actions need testing to ensure that they are appropriate and not a nuisance. Remember Microsoft Clippy?

Disclaimer: The following is just hypothetical. The description is how I would have determined the outcome and may or may not reflect what really happened.

Back to our Facebook use case. At some point, it was observed that there was a frequent use of the @ symbol (and not used in the context of a domain address). The Situation-Event-Condition-Action can be described as follows:

Situation: Significantly high use of @ symbol across all user comments.
Event: @ symbol is entered
Condition: user is entering comments in the comment box

Determining the action requires observation. We need to understand what the user does immediately after typing the '@' symbol in the comment box. In some cases there may be no action that you can predict with high probability. However in the Facebook case, at some point it would have become clear that users were typing the name of a friend after the '@' symbol. After some analysis, it would be recognised that the text typed after the @ symbol was data that was already stored in the friends object.

Action: display list of friends as the user is typing characters immediately after the @ symbol.

This would require some modification or addition of the relationship between the comments object type and the friends object type in Facebook. (I'm guessing that is how they have structured the objects in FB).
The end result is that the user has a much improved experience when typing a comment and addressing a specific friend. The enhancement enables the user to complete the task in a shorter time period. Additionally it reinforces the value of maintaining your friends list in Facebook.

This type of analysis is not limited to small user gestures - but can be applied to any interaction. Ask yourself this question, does your Enterprise Application help you or encourage you to manage or maintain your data in this way?
If you are looking for an asset in an ERP application or a contact in a CRM application - will the application provide hints, links or smarts to help you complete the task you are undertaking? Chances are you are looking for that asset because there's a fault (or an order) that needs addressing - or you are looking for a contact because you need to make a phone call or follow up on an inquiry. So did you have to carry out two steps or one to get the job done?

Quick start: Add Google Analytics to your Adobe AIR application in 4 easy steps

Preamble
I've been an advocate of Adobe AIR since I used the eBay "San Dimas" application (AIR was called Apollo, San Dimas became eBay Desktop). The potential to modern, build rich and smart client applications that have all the benefits of a web application (automated application updates) but the elegance and simplicity of a desktop application (native desktop integration) is a great option for developers who want to build modern client/server applications.
Recent developments by Salesforce (database), Amazon (email) and Google (XMPP) only strengthen the value of using AIR for rich client applications backed by cloud based Enterprise services. (AIR doesn't solve everything and modern browsers such as Chrome and Safari have comparable capabilities through their support for HTML5 ).

One of the downsides of using Adobe AIR, and Flash in general, has been the lack of support for Google Analytics. It was something that we severely lacked when we built the desktop gadgets at Oracle (for Siebel and CRM OnDemand). While we could track the downloads from the Oracle site, there was very little else that we could do (short of putting logic into the gadgets themselves to writeback to a web service) to track basic statistics such as how often a gadget was used, what features were used etc.
Lately I've been working with Adobe AIR for Android and have the same requirements for mobile application development.

The wait is over...
The recent addition of the AS3 library for Google analytics now enables any AIR/Flash developer to add Google analytics to their code and gain the same insight that web developers have enjoyed for years. And the other good news is, it is just as easy to add tracking to your mobile or desktop application as it is for a traditional web application.

Get up and running in 4 easy steps

1. Download the tracking library here

2. Add two import statements (tracking class, interface that the GATracker implements) and declare a tracking variable


com.google.analytics.GATracker;
com.google.analytics.AnalyticsTracker;

public var tracker:AnalyticsTracker;


3. Initialise the tracking variable with your Google Analytics tracking token (replace "UA-123-4567" with your own token).


private function onComplete():void
{
tracker = new GATracker( this, "UA-123-4567", "AS3", false );
}


4. Start tracking.


public function onButtonClick():void
{
tracker.trackPageview( "/hello/world" );
}


In this example, every time a user clicks the button, a virtual pageview of "hello world" is sent to the Google Analytics tracking servers.

That's it, you're done!! Of course in your mobile or desktop application you don't have pages etc., but the great thing is that the mechanism is so flexible that you can choose to implement based on your own needs. I used the simple URL scheme to map specific modules and actions.

This post is just enough to get you started. If you want more information, the Google Project can be found in the Google Projects Labs section here.

Brilliant work by the folks who worked on this !! Many thanks !!