Sunday, February 20, 2011

Product Kaizen: Observing how users interact with products to drive future enhancements

I discovered two new things today. Not big things, mind you, but very helpful.

1. Typing '@' in a Facebook comment stream immediately brings up your friends list and starts filtering your text as you type

2. Selecting text from an error message in Adobe Flex and hitting right click brings up a menu that has 'search in google' as the first option.

Both are great examples of observing user behaviour after the product has shipped and using those observations to drive improvements to the product. Most user testing occurs during the design cycle (in agile) or after the code has been written (lets face it - in waterfall development) - but rarely post production.

In the Facebook example, people have been typing '@' to direct a comment towards a specific friend in a discussion thread. The use of '@' in this way carried over from Twitter, at least that's why I started doing it.
In the Adobe Flex example, people getting error messages would select the text from the error message, paste the message in a Google search, where they would most likely find a) other people who had the problem and b) solutions to the problem they encountered. (that's how I found the feature). It's not surprising, as most product documentation falls short on "problem diagnosis with corrective actions", so the user community fills the gap.

Both cases sound obvious at first, but you will be amazed at how these things get overlooked by traditional software companies. Web analytics, deployed to understand user behaviour in Enterprise Applications is still a novelty.
In these use cases, at some point, the product organisations observed a repeatable pattern occurring with the use of the product; they looked at the pattern and determined there was a way to optimise the product as a result.
But how did they recognise the pattern in the first place? Where did the pattern start?

We start by recognising events. An event is an occurrence within a system or domain. In our case, it is something that has happened within the application. Many events are beyond our scope of interest and do not require any reaction. The events that are of interest to us are called 'situations'. A situation is an event occurrence that we determine requires a reaction. Through observation we see that there are certain conditions in which the situation arises. It may take some time to determine what the conditions are, (some could be co-incidental) - so iterative testing is required to ensure that the conditions we will watch for are appropriate.
Finally we must determine what the appropriate behaviour, that is, the reaction, of the application to the situation; these are the actions. The actions need testing to ensure that they are appropriate and not a nuisance. Remember Microsoft Clippy?

Disclaimer: The following is just hypothetical. The description is how I would have determined the outcome and may or may not reflect what really happened.

Back to our Facebook use case. At some point, it was observed that there was a frequent use of the @ symbol (and not used in the context of a domain address). The Situation-Event-Condition-Action can be described as follows:

Situation: Significantly high use of @ symbol across all user comments.
Event: @ symbol is entered
Condition: user is entering comments in the comment box

Determining the action requires observation. We need to understand what the user does immediately after typing the '@' symbol in the comment box. In some cases there may be no action that you can predict with high probability. However in the Facebook case, at some point it would have become clear that users were typing the name of a friend after the '@' symbol. After some analysis, it would be recognised that the text typed after the @ symbol was data that was already stored in the friends object.

Action: display list of friends as the user is typing characters immediately after the @ symbol.

This would require some modification or addition of the relationship between the comments object type and the friends object type in Facebook. (I'm guessing that is how they have structured the objects in FB).
The end result is that the user has a much improved experience when typing a comment and addressing a specific friend. The enhancement enables the user to complete the task in a shorter time period. Additionally it reinforces the value of maintaining your friends list in Facebook.

This type of analysis is not limited to small user gestures - but can be applied to any interaction. Ask yourself this question, does your Enterprise Application help you or encourage you to manage or maintain your data in this way?
If you are looking for an asset in an ERP application or a contact in a CRM application - will the application provide hints, links or smarts to help you complete the task you are undertaking? Chances are you are looking for that asset because there's a fault (or an order) that needs addressing - or you are looking for a contact because you need to make a phone call or follow up on an inquiry. So did you have to carry out two steps or one to get the job done?