Context-awareness: one of 2013’s top trends

Posted on by

In a recent UX magazine article context-awareness was listed as one of the top UX trends in 2013. Yes, the same context-awareness known from HCI literature since the early 90s (the term was first used by Schilit et al. in 1994 to be precise). So it took nearly 20 years for context-awareness to find its way from labs to real life – another confirmation of what Bill Buxton calls the long nose of innovation. Other examples are the mouse and multitouch displays, which took approximately 30 and 20 years to reach mainstream respectively.

The long nose of innovation (source: http://www.sketchplanations.com/)

So, given the sudden popularity of context-awareness, it feels appropriate to post here an excerpt from my PhD thesis titled The inherent context-awareness of natural user interfaces: a case study on multitouch displays. 🙂

The fields of natural user interfaces and context-awareness, or context-aware computing, share the same goal: making devices and systems easy to use. Another similarity is that the definition of the term context-awareness also comes in different flavours. It has first been mentioned by Schilit et al. in 1994: ‘Such context-aware software adapts according to the location of use, the collection of nearby people, hosts, and accessible devices, as well as to changes to such things over time. A system with these capabilities can examine the computing environment and react to changes to the environment’. This definition and the birth of context-awareness was due to the developments in other fields of computer science. Smaller form factors, networking, the advent of mobile computing and the increase of processing power all contributed to the shift from designing systems for a single anticipated context of use to thinking about the different contexts in which a computing device is likely to be used and how it could adopt to these contexts.

The Oxford English dictionary defines context as ‘the circumstances relevant to something under consideration‘. In this sense, we can say that in context-aware computing the thing under consideration is the user’s interaction with, or use of, a certain computing device and the relevant circumstances include location, time, identity, etc. Abowd et al. identified a ‘minimal set of necessary’ context: ‘where, who, when, what, and why‘. Similarly, Dey defines context as: ‘any information that can be used to characterize the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and applications themselves‘. He further identifies certain types of context that are more important than others: location, identity, activity and time. Brown et al. describe context more generally: ‘the environment, situation, state, surroundings, task and so on.‘ Along those lines Dourish understands context as ‘information of middling relevance,‘ where this information is not something so central to an activity that it defines it, neither is it formed of details which have no bearing on the activity. Finally, Chalmers in his book offers a brief and concise definition: ‘Context is the circumstances relevant to the interaction between a user and their computing environment‘. To make this definition more explicit, he additionally lists some aspects of context: location, co-location, and related locations, identity of the user and of co-located people, current activity, time, sound and light levels, motion, both macro-level speed and location traces as well as micro-level patterns of acceleration, vibration and orientation, available network bandwidth and delay, available computing power, memory and storage, availability of particular interfaces, such as screens, speakers, microphones, and screen size and colour depth. Chalmers further lists a set of categories of context or how this contextual information can be used:

  • Context display: the contextual information gathered can be presented to the user, for example the current geographical location or exact orientation of a hand-held device;
  • Contextual augmentation: annotating data with the context of its generation, for example a GPS-enabled photo camera;
  • Context-aware configuration: for example printing a document on the nearest printer;
  • Context triggered actions: dimming the lights of a GPS navigation device after dark;
  • Contextual mediation: the use of context to modify services provided or the data requested to best meet the needs and limits arising from the context of the interaction;
  • Context-aware presentation: the adaptation of a user interface or the presentation of data based on contextual data, for example responsive websites fall in this category.

The field of context-aware computing tightly relates to ubiquitous computing. In 1991, Weiser wrote: ‘We are therefore trying to conceive a new way of thinking about computers, one that takes into account the human world and allows the computers themselves to vanish in the background.’ The background he is talking about is the context that context-awareness computing tries to capture, understand and exploit. Weiser also coined the term calm technology to describe an approach to ubiquitous computing, where computing moves back and forth between the centre and periphery of the user’s attention. In achieving this vision of calm technology context-aware computing plays an important role as it strives to collect contextual information through automated means and make it easily available to an application. It is then up to the designer of the application to decide what information is relevant and how to deal with it. This frees the user from the need to explicitly interact with the application and helps the computing device running the application to disappear in the background as envisioned by Weiser.

Elementary building blocks of context-awareness are: context acquisition, context modelling and representation, and reasoning based on contextual information.

Context acquisition is the first step in each context-aware application pipeline, the step that defines not only what contextual data is and in what form this data is available but, to some extent, it also defines the architectural style of the system (Chen, 2004). There are three basic approaches to context acquisition: direct sensor access, middleware infrastructure, and context server. Direct sensor access is suitable when the device providing a context-aware service is capable of directly communicating with the sensor, when the sensor is integrated in the device and there is no need for additional data processing. Such approach is straightforward to implement but lacks the ability to handle more complex situations that require managing multiple concurrent sensors. Adopting the middleware infrastructure approach makes it easier to separate context acquisition from context managing and/or context use. Hiding low-level sensing details also eases system extensibility and code reusability. Finally, the context server approach extends the middleware infrastructure approach by allowing access to remote data sources. Delegating context data acquisition and any needed processing to an external source also reduces the resource intensive burden on, usually mobile, resources-scarce, devices running context-aware applications. Independently of the approach taken to acquire contextual data, the sensors used in the process can be categories as physical, virtual and logical (Indulska and Sutton 2003). The most frequently used are physical sensors which measure physical properties like light, speed, acceleration, audio, temperature, presence and location of touch, blood pressure etc. For virtual sensors the source of data are different applications or services, such as e-mails, electronic calendars and keyboard or mouse input dynamics. Finally, logical sensors are those that combine physical or virtual data from physical or virtual sensors with additional knowledge or data from databases or similar sources in order to extract higher level information.

Once contextual data is acquired, it must be stored or represented in a model that will later facilitate the use of this data (by direct exploitation or by additional reasoning based on contextual information). A recent survey of context modelling and reasoning techniques (Bettini et al. 2010) listed the following seven requirements for context models and context management systems: ability to cope with heterogeneity and mobility of different contextual information sources, possibility to represent relationships and dependencies between sources and data, timelessness delivered by a system for handling context histories, robustness to imperfection in contextual data, computationally efficient reasoning to determine whether there has been a change in context and if the change requires the system to react or adapt, usability of modelling formalisms that eases context use and context modelling for application designers, and efficient context provisioning which is not trivial to achieve in the presence of large models and numerous data objects. Similar requirements were also identified by other authors (Strang and Linnhoff-Popien 2004, Korpipää and Mäntyjärvi 2003) and various solutions on how to meet them were proposed. Briefly, these solutions are: key-value models, markup scheme models, graphical models, object-oriented models, logic-based models and ontology-based models. Furthermore, Bettini et al. explore the possibilities to combine different models and reasoning techniques into hybrid models that can truly address all the above mentioned requirements.

In Makkonen et al. 2009, the authors present a short survey of context-awareness use cases. The first is a general case named technology enhancing HCI, where context-awareness is used to enable devices to act smartly and make life easier for the user. Examples range from smart home appliances like gesture controlled DVD players to enhanced desktop applications like a messaging system that minimizes interruptions to the user or adopts the possible input space based on the context. Ambulatory monitoring, remote assisted rehabilitation, abnormality detection and activity monitoring are examples of the second use case for context-awareness, i.e. healthcare. The next use case consists of context-aware systems aimed at diaries and memory support by tagging. Context-awareness is also beneficial for mobile guides: on-site tourist guides, recommendation systems that exploit information about the user, current location and perhaps information about what other users liked. These same information can also be exploited by context-aware systems for advertising. Finally, the two most useful use cases are work assistance and learning. Here context-aware systems can remind the worker of the correct order of working phases, perform automatic quality control, enable learning at any time and any place in situations that are the most appropriate for the skill being learned etc.

The possibilities where context-awareness can be applied are endless and can be very different from each other. This diversity poses a lot of challenges and concerns that need to be addressed before context-awareness can reach maturity and wide adoption. Schmidt identified the central research challenges in context-awareness as:

  • Understanding the concept of context: how is context connected to situations in the real world, how can context be represented and stored in a universal way;
  • How to make use of context: once context information is available, what is it useful for, what type of applications can be enhanced, what about ambiguity and reliability, the joint interpretation of `standard’ and contextual input;
  • How to acquire context information: the process of capturing a real world situation, assessing its relevant features and storing it in an abstract representation, a prerequisite for any context-aware system;
  • Connecting context acquisition to context use: in situations where context acquisition and context use is distributed, mechanisms for communication, common understanding and representation of contextual information are required;
  • Understanding the influence on human computer interaction: the user’s understanding and control of the system;
  • Support for building context-aware ubiquitous computing systems: providing support for context acquisition, context provision, and context use in order to make the process of implementing context-aware applications much simpler;
  • Evaluation of context-aware system: evaluation must be done in context, which may in turn influence the evaluation itself.

But enough about context-awareness, next time maybe something about natural user interfaces.