Usability testing avalanche bulletins of the Alpine arc

As a follow-up study to the redesign of the Slovenian avalanche bulletin from the past year, the Computer Systems Department of the Jožef Stefan Institute is conducting an online usability evaluation of the avalanche bulletins from the Alpine arc region. There are 20 SUS questionnaires, 1 for each bulletin, waiting for responses at the links below:

Please help us spread the word and get as many responses as possible from people who use the bulletin. It will help us evaluate future bulletin redesigns and hopefully improve the safety of our mountains.

Interactive Video Management by means of an Exercise Bike

Jan Štrekelj, Branko Kavšek
University of Primorska Faculty of Mathematics, Natural Sciences and Information Technologies Glagoljaška 8, 6000 Koper, Slovenia

Abstract: This paper describes the concept of virtual reality and the use of this technology in practice. The main part of the work is about reviewing the various stages of the development of a prototype system for interactive video management by means of an exercise bike. Systems, that are currently available on the market, are, due to their ease of use, closed units, for which upgrades are not possible or come at a great expense. The main advantage of the presented prototype, besides affordability, is a simple option to upgrade the system by adding sensors and/or modules; this allows us to extend the system in every stage of development. A low-cost computer (Raspberry Pi) is used as a processing unit, for calculating the speed of the wheel and sending this information to the control unit. The control unit processes the received data and sets the playback speed of the video clip accordingly. There is great potential for improvement on the developed prototype. Thus, ideas for further development are presented in the concluding section.



Full paper: link

I was here: a system for creating augmented reality digital graffiti in public place

Erik Šimer, Matjaž Kljun, Klen Čopil Pucihar
University of Primorska, FAMNIT, Glagoljaška 8. Koper, Slovenia

Abstract: Since ancient times travelers and tourists try to leave their marks in places they visit. However, carving or writing on historic landmarks can cause irreversible damage on such sites. One possible solution are digital graffiti. These can for example be created through projection mapping where beams of light wrap the object with the digital graffiti created by users so everyone at the site can see them. However this may disturb other visitors being there at the same time. In this paper we explore an alternative solution for creating digital graffiti by utilizing Mobile Augmented Reality (MAR) technology. We developed a mobile application which allows users to: (i) select an object or a building, (ii) map a 3D mesh onto it in order to prepare its 2D plane , and (iii) draw a graffiti on this plane. After completing the drawing the application wraps the object or the building with a modified 2D texture creating an illusion of digital graffiti. In order to (i) evaluate the social acceptance of placing digital graffiti onto historic landmarks and to (ii) evaluate if the use of our prototype is socially acceptable in public spaces, we carried out a small reflective user study. We created a couple of simple graffiti on different historic buildings and posted them on social networking site Facebook. Despite amateur appearance, posted photos received attention and generated some positive responses and questions.

Virtual graffiti

Virtual graffiti

Full paper: link

Towards affordable Mobile Crowd Sensing device

Gal Pavlin
University of Ljubljana, Faculty of Electrical Engineering Trzaska 25 1000 Ljubljana
Marko Pavlin
Jožef Stefan Institute, Jamova cesta 39, 1000 Ljubljana, Slovenia

Abstract: In this paper, we describe first prototype of mobile crowd sensing device. The device serves as a source for signals in the potential crowd sensing studies. Presented device has no intention to compete with the existing mobile devices, such as mobile phones, but to complement them where they lack of the features like affordability, simple use and new opportunities in different segments of our lives. Our main goal was to develop a device, which can cover all aspects of mobile crowd sensing and at the same time to keep the device cost at very affordable level. The described device is capable of integration into most widely available sunglasses. The complete device consisting of two separate “lenses” forms distributed ecosystem serving as source for sound, light, acceleration and temperature signals while at the same time providing actuator function with integrated LED matrix display.


Full paper: link

Towards the improvement of GUARD graphical user interface

Žiga Kopušar
Guardiaris d.o.o., Podjunska ulica 13, Ljubljana, Slovenia
Franc Novak
Jožef Stefan Institute, Jamova cesta 39, 1000 Ljubljana, Slovenia

Abstract: In this paper, we describe a case study of usability testing of the GUARD Control Desk graphical user interface, which is a part of the GUARD simulator and is used for exercise planning, execution and evaluation in soldier training. The usability testing was performed in the development phase of a new version of user interface.


Full paper: link

Wizard of Oz experiment for Prototyping Multimodal Interfaces in Virtual Reality

Blaž Gombač, Matej Zemljak, Patrik Širol, Damir Deželjin, Klen Čopič Pucihar, Matjaž Kljun
University of Primorska, FAMNIT, Glagoljaška 8, Koper, Slovenia

Abstract: In recent years the field of virtual reality has witnessed a rapid growth with significant investments in both hardware and software development. It has several potential applications for entertainment, education and enterprise where users benefit from being immersed into virtual worlds. VR headsets are available in several forms and price ranges from simple and inexpensive Google Cardboard to more complex products such as Oculus Rift. Nevertheless, designing fully operational virtual reality applications for researching new complex multimodal interaction possibilities (e.g. mid-air gesture, voice, haptics, etc.) may be difficult to implement, costly and time consuming. For this reason we have looked into ways of rapidly prototyping virtual reality interactions. Our approach consists of the Wizard of Oz experiment in which subjects interact with a computer system believing to be autonomous, but is in reality operated by researchers. The presented system allows non-technical designers to explore various multimodal interactions with rapid prototyping of VR environments.

Full paper: link

Evaluation of common input devices for web browsing: mouse vs touchpad vs touchscreen

Andrej Maleckar, Matjaž Kljun, Peter Rogelj, Klen Čopič Pucihar
University of Primorska, FAMNIT, Glagoljaška 8, Koper, Slovenia

Abstract: With the ever increasing connectivity to the Internet the use of the web has spread from static environments of desktop computers to mobile context where we interact with the web though laptop computers, tablet computers, mobile phones and wearable devices. Recent studies have shown that young people access the web using various devices and input techniques and spend on average more than 20 hours a week on the web. In this paper we plan to investigate which input technology is most usable or preferred for performing different tasks on the web. We decided to compare and evaluate the usability of the three most used input devices for web browsing, namely: a computer mouse and a touchpad on a laptop, and a touchscreen on a smartphone. For this purpose we have built a custom web page where users had to perform seven common tasks on web: open a URL address, copy/paste a URL address, copy/paste text, scroll up-down, scroll left-right, zoom in the context of a web page, and navigate a map. The results show that the mouse is still a preferred input device with shortest completion times, followed by the touchscreen interface even if it performed slower at some tasks compared to touchpad, which was marked as least preferred.


Full paper: link

Improving the usability of online usability surveys with an interactive Stripe scale

Matevž Pesek, Alja Isakovic, Gregor Strle, Matija Marolt
University of Ljubljana, Faculty of Computer and Information Science, Laboratory for Computer graphics and Multimedia, Večna Pot 113, 1000 Ljubljana, Slovenia

Abstract: The paper introduces Stripe, an interactive continuous scale for online surveys that makes it easy to compare multiple answers on a single screen. The Stripe is evaluated as an alternative to the n-point Likert scale, which is commonly used in online usability questionnaires like the System Usability Scale (SUS). The paper presents the results of a user study, which confirmed the validity of results gained with the proposed Stripe interface by applying both the Stripe and the Likert interface to an online SUS questionnaire. Additionally, the results of our study show that the participants favor the Stripe interface in terms of intuitiveness and ease of use, and even perceive the Stripe interface as less time consuming than the standard Likert scaled interface based on radio buttons.


Full paper: link

Redesign of Slovenian Avalanche Bulletin

Vanja Blažica, Janez Jaka Cerar, Aleš Poredoš
Slovenian Environment Agency, Vojkova 1b, 1000 Ljubljana, Slovenia

Abstract: We present the redesign of the Slovenian avalanche bulletin, published regularly during the winter season to warn against avalanche danger and to provide specific information for advanced users. The former version included an estimation of danger on a scale from one to five with supporting text for the whole country, while the new one offers an additional graphical description, specified for several geographical regions. The redesign profoundly influenced the work of avalanche forecasters by introducing a new interface, additional input and database storage. At the same time, users welcomed the additional information, international comparability and user friendliness of the new bulletin.


Full paper: link

Natural User Interfaces – NUI

“Computing machines can do readily, well, and rapidly many things that are difficult or impossible for men and men can do readily and well, though not rapidly, many things that are difficult or impossible for computers. This suggests that a symbiotic cooperation, if successful in integrating the positive characteristics of men and computers, would be of great value” (Licklider, 1960).

In 1960, Licklider identified the problems for realization of such ‘man-computer symbiosis’ in speed mismatch between men and computer, memory hardware requirements, memory organization requirements, differences between human and computer language and input and output equipment, the latest being the most problematic.

Fifty years later this still holds true.

NUI - importance of reusing skills

Word cloud based on this post about NUIs. The most emphasized
word, ‘skills’ highlights the importance of making NUIs feel intuitive to use, which can
be achieved by designing the interface so that it relies on skills that users already possess

Today Human-Computer Interaction (HCI) is concerned with the above-mentioned problems i.e. the design, evaluation and implementation of interactive computing systems for human use and the study of major phenomena surrounding them (Hewett et al., 2009). HCI is an interdisciplinary area deriving its knowledge from computer science (application design and engineering of human interfaces), psychology (the application of theories of cognitive processes and the empirical analysis of user behaviour), sociology and anthropology (interactions between technology, work, and organization), and industrial design (interactive products).

Interaction between human and computers takes place at the user interface. This is why research and development of user interfaces lies at the very core of HCI. We will first briefly present the development that led to today’s user interfaces and then focus on multitouch displays, their construction and use cases.

From Punch Cards to Natural User Interfaces

Early digital computers used batch interfaces. They consist of punch cards for programs and data input and prints as output. Batch interfaces are non-interactive, which means that the user has to specify everything prior to processing and is never prompted for additional input until the job is done. With the connection of teletype machines to computers and the advent of dedicated text-based CRT terminals in the late 50s, batch interfaces gave space to command line interfaces (CLI). CLIs brought speed and interactivity to human computer interaction. In the early 60s, Douglas Engelbart invented the first mouse prototype used for manipulating text-based hyper-links as a part of his ‘augmenting human intellect’ project (Engelbart, 1962). Later, researchers at Xerox PARC extended the concept of hyper-links to graphics and by doing so created the first graphical user interface (GUI). In respect to the CLI, GUI facilitates a more intuitive interaction because there is no need to learn commands by heart as available commands are presented on the screen in the form of windows, icons, menus and a pointer. This interaction paradigm can be emphasized by the acronym WIMP. Further efforts focused on the development of interfaces that interact with humans in a seamless way by understanding natural expressions of the user’s intent instead of forcing him/her to learn new rules for interaction. The results of these efforts are, among others, various speech and gesture recognition techniques known as Natural User Interfaces or NUIs.

Natural User Interfaces

NUI Group, a global research community focused on the open discovery of natural user interfaces, defines natural user interfaces as an emerging computer interaction methodology which focuses on human abilities such as touch, vision, voice, motion and higher cognitive functions such as expression, perception and recall. A natural user interface seeks to harness the power of a much wider breadth of communication modalities which leverage skills people gain through traditional physical interaction (NUI Group Community, 2009). Some representative examples of such interfaces are:

  • Multitouch displays (Han, 2005): tracking and recognizing multiple fingers (and objects) on a display, which leads to new interaction techniques. These displays differ from touchscreens capable of detecting one touch in that they allow for a different interaction paradigm, while touchscreens are just a replacement for the input device, usually the mouse, in a GUI.
  • Speech recognition (Rosenfeld et al., 2000): interfaces based on speech recognition, speech synthesis and natural language processing.
  • Space gesture recognition (Hay et al., 2008): 3D tracking or motion capture using Nintendo Wii or similar controllers as stereo vision systems. The user interacts with the system by performing, usually predefined, spatial gestures.
  • Interfaces based on electrophysiological signals (e.g. Brain-Computer Interfaces, Wolpaw (2002)): these interfaces determine the intent of the user from electrophysiological signals caused by brain, muscle, and cerebral cortex activity. For example, classifying finger gestures by interpreting forearm electromyography (EMG) signals caused by muscle movements (Saponas et al., 2009). These interfaces are also known as direct neural interfaces, mind-machine interfaces or brain-machine interfaces.

The main property of these interfaces is their intuitiveness. Ideally, a natural user interface does not require the user to undergo any training in order to interact with it. In other words, NUIs excel in terms of learnability and discoverability. NUIs build on the knowledge users get in their everyday life and exploit interaction metaphors with real-world objects to interact with digital objects in the digital world. The actual definition of NUIs is subject of ongoing debate and there are many definitions available. Some of the more concise definitions are:

  • TechTarget: “A natural user interface (NUI) is a system for human-computer interaction that the user operates through intuitive actions related to natural, everyday human behaviour … A NUI may be operated in a number of different ways, depending on the purpose and user requirements. Some NUIs rely on intermediary devices for interaction but more advanced NUIs are either invisible to the user or so unobtrusive that they quickly seem invisible.”
  • TechTerms: “A NUI is a type of user interface that is designed to feel as natural as possible to the user. The goal of a NUI is to create seamless interaction between the human and machine, making the interface itself seem to disappear.”
  • Wikipedia, Ron George: “…a user interface that is effectively invisible, or becomes invisible with successive learned interactions, to its users.”

Some authors offer a more lengthy definition of natural user interfaces or give a short definition accompanied by a lengthier explanation, like Joshua Blake’s definition of NUIs:

“A natural user interface is a user interface designed to reuse existing skills for interacting directly with content (Blake, 2013).”

He explains his definition as follows:

NUIs are designed. First, this definition tells us that natural user interfaces are designed, which means they require forethought and specific planning efforts in advance. Special care is required to make sure NUI interactions are appropriate for the user, the content, and the context. Nothing about NUIs should be thrown together or assembled haphazardly. We should acknowledge the role that designers have to play in creating NUI style interactions and make sure that the design process is given just as much priority as development;

NUIs reuse existing skills. Second, the phrase ‘reuse existing skills’ helps us focus on how to create interfaces that are natural. Your users are experts in many skills that they have gained just because they are human. They have been practicing for years skills for human-human communication, both verbal and non-verbal, and human-environmental interaction. Computing power and input technology has progressed to a point where we can take advantage of these existing non-computing skills. NUIs do this by letting users interact with computers using intuitive actions such as touching, gesturing, and talking and presenting interfaces that users can understand primarily through metaphors that draw from real-world experiences. This is in contrast to GUI, which uses artificial interface elements such as windows, menus, and icons for output and pointing device such as a mouse for input, or the CLI, which is described as having text output and text input using a keyboard. At first glance, the primary difference between these definitions is the input modality – keyboard versus mouse versus touch. There is another subtle yet important difference: CLI and GUI are defined explicitly in terms of the input device, while NUI is defined in terms of the interaction style. Any type of interface technology can be used with NUI as long as the style of interaction focuses on reusing existing skills;

NUIs have direct interaction with content. Finally, think again about GUI, which by definition uses windows, menus, and icons as the primary interface elements. In contrast, the phrase ‘interacting directly with content’ tells us that the focus of the interactions is on the content and directly interacting with it. This doesn’t mean that the interface cannot have controls such as buttons or checkboxes when necessary. It only means that the controls should be secondary to the content, and direct manipulation of the content should be the primary interaction method.”

In his keynote speech at Interact 2011, Antão Almada emphasizes Blake’s ‘interaction style over input modality’ point of view: “People feel naturally what they are supposed to do. Natural means ease of use, non-invasive sensors, to simplify as much as possible, to give intelligence to the interface so that the users get only the useful information.” The company he works for uses NUIs mainly in marketing applications, where people do not have time to learn how to interact with the applications they build and the need for easy, learnable interaction based on skills people already have is particularly strong.

Richard Monson-Haefel’s presents yet another variant of NUI definition: “A Natural User Interface is a human-computer interface that models interactions between people and the natural environment.” He continues specifying a couple of aspects of this definition that are especially important:

“NUI is a form of HCI. It is important that we make that explicit;

NUI models natural interactions. That means it leverages and uses as a template the interactions people have with each other (e.g. speech and gestures); NUI also models interactions between people and the natural environment (e.g. water and rocks) as opposed to their artificial environment (e.g. computers and cars.).”

Dennis Wixon talks about NUIs and interfaces in general in terms of principals and guidelines (Wixon, 2008). He believes that from a historical analysis of user interfaces we can extract a set of principles and guidelines for each interface type. “The principles are what drives the design. The guidelines are simply derivations of the principles for an individual context. Principles are what’s important. Principles and data drive successful design.” In this sense he analyses three different interface types that primarily build upon text—command line interfaces, graphics—graphical user interfaces and objects—natural user interfaces. He notes that interaction in CLIs builds upon the psychological function of recall as the user is disconnected from the static system he interacts with and has to direct the system by learning/recalling a high number of commands. He continues by describing GUIs as an exploratory and responsive type of interaction, where the user scans through menus and recognizes the commands he had to learn and recall within CLIs. Wixon argues that interaction with GUIs is indirect as the user actually controls the mouse or the keyboard and not the GUI itself. NUIs, on the other hand, provide an unmediated interaction that is evocative and thus relies on user intuition. The commands in a NUI are few and the interaction is fast. Wixon states that NUIs are also contextual and that they understand the environment they are in and react to it naturally. He concludes this analysis of interfaces by extracting a set of principles that should apply to NUI design: the principal of performance aesthetics (the interaction should be enjoyable), the principle of direct manipulation, the principle of scaffolding (the system should support actions as you move forward and should reveal itself in those actions), the principle of contextual environments and the principal or the super real (to take real things and extend them in a logical yet unrealistic way in the digital world). These 5 principles build upon three core principles: social, seamless, spatial. These are explained on an example – the Microsoft Surface multitouch tabletop computer. Surface is social as it encourages social interaction around a shared information space and brings people together instead of isolating them, the actions of a Surface user and the interaction with the Surface are the same, which makes the Surface seamless and spatial as the objects in the interface have an implied physicality and the interaction with the Surface leverages spatial memory.

Wixon co-authored a book about NUIs with Daniel Wigdor, called ‘Brave NUI world’ (Wigdor and Wixon, 2011). The book presents guidelines stemming from the abovementioned principles and tries to define NUIs from a user centred perspective: “The word ‘natural’ in natural user interfaces describes the user’s feelings during interaction and not the interface itself. Interfaces often cited as NUIs, e.g. multitouch displays with their highbandwidth input modality, are not natural per se – they instead provide a higher potential for developing a natural user interface, if and only if an all-new interface is designed with new input actions, new affordances – in short, a new paradigm.” In their view, creating a natural user interface is a design goal. It can be achieved through “a clear viewpoint, hard work, careful design, rigorous testing and some luck.” The clear viewpoint here is their vision of natural user interfaces: “Our vision is that a natural user interface is one that provides a clear and enjoyable path to unreflective expertise in its use. It makes skilled behaviour seem natural in both learning and expert practice. It makes learning enjoyable and eliminates the drudgery that distracts from skilled practice. It can make you a skilled practitioner who enjoys what you are doing. Natural in this sense does not mean raw, primitive, or dumbed down.” Finally, they summarize their definition of natural user interfaces as follows:

“A NUI is not a natural user interface, but rather an interface that makes your user act and feel like a natural. An easy way of remembering this is to change the way you say ‘natural user interface’ – it’s not a natural user interface, but rather a natural user interface.”

Bill Buxton supports this natural user interface over natural user interface point of view: “It’s not about speech, it’s not about gesture, it’s not even about the phone, and it’s not about human-to-human communication,” he says. “How these things work together in a natural and seamless way that reduces complexity for the users — that’s what we’re about. Getting these things right opens up another dimension in how we have technology integrated into our lives” (Buxton, 2010). His understanding of what natural in natural user interfaces means is: “designing to take advantage of the skills we acquired in a lifetime of living in the real world. These skills are motor-sensor skills, cognitive skills and social skills.” In this sense, a designer of natural user interfaces must take care not to waste people’s skills. This design paradigm also leads to interfaces that are natural to some users and not natural to others. For example, someone that has invested the time and effort to learn touch typing may find editing text in ‘vi’ natural. On the other hand, despite touch being usually regarded as a natural means for interaction, Foehrenbach et al. (2008) surprisingly report how tactile feedback added to gestural interaction with high resolution interactive walls increases error rates by 10 %. Kurfess (2013) shares this hardware-agnostic point of view on NUIs. In his opinion, natural user interfaces are not about input modalities, but about interaction style. A user interface becomes natural with careful interaction design and planning; the word natural here means that interaction is appropriate for the user, the content, and the context in which the interaction takes place. This can be achieved by reusing skills that the user already has and building upon experience and expertise often unrelated to computer use. Kurfesses key guidelines for developing natural user interfaces are the use of direct manipulation where possible, enabling instant expertise for the user, reducing cognitive load and inducing progressive learning.

The roots of this debate about the essence of natural interaction can be traced back to the late 70s, and early 80’s. Although at the time the expression natural user interfaces was not yet born, Ben Shneidermann’s work on direct manipulation belongs in this context. In (Frohlich, 1993) Frohlich et al. summarized direct manipulation as “a style of interaction characterized by the following three properties:

  1. Continuous representation of the object of interest,
  2. Physical actions or labelled button presses instead of complex syntax, and
  3. Rapid incremental reversible operations whose impact on the object of interest is immediately visible.”

The benefits of direct manipulation are “ease of learning, ease of use, retention of learning, reduction and ease of error correction, reduction of anxiety and greater system comprehension” ( Shneiderman (1982), p. 253). Due to the overlapping of these benefits and the various definitions of natural user interfaces and their goals we can say that findings related to direct manipulation can be applied to NUIs as well. In 1985, Hutchins et al. (1985) deconstruct the term directness in direct manipulation in two parts: the psychological distance between user goals and the action a specific interface requires to achieve these goals, and to psychological engagement of feeling oneself to be controlling the computer directly rather than through some intermediary. “Essentially distance refers to the mismatch between the way a user normally thinks about a problem domain and the way it is represented by a computer. Systems which reduce distance reduce this mismatch and the associated mental effort of working out what can be done in the system (semantic distance) and how to do it (articulatory distance). Engagement, on the other hand, refers to a particular style of representation based on a model world metaphor rather than on a conversational metaphor for interaction. Systems can encourage a feeling of engagement by depicting objects of interest graphically and allowing users to manipulate them physically rather than by instruction.”( Frohlich (1993), p. 3) NUIs’ strategy to reduce the above-mentioned psychological distance between user goals and action is to exploit skills users already possess. It is this ‘recycling/remixing/reusing of skills’ that makes NUIs intuitive. However, a lower distance between goals and actions is not always good – to accomplish more abstract tasks, this distance is actually beneficial, for example: repetitive tasks are easier to accomplish within a command line interface with a ‘for loop’ than manually in a more direct user interface. In this sense, Hutchins et al. conclude that direct manipulation systems benefit from the simplified mapping between goals and actions needed in an interface to achieve those goals, but at the same time lose some expressive power due to the loss in abstraction corresponding to the simplified mapping. As Bill Buxton often puts it: “Everything is best for something and worst for something else.” Therefore in 1993 Frolich (Frohlich, 1993) proposed a shift in interaction from directness to gracefulness: “The puzzle of desirable indirectness in interaction is solved if we shift to a more social definition of directness as interaction in which there is least collaborative effort expended to achieve a users’ goals. Activities which are cognitively indirect can then be seen as socially direct in that they have the effect of minimizing the joint work carried out by system and user entailed in achieving task success.” Frolich’s notion of graceful interaction builds upon Hayes’ and Reddy’s (Hayes and Reddy, 1983), who in 1983 proposed a decomposition of the term graceful interaction into a set of skills: “skills involved in parsing elliptical, fragmented, and otherwise ungrammatical input; in ensuring robust communication; in explaining abilities and limitations, actions and the motives behind them; in keeping track of the focus of attention of a dialogue; in identifying things from descriptions, even if ambiguous or unsatisfiable; and in describing things in terms appropriate for the context.” We can say that graceful interaction addresses some of the generally acknowledged problems of NUIs. Don Norman in (Norman, 2010) argues that a pure gestural system makes it difficult to discover the set of possibilities and the precise dynamics of execution that an interface requires: “It is also unlikely that complex systems could be controlled solely by body gestures because the subtleties of action are too complex to be handled by actions – it is as if our spoken language consisted solely of verbs. We need ways of specifying scope, range, temporal order, and conditional dependencies. As a result, most complex systems for gesture also provide switches, hand-held devices, gloves, spoken command languages, or even good old-fashioned keyboards to add more specificity and precision to the commands. Gestural systems are no different from any other form of interaction. They need to follow the basic rules of interaction design, which means welldefined modes of expression, a clear conceptual model of the way they interact with the system, their consequences, and means of navigating unintended consequences.” He continues: “Whether it is speech, gesture, or the tapping of the body’s electrical signals for ‘thought control,’ all have great potential for enhancing our interactions, especially where the traditional methods are inappropriate or inconvenient. But they are not a panacea.”

If direct and graceful manipulation have been terms used to discuss NUI-like interaction in the past, some authors suggest new terms for future discussion. Among them are Oviatt and Cohen’s Perceptual User Interfaces (PUI) (Oviatt and Cohen, 2000), Wixon’s Organic User Interfaces (OUI) (Wixon, 2008) and Petersen and Stricker’s Continuous Natural User Interfaces (CNUI) (Petersen and Stricker, 2009). These are often restatements or combinations of other established and NUI related research fields we discussed here. For example, perceptual user interfaces are a combination of natural user interfaces and intelligent user interfaces: “PUIs are characterized by interaction techniques that combine an understanding of natural human capabilities (particularly communication, motor, cognitive, and perceptual skills) with computer I/O devices and machine perception and reasoning. They seek to make the user interface more natural and compelling by taking advantage of the ways in which people naturally interact with each other and with the world — both verbally and non-verbally” (Oviatt and Cohen, 2000).

To sum up …

We can say that all the presented definitions generally agree that natural user interfaces should not revolve around the interface itself but should be focused on how a user perceives the interface. A concise and in this sense correct definition would be:

“Natural User Interfaces are interfaces that are intuitive to use.”

The word cloud presented in the figure above concurs with this definition. We created the word cloud with the text of this section, which surveys various definitions of NUIs. After stop words removal1 , the emphasis of each word is proportional to its normalized frequency in the text. We can see that the word, which strikes out the most, is ‘skills’; for an interface to feel natural it must be intuitive and therefore it must rely on skills that the user has already obtained in life. For example, due to their widespread use, the keyboard and mouse may also be considered as natural user interfaces. In this sense we can say, that once the skills to operate an interface are acquired, any interface can be regarded as natural. To some people, the gesture of waving on a sidewalk (to stop a taxi) is an example of a natural user interface, while to some it is not. It depends on the user, but also on the context in which the interaction takes place. Exploiting contextual information to the benefit of human-computer interaction broadens the communication channel between user and computer even further which in turn opens up new possibilities for designing interaction that exploits skills that users already possess. On one hand, natural user interfaces ease the acquisition, understanding and exploitation of context and on the other hand, context makes achieving natural interaction easier.

Excerpt  from the PhD thesis available here.

Posted in HCI