Where the action is

Where the Action Is

The Foundations of Embodied Interaction

Paul Dourish

This is a very good book, but one that is not very easy to assimilate from first reading. This is a condensed version prepared mainly by 're-typing' with some rephrasing or summarizing here and there.

Where the Action Is

The Foundations of Embodied Interaction

Embodied Interaction is interaction with computer systems that occupy our world, a world of physical and social reality, and that exploit this fact in how they interact with us. it denotes a form of participative status. Embodied phenomena are ones we encounter directly rather than abstractly

It is also defined as the creation, manipulation, and sharing of meaning through engaged interaction with artifacts.

[The opposite of embodied interaction is that of cognitivism in which the human brain is modeled by information theory and it acts on an abstract representation of the world, and in which a disembodied brain could think about the world just as we do.]

In contrast, and in the embodied interaction view, such a disembodied brain could not experience the world in the same ways that we do, because our experience of the world is intimately tied to the ways in which we act in it. Physically our experiences cannot be separated from the reality of our bodily presence in the world; and socially, too, the same relationship holds because our nature as social beings is based on the ways in which we act and interact, in real time, all the time.

From Tangible and Social Computing to Embodied Interaction

The book argues that social and tangible interaction are based on the same underlying principles, despite their differences in the approaches they adopt and the ways in which they apply to design of interactive systems. The common elements are:

- They both exploit our familiarity and facility with the everyday world—whether it is a world of social interaction of physical artifacts.

- Both draw on the fact that the ways in which we experience the world are through directly interacting with it, and that we act in the world by exploring the opportunities for action that it provides to us—whether through its physical configuration or through socially constructed meaning.

In other words, they share an understanding that you cannot separate the individual from the world in which the individual lives and acts.

This is in contrast to the cognitivism approach which makes a strong separation between on the one hand the mind as a the seat of consciousness and rational decision making, with an abstract model of the world that can be operated upon to form plans of action; and, on the other hand, the objective, external world as a largely stable collection of objects and events to be observed and manipulated according to the internal mental state of the individual. From this perspective a disembodied brain could think about the world just as we do.

Embodiment is relevant to interactions with computers in three relevant ways:

1- designers of interactive systems have increasingly come to understand that interaction is intimately connected with the settings in which it occurs.

2- The focus on settings reflects a more general turn to consider work activities and artifacts in concrete terms rather than abstract ones. (instead of abstract accounts of mythical users, HCI increasingly employs field studies).

3- The artifacts of daily interaction can play many different roles through their direct embodiment in the world we occupy.

The book has four hypothesis:

1- Tangible and social computing have a common basis

2- Embodiment is the core element they have in common

3- Embodiment is not a new idea, but has been a primary topic for phenomenology

4- Phenomenology and related investigations of embodiment can provide material for developing a foundation for embodied interaction.

Definitions:

Phenomenology: a branch of philosophy principally concerned with human experience. It is primarily concerned with how we

- Perceive

- Experience, and

- Act

In the world around us.

This in contrast to having build an abstract model of the world in which we understand and see, but rather we see then understand.

Ontology: The study of nature of being and categories of existence

Epistemology: the study of knowledge

Tangible computing

Common features of tangible computing

1- No single point of control or interaction

2- Parallel interaction

3- The physical properties of the interface suggests its use

There is still no theory of tangible interaction. For example

- Why does tangible interaction work?

- Which features are important, which are merely convenient, and which are simply wrong?

- How does tangible computing mediate between the environment and the activity that unfolds it?

In this book, the essence of tangible computing lies in the way in which it allows computation to be manifest for us in the everyday world; a world that is available for our interpretation, and one which is meaningful for us in the ways in which we can understand and act in it.

Social computing

Broadly speaking, social computing refers to the application of sociological understanding to the design of interactive systems.

Sociology is concerned with the structure and function of society, and interactive systems are tools that people use. We are interested in the relationship between sociological and technical issues where sociologists and technologists work together in the design process.

The different theoretical and sociological approaches discussed in this book share 3 common c/cs:

1- the are concerned with the details of the organization of social conduct rather than broad social trends.

2- they are primarily oriented toward real activities and experiences rather than abstractions or models,

3- they all adopt an anthropological perspective on collecting, interpreting, and using field materials.

Ethnography emphasizes on detailed understanding of culture, through intensive, long-term involvement. It involves participant-observation in which the ethnographer immerses himself in the culture and explore the member’s own view of his life and culture. Describe not just what the member of culture do, but what they experience in doing it.

Ethnomethodology is the study of common sense methods by which people manage and organize their everyday behavior. The object of its investigation is common sense (what everyone knows that everyone knows).

Use ethnographic methods to collect information, but use ethnomethodology to inform the analysis.

Technomethodology is a term coined by the author of the book in (Dourish and Button 1998). It is used to describe a deeper relationship between technological design and ethnomethodology. The deeper relationship satisfies two criteria:

1- it attempts to draw not simply on a set of observations of a specific working setting, but rather on ethnomethodology’s fundamental insights about the organization of action as being a moment-to-moment, naturally occurring, improvisational response to practical problems

2- it attempts to relate these understandings not simply to design of a specific interactive system aimed at a specific setting, but rather, at the basic, fundamental principal around which software systems are developed – ideas such as abstraction, function, substitution, identity, and representation.

In their study they try to find overlap in such conceptions as accountability and abstraction.

Accountability: in ethnomethodology, accountability means “observable and reportable”. the observable and reportable nature of conduct is available only to members of a community as they are the ones who make sense of what is happening so that they can observe it and report it correctly. The methods of understanding and making sense of action and the methods for engaging in it are the same methods (the activities whereby members produce action.. are identical with their procedures for making those actions ‘accountable’). We can speak of an accountability of a social action. that is a social action is accountable if it is observable and reportable.

The analytical concept of accountability emphasizes that the organization of action, as it arises in situ, provides others with the means to understand what it is and how to respond in a mutually constructed sequence of action. It turns our attention away from simple perceived result or outcome of an action, to include how that result is achieved. We pay attention not just to destination, but also to the route taken to get there. Ethnomethodological investigations, such as those into the organization of conversation, show how this is critically important in providing a basis for rational mutual action.

Abstration: The very essence of software system design is the manipulation, combination, and creation of abstractions (eg. menus, buttons, scrollbars). Abstraction hides implementation (eg. the gas paddle is an abstraction for the engine which is the implementation).

Abstraction and accountability: In everyday interaction, ethnomethodology argues that accountability is the key feature that enables users to be able to decide what to do in order to get things done. The way the activities are organized makes their nature available to others; they can be seen and inspected, observed and reported. But this feature – the way that actions are organized – is exactly what is hidden by software abstractions. Not by accident, but by design. In the “information hiding” approach, the information that is hidden is information about how the system is doing what it does, how the perceived action is organized.

In system design, then, accountability means that the interface should be designed so as to preserve as a part of its actions, an “account” of what is happening. In short, it is a good idea to built systems that tell the user what they are doing. So the account should not simply be an abstract description of the system’s behavior, but rather an explication of how the system’s current configuration is a response to the sequence of actions that has led up to this moment, and a step on the path toward completing the larger action in which it is engaged.

The important issue in the book is not the account, but accountability – that is, how the account is related to the behavior it describes. This requires a technical approach that provides three primary features:

1- find a way to ensure that the account that is offered of the system’s behavior—a representation of that behavior—is strongly connected to the behavior that it describes. (the programmer should be in control over the ways that these two agree or differ)

2- find a way to allow this representation to be tied to the action in such a way that the account emerges along with the action rather than separately from it.

3- ensure that the account that is offered is an account of the current specific behavior of the system, in its current configuration and tackling exactly this piece of work rather than a generic account that says little more than “Oh, this is how the system generally behaves”.

One way to do this is through a reflective technique where the software design closely reflects the domain that the program is dealing with (cells in a spread sheet, or paragraphs in a document). The representation of the program not only describes the program, but also gives rise to it; the program is in effect, no more than the “performance” of the representation.

Space and Place

Space is largely concerned with physical properties (or metaphorical physical properties). It concerns how people and artifacts are configured in a setting; how far apart they are, how they interfer with lines of sight, and so on. By configuring the space in different ways, different kinds of behaviors can be supported.

Place refers to the way that social understangins convey an appropriate behavioral framing for an environment. that is why we say “out of place” and not “out of space”. So people might behave differently in two exactly different space arrangements when they are in two different places (table and chair in a meeting room, or a dining room).

So the different between space and place is that between the physical and the social.

Therefore, there are three design implications for taking a view that is centered on place rather than space:

1- this turns our attention away from the structure of the space and toward the activities that take place there.

2- The place reflects the emergence of practice. Users (not designers) need to be able to appropriate the space to the purpose at hand. This means that true places emerge only when really occupied day-to-day, not in demonstrations or experiments that last a few hours, and that place can’t be designed, only designed for.

3- The idea of place is relative to a particular community of practice.

The Locales Framework

The Locale framework has five primary components called:

1- Foundations: the social world being addressed and the sites and means that make up the local. (membership, duration, structure, roles, culture, focus and tasks)

2- Civic structure: examines how the locale relates to the others.

3- Individual views: addresses the individuals with their own perspectives, concerns, roles and forms of participation

4- Interaction trajectory” refers to the emergence of a particular course of action as it may be evolved through time and involving multiple actors. This ties together the way in which actions are situated within particular histories and the notion of the collective action of social worlds. this applies to social worlds and individuals. This helps in understanding how social worlds develop, how people enter and elave, and how the activities in which they engage contribute to the various courses of action in which the social world is engaged.

5- Mutuality. this explores the way in which these elements are made manifest or present in a space as a consequence of the manifestation of entities in a shared environment, made mutually accessible to the other participants.

Presence and awareness (taken from Fitzpatrick 1998)

For interaction to happen in a locale, there are basic requirements for presense and awareness.

1- the potential interactants need some form of representation or way of making themselves (or being made) present in the locale

2- the potential interactants need some way of being aware of the other’s presence.

Both presence and awareness possibilities for the interactants are facilitated by marious mechanisms that are part of the locale domain(s)..For the purpose at hand, the interactants will make selective choices, consciously or unconsciously to do so. Hence mutuality is the interplay of presense and awareness for interaction purposes, mediated by capability and choice.

Both the Locale Framework and Technomethodology try to draw our relationships between aspects of technological design and aspects of sociological analysis.

A philosophical discussion on the mind and meaning

Most philosophers since Descartes had taken the position that the mind is the seat of reason and meaning. The mind observes the world, gives it meaning by relating it to abstract understandings of an idealized reality and, on the basis of that meaning, formulates a plan of action. Heidegger turned that around. From his perspective, the meaningfulness of everyday experience lies not in the head, but in the world. It is a consequence of our mode of being, of the way in which we exist in the world. Where traditional philosophical approaches argued that we proceed from perception to meaning to action, Heidegger stressed the way that we ordinarily act in a world that is already organized in terms of meaning. the world has meaning for us in the ways in which we encounter it and the ways that it makes itself available to us.

Ready-to-hand & ready-at-hand

Based on Heidegger, these are two ways that we encounter the world and act through it and it had its consequences on HCI

When the tool is ready-at-hand it disappears from the view (withdraws) as an independent entity although we are acting through it (like a hammer in a normal case) or a mouse when operating normally.

When a breakdown appears, the tool becomes present-at-hand, like when the mouse ball gets dirty, or the mouse goes outside its mouse pad. that is it draws attention to itself (becomes present)

Being in the Physical World

Affordances: an affordance is a property of the environment that affords action to appropriately equipped organisms. For example, the glass of my window affords looking to me, because I have eyes that operate in that part of the electromagnetic spectrum to which the glass is transparent. My office chair affords sitting to me, because its seat matches the length of my legs. My office chair does not afford sitting to a horse or a rabbit; they are not appropriately equipped.

In other words, an affordance is a three-way relationship between the environment, the organism, and the activity

Environment(chair) ßà organism(human) ßàactivity(sitting) (this points back to env.)

Embodied skills depend on a tight coupling between perception and action. Polanyi distinguishes between what he calls proximal and distal phenomina. Loosely, proximal means “close by” or “at hand” while distal means “at a distance”. he argues that, in cases of tacit skills, our focus in on the distal phenomena, while the proximal phenomena are thouse through which the distal is achieved. Like the example of using a stick to feel the way in the dark. the feeling of the ground (distal) is through the sensory impressions we are getting through the stick (proximal) (similar to activity theory and present at hand and ready to hand)

Being in the social world

Meaning

The three aspects of meaning: Ontology, intersubjectivity, and intentionality.

Ontology:

Ontology is the branch of metaphysics concerned with the existence of objects and entities. It deals with how the world can be separated into a collection of entities whose meanings can be established, separated, and identified, and how these entities can be related to each other; how, for example, my world can be populated with entities such as computers, deadlines, chairs, and political convictions that play no part in the world of grasshoppers. In particular. in attempting to talk about entities, ontology addresses the question of how we can individuate the world, or distinguish between one entity or another; how we can understand the relationships between different entities or classes of entity; and so forth.

Ontology deals with the furniture of the world that differs depending on whose world it is. in the industrialized cultures, we live in a world furnished with political scandals, internet technologies, stock crashes, carpool lanes, and satellite broadcasts; but none of these featured in the worldview of original Australians.

Ontology is an aspect of meaning in the sense that it provides the structure from which meaning can be constructed.

Ontological problems manifest themselves quite quickly in software design. Designing a software system involves making decisions about entities, about their types and relationships, and determining how these will “line up” with elements of the real world to which they refer. When designers of a hotel registration system debate whether a reservation should be recorded as a feature of a room or of a day or of a person, they are designing the “ontology” of their system.

Ontology is not shared and is not static.

Intersubjectivity

Intersubjectivity is about how the meaning (as discussed by ontology) can be shared. The problem of Intersubjectivity is how two people can come to share an understanding about the world and about each other despite the fact that they have no immediate access to each other’s mental states.

The problem of establishing intersubjective understanding in the design of interactive technologies surfaces in two ways:

1- The first, related to the questions of ontology, concerns communication between a designer and a user, and how it is conveyed through an interactive system. The designer must somehow communicate to a user a set of constraints and expectations about how the design should be used. the system is the medium through with the designer and the user communicate. The successful achievement of this communication depends on those relevant aspects of the designer’s model being made available to the users in the course of their activity.

2- The second occasion of Intersubjectivity in interactive system is communication between users, through the system. (not referring to email and such). It refers to the way that people develop and communicate shared ways of using software systems and ways of doing their work with software systems. Systems come to be appropriated by their users and are put to work within particular patterns of practice.

Intentionality

It is the term philosophers use to refer to the directedness of meaning. Intentionality proposes meaning as a relationship between some entity (perhaps a thought or utterance) and some other entity (its meaning). For instance, when I think of my editor , there is an intentional relation between my thought and a bearded man called Bob. When we say that the word tree means an example of that class of plants with woody bark, say, then we imply that there is an intentional reference, directed from the word to the concept. Thoughts, memories, and imaginings, then, are intentional acts.

It concerns the relationship between what is done and what is meant

Coupling: is the way we can build up and break down relationships between entities, putting them together or taking them apart for the purpose of incorporating them into our actions. When I lift a rock using a lever, the rock and the lever are coupled.

When a hammer is ready-at-hand (invisible) it is coupled with my arm.

Coupling allows us to revise and reconfigure our relationship toward the world in use, turning it into a set of tools to accomplish different tasks.

[We must have options in choosing which tools or abstractions we want to use (couple with) in order to perform the activity and have the options to decouple and choose other coupling]

We work with abstractions, but rely on implementations to make them real. But by focusing on the abstractions, we often ignore the practical consequences of implementation.

while intentionality concerns the relationship between what is done and what is meant, coupling is concerned with how that relationship is maintained.

Summary: Meaning and coupling

Meaning involves a set of related but distinct phenomena, including intentionality, ontology, and Intersubjectivity. Each of these plays a role in understanding embodied interaction. Intentionality concerns the directedness of our actions, and the effects that our actions are designed to cause. Ontology concerns the ways in which, through our interaction with technological systems, we come to understand the computational world in which and through which we operate. Intersubjectivity reflects the fact that this world is one we share with other individuals; the understandings we develop of technological artifacts and social actions are ones that emerge in concert with other people. Coupling shows not just how we can understand and interpret interactive systems, but how we can operate through them. Effective action involves being able to reorient ourselves towards the technology, turning it from an object of enquiry and examination, into a tool that can be used. Technological artifacts have to be incorportated as part of a pattern of action.

The primary characteristic of technologies supporting embodied interaction is that they variously make manifest how they are coupled to the world, and so afford us the opportunity to orient to them in a variety of ways. The embodied interaction perspective begins to illumnate not just how we act on technology, but how we act through it.

Theory and Design

Theory and design are fundamentally different sorts of activities, carried out by different people with different training and presented to different audiences. the goals and criteria for theoretical examinations are quite different from those for design excercises.

Theory grounds design by providing a framework within which hypotheses can be constructed and tested, options explored and compared, and results analyzed and evaluated, and verified. From this perspective, design is simply speculative without an understanding of how and why it works; theory makes design real, because it places design in context that explains it. Whichever position we hold, though, a working relationship between theoretical understanding and design practice is crucial.

At the heart of tangible computing is the relationship between activities and the space in which they are carried out. Tangible computing explores this in three related ways: through the configurability of space, through the relationship of body to task, and through physical constraints. By the configurability of space , it is meant the ways in which tangible computing allows users to arrange the environment to meet their own particular needs. Tangible design often associate particular sors of functionality with different physical objects, whose distribution in a workspace is then under the control of the user. Because the body location and configuration in space is also adjustable, the idea of reconfigurable space leads naturally to the second issue, that of the relationship of the body to the task. Carrying out different aspects of an activity, we may need to be closer, farther away, or in different orientations to the objects of work at hand. We move around the action as the task requires. So the distribution in space introduced by tangible computing also supports a negotiable relationship between body configuration and the computation being employed in a task. Configuring the space and configuring the body are carried out relative to each other.

Exploiting physical constraints is an important part of tangible-computing design. For example two objects cannot be at the same place at the same time, so a mutual exclusion constraint can be embodied directly in the mapping of data objects onto physical ones.

In social computing, the design of interactive systems should be changed in two ways. The first is to support the improvised sequential organization of action by giving users more direct control over how activity is managed, perhaps by organizing the interaction as informal assemblage of steps rather than a rote procedure driven by the system. The second response is to help support the process of improvised, situated action by making the immediate circumstances of the work more visible. The insight here that the setting in which the work emerges includes the current state of the system; the system should make information available to the user to guide their activity moment by moment.

Six design principals:

1- Computation is a medium

[Computation (not computers) is a medium through which we accomplish our goals. The focus is on computation, because the computer itself should be hidden and we should just be aware that some computation is being done to help us achieve our goal.]

The medium is not simply the representation that is conveyed, but how that representation

2- meaning arises on multiple levels

Objects carry meaning on multiple levels: as entities in their own right, as signifiers of social meaning, as elements in systems of practice, and so on. The consequence of this in supporting embodied interaction system is in that the design should be oriented toward multiplicity of meaning that may be conveyed through them. The designer, therefore, has a critical role to play in making systems open to multiple forms of use. Something can simultaneously be representation, object, and action, carrying different meanings, values, and consequences.

3- users, not designers, create and communicate meaning, and

- users, not designers, manage coupling,

4- embodied technologies participate in the world they represent

5- embodied interaction turns action into meaning.