Experience Sampling with Smartphones

We tried something new in ID Research last week. I've long wanted to do "beeper studies" using smartphones and the simple capabilities of text messaging - texting participants and asking them to provide a brief description of what's happening in the moment, plus a snapshot. 

This is an old method that allows researchers to collect "samples" of everyday experience that would otherwise be difficult to capture. It was done using beepers back in the day, hence the name Beeper Studies.*

There are ethnographic research apps available that can be used for this, but I wanted to see if we could do it simply and without having people buy a special-purpose app. Using the class as our participants, my TA, Jeff Lin, texted them at various times of day for a week and asked them to text back with a rating of how stressed they were in that moment, along with a brief description and a photo of what they were doing.

We compiled all of the samples into a matrix and viewed the results - a picture of one week in the life of fifteen ArtCenter students. The method worked pretty well. We have a few notes on what we will change the next time we do this, but I think I'll add this to our list of generative tools.

*  For more on Beeper Studies, you really should try to find a copy of the book, "Experience Sampling Method," by Mihaly Csikszentmihalyi and others. He is one of my research heroes, and every designer should have a copy of his book, "Flow: The Psychology of Optimal Experience" on their shelf. Here's a link to his TED talk, on the topic of Flow.

Metaphor as key

I've been reading Lakoff's seminal work on metaphor, Metaphors We Live By. Written originally in 1980, the new version has an afterword written in 2003 that alone will floor you. I'd been listening to him on his recent book tour promoting his latest book, The Little Blue Book: The Essential Guide To Thinking and Talking Democratic, in which he tells progressives how to frame their ideas, and was reminded that he is one of the noted experts on the subject of framing, and therefore, the use of metaphor.

I've used the idea of Metaphor as a "wild card" in my deck of analytical tools for desgin research for some time. Instinctively, I and my students saw that if we could find the right metaphors that best expressed what we were seeing in the research, we would go a long way toward explaining the essence of what we had learned. For example, one of a team studying the phenomenon of adults playing in kickball leagues turned to me and said, "Kickball is a social filter." I understood him instantly. By choosing to join the kickball team instead of the softball or dodgeball team, one is choosing the type of people one wants to hang out with. Each crowd is different, and we had heard the kickball players saying they "wouldn't be caught dead" with the dodgeball players—a rowdier crowd with fewer brains on board (their words, not mine, so don't start in on me...!). It's easy to see how many of the activities we choose, from sports to knitting, act as social filters.

So we've long asked our students to consider the metaphors in play regarding the topic they are studying. But Lakoff brings depth and scholarship to the discussion, to the point where I'm a bit lost on how to convey to my students the breadth of his work and the many ideas they'd find useful.

Take marriage. To Lakoff, marriage can be a partnership, a journey through life together, a haven from the outside world, a means for growth, or a union of two into a third, larger entity. If one partner sees marriage as a partnership and the other, a haven, we can easilty see that the responsibilites inherent in the partnership frame will be at odds with the relief from responsibility that comes with the haven frame, and a rocky road would lie ahead.

Our research is based on the fact that one person's frame of reference will be different from another's, and it's our business to find out which frame a particular participant might be using. That's why we use the laddering technique in our interviews—to not accept any idea at face value, but to dive deeper into what a person means, as precisely as possible. At the heart of this idea is understanding the metaphors that people use as their operating system.

One of Lakoff's most interesting ideas is that aside from being lenses through which we look at the world, metaphors can offer new meaning. His example is the new metaphor of Love as a Collaborative Work of Art, compared to conventional metaphors of love—as a physical force, a patient (a sick relationship, a healthy marriage), madness, magic, war, a journey, and so on. The new metaphor has what Lakoff calls "reverberations" that can act as a guide: "The metaphor provides an organization of important love experiences that our conventional conceptual system does not make available.... It is work that requires that special balance of control and letting go that is appropriate to artistic creation.... it gives love a new meaning.... it can sanction actions, justify inferences, and help us set goals.... The meaning will be partly culturally determined and partly tied to past experiences."

All of this is potent fuel for the designer, and uncovering this fuel is the job of design research.

Designing Design Research

By way of an explanation for why I haven't posted lately, this last term was consumed by two projects: finishing the plan for what I've come to call the "tool picker" (above) to help designers new to qualitative research expand their palette of methods, plus a set of analytical tools to use on the research data.

This, on top of a term of research for a multi-term project for the American Red Cross, kept me busier than a dot painter in a paisley tie factory. I'll post more on all of this in the upcoming weeks.

The so-called "tool picker," above, is an attempt to help designers explore beyond a set research methodology. As currently taught (and sometimes practiced), design research is often treated as a constant set of tools and, as a result, students tend to think that it's a standard process. The field of design research has evolved into a complex landscape of approaches, however, and good design practice stays abreast of these developments.

In order to help my students break out of a narrow approach and yet negotiate the complexity of the myriad methods in practice today, I am attempting to acquaint them with a comprehensive and yet manageable set of methods. Also, I need to equip them with an understanding of why, and in which situations, a particular approach would be effective.

Currently, the research approach is chosen by those with expertise. There is a "guru" who brings years of experience to bear on the decision. Is there a way to enable beginners to more quickly gain the experience necessary to know which approach might be best for a given problem?

I distilled the complex set of approaches in use today into a set of eighteen (you see them down the right-hand side of the diagram, above). I will be creating a decision-making tool to guide the students through the decision process by asking a series of questions about what type of knowledge they seek for a given topic.

Starting at the left-hand side with a careful choice of topic, students are asked to generate a research objective statement. We discuss issues of ethics, scope, appropriateness, and so forth, and gain explicit knowledge of the researcher's bias.

Moving on to the decision process (while at the same time generating specifications of which sorts of participants will be recruited and engaging in the recruitment process), students begin to consider the type of knowledge they seek. We consider three general areas of knowledge about the user: what they do, what they feel, and who they are. Moving right-ward through the diagram, you can see how we move into finer levels of discrimination, arriving at a recommended set of methods.

This is a first rough design for the tool. When I first completed this version, I was disheartened at first by seeing that, if one worked backwards through the chart one could see that a skilled researcher could use any of the tools to uncover any of the types of knowledge desired. But I reminded myself that this is a decision tree that helps beginners and widens their view beyond a limited single-thread process. The tool is designed to lead them to the most appropriate choice, by no means the only choice possible. Once they've used the tool for a few projects, they will begin to gain knowledge of the wider set of approaches and begin to see how the different methods work in different cases. Once they begin to see that the tools actually can be tailored to many purposes, they are right where I want them: imbued with a robust working knowledge of the multivariate research process.