Symbolic Olfactory Display

Joseph Nathaniel Kaye

S.B. Brain & Cognitive Science,

Massachusetts Institute of Technology

1999

Submitted to the

Program in Media Arts and Sciences

School of Architecture and Planning

in partial fulfillment of the requirements

for the degree of Master of Science at the

Massachusetts Institute of Technology

May 2001

Author

Certified by

Accepted by

Symbolic Olfactory Display

Joseph Nathaniel Kaye

Submitted to the

Program in Media Arts and Sciences

School of Architecture and Planning

in partial fulfillment of the requirements

for the degree of Master of Science

Abstract

This thesis explores the problems and possibilities of computer-controlled scent output. I begin with a thorough literature review of how we smell and how scents are categorized. I look at applications of aroma through the ages, with particular emphasis on the role of scent in information display in a variety of media. I then present and discuss several projects I have built to explore the use of computer-controlled olfactory display, and some pilot studies of issues related to such display.

I quantify human physical limitations on olfactory input, and conclude that olfactory display must rely on differences between smell, and not differences in intensity of the same smell. I propose a theoretical framework for scent in human-computer interactions, and develop concepts of olfactory icons and ‘smicons’. I further conclude that scent is better suited for display of slowly changing, continuous information than discrete events. I conclude with my predictions for the prospects for symbolic, computer-controlled, olfactory display.

Thesis Supervisor: Michael J. Hawley

Assistant Professor of Media Arts & Sciences

Acknowledgements

I would first like to acknowledge my research advisor for the last five years, Michael J. Hawley, and my thesis readers, Hiroshi Ishii, Marc Canter, and Peter Brown. In particular, Hiroshi has spent time and effort beyond the call of duty. Thank you.

My first thanks must go to Amy Holden, Erin Panttaja, Wendy Ju and Chris Newell, who all gave extensive help and encouragement. Thank you all, for everything.

I would not have been able to accomplish what I have done at the Lab without the undergraduates who worked with me. The projects in this thesis were made possible by Junius Ho and Daniel Bedard, and, initially, Aleksandra Szelag. I have also learned from and been helped by working with Niko Matsakis, Niko Michaelakis, Ming Zhang, and Paul Thordarsen.

I’ve been part of the community of Personal Information Architecture for five years. I need to thank the current and past members of PIA who paved the way, and particularly the Counter Intelligence team, especially Tilke Judd and Becky Hurzwitz, who just make me happy to see them.

Many current and former members of the Media Lab helped me in many ways. In particular and in no particular order: Justine Cassell, Glorianna Davenport, Ted Selker, Rob Jacobs, Brygg Ullmer, Linda Peterson, Nitin Sawhney, Scott Eaton, Adam Smith, Karrie Karahalios, Kelly Dobson, Sile O’Modhrain, John Callihan, Ali Rahimi, Selene Mota, Ashish Kapoor, Joanne Broekhuizen, Lisa Lieberson, Roz Picard, Kristin Hall, Liz Yonda, Linda Lowe, Craig Wizneski, Dana Kirsch, Deb Cohen, David Riquier, Julie Fresina, Matt Tragert, David Small, Matt Gorbet, Sawyer Fuller, and Josh Goldberg.

My background research would not have been possible without the support of the MIT Libraries staff, and particularly their Interlibrary Borrowing Service, who went to great lengths to track down sources for me.

The various sponsors of the Media Lab paid for this work; several individuals did more than was necessary to help out. Peter Brown, my thesis reader, immediately comes to mind; his collegues at Kraft, including my friend Dave Behringer, who introduced me to Peter in the first place, Joseph Cipriano, William Croasmun, Young Kim, Erin Alexander and their colleagues. Sergio Vitali encouraged me to focus and always pushed me further; thank you.

Many people have put up with unsolicited phonecalls, emails and meetings and given information and advice These include: Jenny Tillotson, Genevieve Bell, Heather Martin, William Gaver, Robert Strong, Jane Livingston of the English National Opera, Berry Engen, Beatrice Witzgall, Fred Clayton, and Sarah at Atlantic Aromatherapy/Bliss, Co. Galway.

Further thanks for information and support goes to Bradley ‘Ladybug’ Rhodes, Robert ‘Leper’ Calhoun, Lenny Foner and the denizens of eit. Also Seraglio, Tep, and their residents and their community have been support, friends and family all the away along. May they survive for twenty-two generations.

I couldn’t have got this far without a strong base to stand on: thank you, Southbank, particularly Marcia Wallin and Barnaby Horwood, and Marie Tada before that. Roadkill Buffet taught me how to create, made me think about how I do it, and kept me sane. And thanks to Paul Nemirovski for the animated discussion that led to the original idea.

Finally, any and all of this would not have been possible without the truly magnificent and constant support of the most generous, talented and loving people I know: my parents, Brian & Jude, and my sister Sarah. Thank you. I love you.

Table of Contents

0. Prefatory Material

Title

Committee

Abstract

Acknowledgements

Table of Contents

1. Introduction 13

2. Background 14

2.1 Human Physiology 14

2.2 Electronic Noses 16

2.2.1 How electronic noses work

2.3 Chemistry of Smell 18

2.3.1 Introduction

2.3.2 Wright’s Theory

2.3.3 Amoore’s Theory

2.3.4 Turin’s Theory

2.3.5 Chiral molecules & other tricks

2.3.6 Smell chemistry – conclusions

2.4 Classification Schemes 23

2.4.1 Introduction

2.4.2 Why are classification schemes hard?

2.4.3 Historical classification schemes

2.4.4 Cultural classifications

2.4.5 Specific anosmia

2.4.6 Perfumery

2.4.7 Wine, whisk(e)y & beer

2.4.8 Recent classification schemes

2.5 Olfactory Psychophysics 35

2.5.1 Differences between individuals

2.5.2 Odor Quantity: How much?

2.5.3 Odor Quality: How many?

2.5.4 Thresholds

2.5.5 Adaptation

2.5.6 Mixtures

2.6 Human Interaction & Smell 45

2.6.1 Emotions

2.6.2 Learning & Memory

2.6.3 Sleep & Alertness

2.7 The Vomeronasal Organ 51

2.7.1 Discovery & Rediscovery

2.7.2 Lower mammals & the VNO

2.7.3 Menstrual synchronicity

2.7.4 Effects on human interaction

2.7.5 Ramifications for this thesis

3. Existing Applications 61

3.1 Literature 62

3.2 Scratch & Sniff 63

3.3 Films and Theatre 63

3.4 Museums 67

3.5 Computing 67

3.5.1 Sensorama

3.5.2 Virtual Reality

3.5.3 D.I.V.E.

3.6 Wearable scent output 70

3.6 Patents 71

3.7 Companies 72

3.7.1 Standalone systems

3.7.1.1 Scent Air

3.7.1.2 aerome

3.7.1.3 Aroma System

3.7.2 Computer-controlled systems

3.7.1.1 Digiscents

3.7.1.2 TriSenx

3.8 Smell & Information 76

3.8.1 Disease

3.8.2 Aromatherap

3.8.3 Presence Awareness: Scent

3.8.4 Incense Clocks

3.8.5 Incense Ceremony

4. Projects 81

4.1 inStink 81

4.1.1 Concept

4.1.2 Implementation

4.1.3 Discussion

4.2 Dollars & Scents 86

4.2.1 Concept

4.2.2 Implementation

4.2.3 Discussion

4.3 Scent Reminder 90

4.3.1 Concept

4.3.2 Implementation

4.2.3 Discussion

4.4 Honey, I’m Home 92

4.4.1 Concept

4.4.2 Implementation

4.4.3 Discussion

4.5 Electronic scratch-n-sniff 95

4.5 Experiments:

Characterizing Bitrates 96

4.5.1 Concept; Related Work

4.5.2 Implementation: Pilot #1

4.5.3 Results: Pilot #1

4.5.4 Implementation: Pilot #2

4.5.5 Results: Pilot #2

4.6 Inkjet Aromatron 101

5. Discussion & Conclusions 103

5.1 Olfactory Iconography 103

5.2 Smell & Information Theory 111

5.3 Smell & Ambient Media 113

5.4 Advantages & Disadvantages 114

5.4.1 Advantages of Olfactory Display

5.4.2 Disadvantages of Olfactory Display

5.4 The Future of Symbolic 117

Olfactory Display

6.0 References 123

6.1 Abbreviated & annotated 123

6.2 Full references 125

1. Introduction

Imagine that computers can emit scents as easily as they currently play music. Sniffing the air tells you the state of the world, not just spring flowers blooming outside, but abstract information: inhaling the knowledge that someone loves you, or the whiff of your portfolio rising.

And why not? William Buxton (1994) conjectures as to what conclusions a future anthropologist might conclude from the tools (computers) we use. They’d surmise a strange creature, with a well-developed sense of vision, but a somewhat impoverished ability to hear. Apparently capable of crude physical motions, but with no ability to receive feedback through that same channel. And, needless to say, no olfactory ability at all.

In this thesis, I start to address this latter deficiency, by starting to define and explore the field of olfactory display. We have always used our sense of smell to gather information about the world, but emitting scent under computer control opens up a wealth of opportunities.

I present the results of an extensive background survey; the field of computer-controlled aroma may be new, but the study of scents and olfaction goes back some six thousand years. The novel work begins with Chapter 4: I also present several projects I have built that use olfactory display, to display information from the abstract to the comparatively concrete. In Chapter 5 I attempt to synthesize an understanding of the role scent can play in information display, quantify some ways to think about and use smell in this manner, and to think about the future of olfactory display.

“...when I lost [my sense of smell] – it was like being struck blind. Life lost a good deal of its savour - one doesn’t realize how much ‘savour’ is smell. you smell people, you smell books, you smell the city, you smell the spring, maybe not conciously but as a rich unconcious background to everything else. My whole world was suddenly radically poorer.”

(Sacks, 1981)

2. Background

2.1 Human Physiology

The majority of smells are detected through the nose; air containing aromas flows in the nostrils, and up into a pair of cavities, separated by a septum. This route is known as the orthonasal pathway. However, particularly in eating, scents travel from the mouth and up a passage at the back of the mouth known as the retronasal pathway.

The cavities are surprisingly large spaces, nearly equal in volume to the brain itself. On the roof of each cavity is a small, 1 cm2 patch of yellowish tissue, known as the olfactory epithelium. To reach it, the air must pass over the three turbinate bones, covered in vascular tissue which expands and contracts unevenly, varying the flow of air through each nostril over time. On average, a single sniff remains approximately 50 cc (Wright 1964)



Volatile molecules temporarily bond with one of the many varieties of sensory cells on the epithelium. Signals are sent from the cells to the olfactory bulb, where they send signals down the olfactory nerve to other regions of the brain. This is the basic mechanism of our sense of smell. (Watson 2000, 7) To the best of our current knowledge, olfactory receptor neurons are the only neurons capable of replacing the cell body and regenerating axons in the nervous system: in fact, olfactory receptors are replaced by the body approximately every thirty days. (Kandel and Schwartz 1985)

In addition, there are two other components to the human olfactory system. The first is known as trigeminal stimulation. Trigeminal stimulation occurs in, for example, the cooling effect of peppermint, the spicyness of habañero peppers, or the choking sensation one experiences when breathing a mixture containing a high percentage of carbon dioxide. Trigeminal stimulation is not experienced as a smell, but not really as a taste, either. The trigeminal nerve is part of the fifth cranial nerve, the facial nerve; trigeminal receptors are interspersed with the olfactory receptors in the olfactory epithelium. (Cain and Murphy 1980, Silver and Maruniak 1981)

The final component of our olfactory system is the vomeronasal organ, also known as Jacobson’s Organ, which is located on the inside of the nose approximately a centimeter and a half up the nose from the nostril opening, and is concerned with pheromonal detection. We will discuss the functions and uses of the vomeronasal organ in depth later. (Watson 2000, etc.)

2.2 Electronic Noses

My research concentrates primarily on the role of smell as an output medium. However, it seems reasonable to provide a brief overview of the field of smell input devices, or electronic noses. Research into electronic noses has received a great deal of attention in both academia and industry: a by-now somewhat outdated literature review (Nagel, Schiffman and Guitierrez-Osuna 1998) lists thirteen commercial electronic noses available for sums between $8000 and $100,000, and

2.2.1 How Electronic Noses Work

A typical electronic nose has a sensor array in a chamber. First, a reference sample consisting of clean, dry air, or another gas, is pulled into the chamber to reset the samples. An sample of an odorant is pulled by vacuum pump into the chamber, exposing the sensor array to the odorant, producing a transient response. Within a few seconds to minutes each sensor is at a steady-state condition. Over this time, the results from the sensors are recorded. Then, typically, a washing gas – an alcohol vapor, for example – is applied to the unit for a few seconds to a minute, to remove the odorant from the surface. (Nagel, Schiffman and Guitierrez-Osuna 1998)

The tricky part involves the design of the sensor array itself. There are five main types. Conductive sensors use an array of metal oxides or polymers which react with the odorant, producing a change in the resistance: a given odorant will have a characteristic set of resistance values. Absorbent polymers are also used with quartz crystal microbalance (QCM) devices: as they absorb different molecules, the mass of the device increases, thereby reducing the resonance frequency. Such devices have

been used by the military for several years. (Langer 1996)

Surface acoustic wave (SAW) devices use a similar technique, but the frequency in question is much higher, and travels along the surface of the device, rather than through the volume. Metal-oxide-silicon field-effect-transistor (MOSFET) devices involve odorants interacting directly with the gate of a transistor; they are one of the least developed of electronic nose technologies. (Nagel, Schiffman and Guitierrez-Osuna 1998)

Two notable research efforts are underway concerning optical methods of odorant detection. Tufts University has developed dye mixtures containing chemically active fluorescent dyes in an organic polymer matrix. As odorants interact with the fluorescent dyes the emitted light changes, producing a color signature. (Schmiedeskamp 2001) Researchers at University of Illinois, Urbana-Champaign have developed a system incorporating a library of vapor-sensing dyes that respond to odorant molecules by undergoing distinctive color changes: they have developed a digital smell camera (Rakow and Suslick 2000, Dagani 2000)

With respect to other varieties of artificial noses, mention should be made of (Henry, Hudson, Yeatts, Myers and Feiner 1991), or their reprinted (Henry, Yeatts, Hudson, Myers and Feiner 1992), which explore the comedic possibilities of prosthetic nasal appendages as augmented interface devices within the field of human-computer interaction.

2.3 Chemistry of Smell

2.3.1 Introduction

“You cannot suppose that atoms of the same shape are entering our nostrils when stinking corpses are roasting as when the stage is freshly sprinkled with saffron of Cilicia and a nearby altar exhales the perfume of the Orient... You may readily infer that such substances as agreeably titillate the sense are composed of smooth round atoms. Those that seem bitter and harsh are more tightly compacted of hooked particles and accordingly tear their way into our senses and rend our bodies by their inroads.”

Titus Lucretius Carus, 47 BC

quoted in (Hainer, Emslie and Jacobson 1954)

Lucretius developed the first molecular theory of smell interaction with the above, some two thousand years ago. We still do not fully understand the chemistry of smell; none the less, several working models have been developed that give enough understanding of the system to be useful.

2.3.2 Wright’s Vibrational Theory

The first significant theory proposed in the last fifty years to explain the mechanism of smell was Wright’s vibrational theory. (Wright 1954) Essentially, Wright proposed that odorous molecules could be classified by absorption peaks in their far infrared spectra, below 700 cm-1. This explained why some molecularly dissimilar molecules had similar smells, but was hard to grasp in an intuitive or generative manner. Furthermore, he did not explain the mechanism by which the olfactory cells sensed the vibrations of odorous molecules.

2.3.3 Amoore’s Stereochemical Theory

The primary opposition to Wright came from Amoore’s stereochemical theory of smell. (Amoore, 1963, and, more accessibly, Amoore et. al 1964). Their conversation back and forth in the pages of various scientific journals over the course of some thirty years makes for entertaining reading. Amoore’s theory proposed that odors were determined primarily by a lock-and-key mechanism, with the shapes of molecules determining their odor. Amoore’s theory states that there are seven primary odors, with corresponding receptors. These seven are:

Diagram Description Example

A ethereal dry-cleaning fluid

B camphoraceous camphor

C musky angelica root oil

D floral roses

E pepperminty mint candy

F pungent vinegar

G putrid bad egg

The latter two are determined by electrophilic and nucleophilic molecules, respectively; the others are entirely structural.

In execution, the essence of Amoore’s theory remains mainly satisfactory; most scents can be explained, and perhaps more usefully, visualized with such a mechanism. The seven-item list is unsatisfactory, however; it fails to fully explain various olfactory phenomena, such as chiral molecules and isotope replacement; these are addressed below.

2.3.4 Turin’s Spectroscopic Theory

A recent development in the field of understanding the relationship between molecular arrangement and odor comes from Turin’s 1996 vibrational theory of olfactory reception. (Turin 1996). Like Wright’s earlier theory (Wright 1954), it is essentially vibrational in nature, looking not at the shape of a molecule but its vibrations, which, like resonant frequencies in a bridge, are a function of the structure.

However, unlike Wright, Turin proposes a mechanism for biological vibrational sensing, namely inelastic electron tunneling. Turin’s theory deals well with a number of problem cases from the literature, including bitter almond, a note shared by some seventy molecules, ranging from the triatomic hydrogen cyanide, (HCN) to the much larger and more complicated benzyl aldehyde. The tunneling electron theory also deals well with L