PREFACE

Writing this book has been a long and difficult undertaking. Since several good textbooks are available that present the basic facts about vision in a clear and readable fashion, the reader may wonder why I embarked on it. Indeed, I often wonder myself! It was not that I thought I could do a better job at what these other books do. Truthfully, I doubt I could. It was that I felt the need for a different kind of textbook, one that accurately reflects the way most modern research scientists think about vision. In fact, the scientific understanding of visual perception has changed profoundly over the past twenty-five years, and almost all the current textbooks are still in the "old" mold in terms of both structure and content. New results are included, of course, but the new approach to vision is not.
So what is this new approach? The change in the nature of visual research began in the 1970s due to the gradual emergence of an interdisciplinary field that I will call vision science. It arose at the intersection of several existing disciplines in which scientists were concerned with image understanding: how the structure of optical images was (or could be) processed to extract useful information about the environment. Perceptual psychologists, psychophysicists, computer scientists, neurophysiologists, and neuropsychologists who study vision started talking and listening to each other at this time because they began to recognize that they were working on the same problem from different, but compatible and complementary, approaches. Vision science is a branch of a larger interdisciplinary endeavor known as cognitive science that began at about the same time. Cognitive science is the study of all mental states and processes -- not just visual ones -- from an even greater variety of methodologically distinct fields, including not only psychology, computer science, and neuroscience, but linguistics, philosophy, anthropology, sociology, education, and others. In my own view, vision science is not just one branch of cognitive science, but the single most coherent, integrated, and successful branch of cognitive science.
Central to this new approach is the idea that vision is a kind of computation. In living organisms, it occurs in eyes and brains through complex neural information processing, but it can, at least in theory, also take place when information from video cameras is fed to properly programmed digital computers. This idea has had an important unifying effect on the study of vision, enabling psychologists, computer scientists, and physiologists to relate their findings to each other in the common language of computation. Vision researchers from disparate fields now read and cite each other's work regularly, participate in interdisciplinary conferences, and collaborate on joint research projects. Indeed, the study of vision is rapidly becoming a unified field in which the boundaries between the component disciplines have become largely transparent.
This interdisciplinary convergence has dominated the cutting edge of vision research for more than two decades, but it is curiously underrepresented or even absent in most modern textbooks about perception. One reason is that most textbooks that cover vision also include hearing, taste, touch, and smell. With the exception of hearing, the computational approach has not yet gained a firm foothold in these other sensory modalities. The attempt to provide a consistent framework for research in all modalities thus precludes using the computational approach so dominant in vision research.
Another reason the computational approach to vision has not been well represented in textbooks is that its essential core is theoretical, and introductory textbook authors tend to shy away from theory. The reasons are several, having partly to do with many authors' lack of computational background, partly with the difficulty of presenting complex quantitative theories clearly without overwhelming the reader, and partly with students' desire to learn only things that are "right". In the final analysis, all phenomena are "right", and all theories (save one) are presumably "wrong" ‹ although some are "wronger" than others. Students are understandably wary of expending much effort on learning a theory that is surely flawed in some way or other. Such considerations have led to a generation of textbooks that are as theoretically neutral as possible, usually by being as atheoretical as possible. But the importance of theories in science lies not so much in their ultimate truth or falsity as in the crucial role they play in understanding known phenomena and in predicting new ones. Given that we have few, if any, truly adequate theories in vision science yet, virtually every insight we have into known phenomena and every predicted new one have been generated by incorrect theories! They are quite simply an essential component of vision science.
In this book I have therefore taken the position that it is just as important for students of vision to understand theories as to know about phenomena. Most chapters include a healthy dose of theory, and some (e.g., Chapters 2 and 8) are almost entirely theoretical. But I have tried to do more than simply catalog bits and pieces of existing theory; I have tried to present a theoretical synthesis that is internally consistent and globally coherent. This is a tall order, to be sure, for the classical theories of visual perception seem so different as to be diametrically opposed. Structuralist theory, for example, claimed that wholes are nothing but associations of elementary parts, whereas Gestalt theory championed the primacy of wholes over parts. Helmholtz's theory of unconscious inference claimed that vision is mediated by thought-like deductions, whereas Gibson's ecological theory countered that perception is direct and unmediated. How can a theoretically coherent position be fashioned from such apparent diversity? I do not claim to have succeeded completely in this synthesis, for I do have to deny some important tenets of certain positions. But not many. Much has been made of differences that are more apparent than real, and I believe that the computational approach presented in this book can span the vast majority of them without strain. The strong form of Gibson's claim for direct perception is an exception, but weaker forms of this view are quite compatible with the computational view taken in this book, as I explain in Chapter 2.
The unified theoretical viewpoint I present is not my own theory, but rather my construction of what I think of as the current "modal theory". Experts on vision will naturally find aspects of it with which they take exception, but I believe the vast majority will find it consistent with most of their firmly held beliefs. The theoretical framework I advocate owes much to the influential proposals of the late David Marr and his colleagues at MIT, but this is true of the field in general. In many cases I have generalized Marr's specific proposals to make clear how his own detailed theories were examples of a more general framework into which a variety of other specific theories fit quite comfortably. Even so, I do not consider the view I describe as exclusively or even primarily Marr's, for it owes just as much to classical perceptual theorists such as Helmholtz, Wertheimer, Gibson, and Rock. The interweaving of such diverse theoretical ideas is not difficult to achieve, provided one avoids divisive dogma and instead concentrates on the positive contributions of each view.
Because the book is much more theoretical and interdisciplinary than most perception textbooks, it is correspondingly longer and more difficult. It is designed for an upperdivision undergraduate course or an entry-level graduate course on vision, most likely as part of a program of study in Psychology or Cognitive Science. I have tried to explain both theories and phenomena clearly enough to be understood by intelligent, motivated students with no prior background in the field of vision. I do presume that readers have some basic understanding of behavioral experiments, computer programming, and neurobiology. Those unfamiliar with this material may find certain portions of the text more difficult and have to work harder as a result, but the technical prerequisites are intended to be relatively few and low-level, mainly high school geometry and algebra.
Despite the strongly interdisciplinary nature of this book, it is written primarily from a psychological perspective. The reason is simply that I am a psychologist by training, and no matter how seriously I have read the literature in computer vision and visual neuroscience, the core of my viewpoint is still psychological. In keeping with this perspective, I have avoided presenting the complex mathematical details that would be central to a computer scientist's presentation of the same topics and the biological details that would figure prominently in a neuroscientist's presentation. By the same token, I have included details of experimental methods and results that they might well have omitted. Vision science may have made the boundaries between disciplines more transparent, but it has not eliminated them. Psychologists still perform experiments on sighted organisms, computer scientists still write programs that extract and transform optical information, and neuroscientists still study the structure and function of the visual nervous system. Such methodological differences will not disappear. Indeed, they should not disappear, because they are precisely what makes an interdisciplinary approach desirable. What is needed is a group of vision scientists who are well versed in all these disciplines. It is my sincere desire that this book will help to create such a community of scientists.
In addition to its use as a textbook, I hope this book will also be useful as a reference text for members of the expanding vision science community. Although the sections describing one's own field of specialization may seem elementary, the rest of the book can provide useful background material and relatively sophisticated introductions to other areas of vision research. The coverage is not intended to be at the same level as a professional handbook, where each chapter is expected to be a definitive treatment of a specific topic written by a world-class expert, but it is also more accessible and internally consistent than any handbook I have ever seen. It is therefore particularly useful for someone wanting to get a global view of vision science ‹ the "lay of the land," if you will ‹ within which the focussed chapters one finds in professional handbooks will fit comfortably and make more sense.
ORGANIZATION OF THE BOOK
Because the aim of this book is to integrate material across disciplines, each chapter includes findings from many different approaches. There is no "physiology chapter", no "psychophysics chapter", no "developmental chapter", no "neuropsychology chapter", and no "computational chapter" in which the separate, and often conflicting, mini-views within each of these disciplines can be conveniently described in isolation. I have avoided this approach because it compartmentalizes knowledge, blocking the kind of synthesis I am trying to achieve and that I view as essential for progress in the field. Rather, the topic of each chapter is discussed from all relevant disciplines, sometimes including those traditionally ignored by writers of textbooks on vision, such as computer science, philosophy, and linguistic anthropology. Even within the more standard visual disciplines, the coverage is not uniform, however, because the distribution of knowledge is not uniform. We know a great deal more about the physiology of early image processing, for example, than we do about the physiology of categorization and visual imagery. This unevenness is merely a reflection of the current state of understanding.
The overall organization of the book is defined by the its three Parts: Foundations, Spatial Perception, and Perceptual Dynamics.
FOUNDATIONS. The Foundations section covers a basic introduction to the interdisciplinary science of vision. Chapter 1 introduces the problem of visual perception and sets forth an interdisciplinary framework for approaching it. It covers many of the most important perceptual, optical, and physiological facts on which vision is based. Chapter 2 then discusses theoretical approaches to vision from an historical perspective. It covers the classical theories of vision as well as the information processing (or computational) approach, including several important proposals from the work of the late David Marr (1982) that play a large role in defining the superstructure of the rest of the book. The key idea is that visual perception can be analyzed into a sequence of four basic stages: one that deals with extracting image structure (Marr's "primal sketch"), one that deals with recovering surfaces in depth (Marr's "2.5-D sketch"), one that deals with describing 3-D objects (Marr's "volumetric descriptions"), and one that deals with identifying objects in terms of known categories. This sequence of processes ‹ which I call image-based, surface-based, object-based, and category-based ‹ is then traced for each of the major topics covered in the book: color, space, and motion perception. The final chapter of the Foundations section, Chapter 3, is a long, but important one. It tells "the color story", which spans vision science from the physiology of retinal receptors to the linguistic analysis of naming colors in different cultures of the world. Its importance derives from the fact that the current understanding of color processing illustrates better than any other single example in all of cognitive science why an integrated, interdisciplinary approach is necessary for a complete theory.
SPATIAL PERCEPTION. Chapters 4 through 9 cover spatial perception as a sequence of processes: extracting image structure (Chapter 4), recovering oriented surfaces in depth (Chapter 5), organizing perception into coherent objects (Chapter 6), perceiving object properties and parts (Chapter 7), representing shape (Chapter 8), and identifying objects as members of known categories (Chapter 9). This material on spatial processing of images is the heart and soul of classical visual perception. Because it is much more complex than color processing, we understand it much less well. It is hard at times not to be overwhelmed by the mountains of facts and frustrated at the lack of good theory, but I believe we are beginning to get some clearer notion of how this all fits together.
VISUAL DYNAMICS. The final section concerns perceptual dynamics: how visual perception and its aftereffects change over time. Perception of motion and events is the first topic considered (Chapter 10), being essentially an extension of spatial perception to the domain of space-time. Then we discuss ways in which the visual system selects different information over time by making overt eye movements and covert attentional adjustments (Chapter 11). Next we consider memory for visual information within a multi-store framework ‹ iconic memory, short-term visual memory, and long term visual memory ‹ and examine how such stored information can be reconstructed and transformed in visual imagery (Chapter 12). Finally, Chapter 13 takes up what is perhaps the most fascinating of all topics: the nature of visual awareness (and its absence in certain neurological syndromes) and various attempts at explaining it. This topic is very much on the cutting edge of modern vision science and is finally getting the attention it deserves.
TAILORING THE BOOK TO DIFFERENT NEEDS
This book covers the field of vision more comprehensively than any existing text. I chose the subtitle, Photons to phenomenology, in part because it conveys the soup-to-nuts approach I have taken in deciding what topics to include. Standard textbooks on vision do not include much, in any, material on visual attention, memory, imagery, or awareness, for example. I view all of these topics as essential aspects of vision and treat them accordingly. They require the kind of extensive coverage they receive in Chapters 11-13, including integration with classical topics in vision.
One consequence of this comprehensive, integrated approach to vision is that the book contains more topics and material than can comfortably fit into any single-term undergraduate course. I therefore encourage instructors to be selective when they use it in such courses. I have included too much rather than too little partly because I find it easier to skip what I do not want to cover in a single unified textbook than to find external readings that cover the desired material at an appropriate level and in a framework compatible with the main textbook -- a nearly impossible task, in my experience.
There are several ways of tailoring the present book to different needs. Most obviously, certain chapters can be skipped in their entirety. If color is not a high priority, Chapter 3 can be omitted with only minor ramifications to later chapters. Chapter 10 on motion perception is likewise reasonably independent of the rest of the book. For courses restricted to classical visual perception, Chapter 11 on eye movements and attention and Chapter 12 on memory and imagery are probably the least relevant. A course emphasizing high-level vision can reasonably omit much of Chapter 4 on image-based processing. And so on.
Another approach to selective coverage is omitting subsections within chapters. For traditional courses on the psychology of vision, the sections on computational theory and other technical material may be eliminated or assigned as optional. (One effective approach I have used is to teach an "honors section" of the course for additional credit in which the more difficult material is required and standard sections for which it is not.) For convenience in identifying difficult material, I have indicated such sections by black diamonds around the headings [implemented as open rectangles in this manuscript]. These sections are reasonably encapsulated, especially the most difficult passages (e.g., those describing algorithms for lightness constancy, edge detection, stereopsis, edge-labelling, texture segregation, and motion detection). Eliminating this material has the advantages of making the book substantially shorter and easier to understand for students with less technical backgrounds. The developmental sections can also generally be omitted without much affecting the book's continuity and cohesion.
For students with strong scientific backgrounds who are highly motivated to learn about modern vision science, however, I encourage instructors to use as much of the book as possible. It is perfectly reasonable, for example, to cover the entire book in a graduate course on vision that lasts a full semester.
ACKNOWLEDGEMENTS
There are many people I wish to thank for helping me in various phases of writing this book. First and foremost, I gratefully acknowledge my debt to my late colleague and friend, Irvin Rock, to whom this book is dedicated. Irv not only taught me about perception in his own gentle, probing, inimitable way, but he read and commented on earlier drafts of the first nine chapters before his death in 1995. Moreover, his 1975 textbook, An introduction to perception, served as a model for this one in certain important ways. In that book Irv tried to present the phenomena of visual perception at an introductory level, yet within a coherent and principled theoretical view of perception as a problem solving process. While it was still in print, it was my favorite perception text, and I know that some instructors continue to use it in photocopied readers to this day.
Irv's influence on this book has been substantial, as careful readers will surely discover. Had he lived, I believe his continued contributions would have improved it further and kept me from making some mistakes I doubtless have made in his absence. After Irv's death Arien Mack, one of Irv's most distinguished students and collaborators, became my primary reviewer for the remaining chapters of the book. One or the other of them has read and commented on every one.
Many other experts in vision science have also read more limited portions of the book, either at my own request or that of MIT Press, and provided valuable comments on material in their specialty areas. I wish to thank the following scholars, plus several anonymous reviewers, for the time and effort they spent in evaluating portions of the manuscript:
Chapter 1: Irvin Rock, Jack Gallant, Paul Kube
Chapter 2: Irvin Rock, James Cutting, Ulric Neisser, Paul Kube, Jitendra Malik, and an anonymous reviewer
Chapter 3: Irvin Rock, Karen DeValois, Alan Gilchrist, C. Lawrence Hardin, Paul Kay, and an anonymous reviewer
Chapter 4: Irvin Rock, Jitendra Malik, Jack Gallant, Ken Nakayama, and an anonymous reviewer
Chapter 5: Irvin Rock, Jitendra Malik, Ken Nakayama, and an anonymous reviewer
Chapter 6: Irvin Rock, Jitendra Malik, and Michael Kubovy
Chapter 7: Irvin Rock, Arien Mack, and an anonymous reviewer
Chapter 8: Irvin Rock, John Hummel, and an anonymous reviewer
Chapter 9: Irvin Rock, John Hummel, and an anonymous reviewer
Chapter 10: Arien Mack, James Cutting, Dennis Proffitt, and an anonymous reviewer
Chapter 11: Arien Mack, Michael Posner, Anne Treisman, and William Prinzmetal
Chapter 12: Arien Mack and Martha Farah
Chapter 13: Arien Mack, Alison Gopnik, John Watson, Bruce Mangan, Bernard Baars, and C. Lawrence Hardin
Appendix 1: Ken Nakayama and Ervin Hafter
Appendix 2: John Kruschke and Jerome Feldman
Appendix 3: Alan Gilchrist
Several students, postdoctoral fellows, and visitors in my lab have also taken the time to comment on various portions of the book. Without differentiating among chapters, I wish to thank Dan Levitin, Elisabeth Pachiere, Joel Norman, Akira Shimaya, Diane Beck, Justin Beck, Sheryl Ehrlich, Craig Fox, Jonathan Neff, Chuck Schreiber, and Chris Stecker for their helpful comments. In addition, I would like to thank Chris Linnett, Sheryl Ehrlich, Diane Beck, Thomas Leung, and Greg Larson for doing some of the more complex and technical illustrations, Lisa Hamilton for helping to track down some of the references, and Eddie Hubbard for his heroic efforts in tying up all the loose ends (of which there were many).
This book took a long time to write ‹ certainly a good deal longer than I had planned or than I would like to admit ‹ and its writing put a significant strain on all other aspects of my life. During this time, many people have contributed emotional support and understanding, for which they are due both thanks for their help and apologies for the time this project has stolen from them. They include Paul Harris, Stephen Forsling, David Shiver, and Andy Utiger, as well as Linda, Emily, and Nathan Palmer.
TABLE OF CONTENTS

PART 1: FOUNDATIONS
Chapter 1: An Introduction to Vision Science
Chapter 2: Theoretical Approaches to Vision
Chapter 3: Color Vision: A Microcosm of Vision Science

PART 2: SPATIAL VISION
Chapter 4: Processing Image Structure
Chapter 5: Perceiving Surfaces Oriented in Depth
Chapter 6: Organizing Objects and Scenes
Chapter 7: Perceiving Object Properties and Parts
Chapter 8: Representing Shape and Structure
Chapter 9: Perceiving Function and Category

PART 3: VISUAL DYNAMICS
Chapter 10: Perceiving Motion and Events
Chapter 11: Visual Selection: Eye Movements and Attention
Chapter 12: Visual Memory and Imagery
Chapter 13: Visual Awareness
CHAPTER OUTLINES

Chapter 1 AN INTRODUCTION TO VISION SCIENCE
1.1 VISUAL PERCEPTION
1.1.1 DEFINING VISUAL PERCEPTION
1.1.2 THE EVOLUTIONARY UTILITY OF VISION
1.1.3 PERCEPTION AS A CONSTRUCTIVE ACT
Adaptation and Aftereffects
Reality and Illusion
Ambiguous Figures
1.1.4 PERCEPTION AS MODELLING THE ENVIRONMENT
Visual Completion
Impossible Objects
Predicting the Future
1.1.5 PERCEPTION AS APPREHENSION OF MEANING
Classification
Attention and Consciousness
1.2 OPTICAL INFORMATION
1.2.1 THE BEHAVIOR OF LIGHT.
Illumination
Interaction with Surfaces
The Ambient Optic Array
1.2.2 THE FORMATION OF IMAGES
Optical Images
Projective Geometry
Perspective and Orthographic Projection
1.2.3 VISION AS AN "INVERSE" PROBLEM
1.3 VISUAL SYSTEMS
1.3.1 THE HUMAN EYE
Eye and Brain
Anatomy of the Eye
Physiological Optics
1.3.2 THE RETINA
Neurons
Photoreceptor
Peculiarities of Retinal Design
Pathways to the Brain
1.3.3 VISUAL CORTEX
Localization of Function
Occipital Cortex
Parietal and Temporal Cortex
Mapping Visual Cortex
The Physiological Pathways Hypothesis

Chapter 2 THEORETICAL APPROACHES TO VISION
Playing 20-Questions With Nature
2.1 CLASSICAL THEORIES OF VISION
2.1.1 STRUCTURALISM
2.1.2 GESTALTISM
Holism
Psychophysiological Isomorphism
2.1.3 ECOLOGICAL OPTICS
Direct Perception
2.1.4 CONSTRUCTIVISM
Unconscious Inference
Heuristic Interpretation
2.2 A BRIEF HISTORY OF INFORMATION PROCESSING
2.2.1 COMPUTER VISION
The Invention of Computers.
Blocks World
Computational Approaches to Ecological Optics
Connectionism and Neural Networks.
2.2.2 INFORMATION PROCESSING PSYCHOLOGY
2.2.3 BIOLOGICAL INFORMATION PROCESSING
Early Developments
Single Cell Recording
Autoradiography
Brain Imaging Techniques
2.3 INFORMATION PROCESSING THEORY
The Information Processing Paradigm
2.3.1 THE COMPUTER METAPHOR
2.3.2 THREE LEVELS OF INFORMATION PROCESSING
The Computational Level
The Algorithmic Level
The Implementational Level
2.3.3 THREE ASSUMPTIONS OF INFORMATION PROCESSING
Informational Description
Recursive Decomposition
Physical Embodiment
2.3.4 REPRESENTATION
Homomorphisms
2.3.5 PROCESSES
Implicit versus Explicit Information
Processing as Inference
Hidden Assumptions
Heuristic Processes
Hidden Assumptions versus Ecological Validity
Top-down Versus Bottom-up Processes
2.4 FOUR STAGES OF VISUAL PERCEPTION
2.4.1 The Retinal Image
2.4.2 The Image-based Stage
2.4.3 The Surface-based Stage
2.4.4 The Object-based Stage
2.4.5 The Category-based Stage

Chapter 3 COLOR VISION: A MICROCOSM OF VISION SCIENCE
3.1 THE COMPUTATIONAL DESCRIPTION OF COLOR PERCEPTION
3.1.1 THE PHYSICAL DESCRIPTION OF LIGHT
3.1.2 THE PSYCHOLOGICAL DESCRIPTION OF COLOR
Color Space
Hue
Saturation
Lightness
Lightness versus Brightness
3.1.3 THE PSYCHOPHYSICAL CORRESPONDENCE
3.2 IMAGE-BASED COLOR PROCESSING
3.2.1 BASIC PHENOMENA
Light Mixture
Color Blindness
Color Afterimages
Simultaneous Color Contrast
Chromatic Adaptation
3.2.2 THEORIES OF COLOR VISION
Trichromatic Theory
Opponent Process Theory
Dual Process Theory
3.2.3 PHYSIOLOGICAL MECHANISMS
Three Cone Systems
Color Opponent Cells
Reparameterization in Color Processing
Lateral Inhibition
Adaptation and Aftereffects
Double Opponent Cells
Higher Cortical Mechanisms
3.2.4 DEVELOPMENT OF COLOR VISION
3.3 SURFACE-BASED COLOR PROCESSING
Chromatic Surface Reflections
3.3.1 LIGHTNESS CONSTANCY
Adaptation Theories
Unconscious Inference versus Relational Theories
The Importance of Edges
Retinex Theory
The Scaling Problem
Illumination versus Reflectance Edges
Distinguishing Illumination from Reflectance Edges
3.3.2 CHROMATIC COLOR CONSTANCY
Constraining the Problem
Illumination versus Reflectance Edges Revisited
Development of Color Constancy
3.4 THE CATEGORY-BASED STAGE
3.4.1 COLOR NAMING
Basic Color Terms
3.4.2 FOCAL COLORS AND PROTOTYPES
3.4.3 A FUZZY-LOGICAL MODEL OF COLOR NAMING
Fuzzy Set Theory
Primary, Derived, and Composite Color Categories

Chapter 4 PROCESSING IMAGE STRUCTURE
4.1 PHYSIOLOGICAL MECHANISMS
4.1.1 RETINAL AND GENICULATE CELLS
Ganglion Cells
Bipolar Cells
Lateral Geniculate Nucleus
4.1.2 STRIATE CORTEX
Hubel and Wiesel's Discovery
Simple Cells
Complex Cells
Hypercomplex Cells
4.1.3 STRIATE ARCHITECTURE
The Retinotopic Map
Ocular Dominance Slabs
Columnar Structure
4.1.4 DEVELOPMENT OF RECEPTIVE FIELDS
4.2 PSYCHOPHYSICAL CHANNELS
Psychophysics
4.2.1 SPATIAL FREQUENCY THEORY
Fourier Analysis
Spatial Frequency Channels
Contrast Sensitivity Functions
Selective Adaptation of Channels
Spatial Frequency Aftereffects
Thresholds for Sinewave versus Squarewave Gratings
Development of Spatial Frequency Channels
4.2.2 THE PHYSIOLOGY OF SPATIAL FREQUENCY CHANNELS
4.3 COMPUTATIONAL APPROACHES
4.3.1 MARR'S PRIMAL SKETCHES
4.3.2 EDGE DETECTION
Edge Operators and Convolution
The Marr-Hildreth Zero-Crossing Algorithm
Neural Implementation
Scale Integration
The Raw Primal Sketch
4.3.3 ALTERNATIVE COMPUTATIONAL THEORIES
Texture Analysis
Structure from Shading
4.3.4 A THEORETICAL SYNTHESIS
Local Spatial Frequency Filters
Exploiting the Structure of Natural Images
4.4 VISUAL PATHWAYS
4.4.1 PHYSIOLOGICAL EVIDENCE
4.4.2 PERCEPTUAL EVIDENCE

Chapter 5 PERCEIVING SURFACES ORIENTED IN DEPTH
5.1 THE PROBLEM OF DEPTH PERCEPTION
5.1.1 HEURISTIC ASSUMPTIONS
5.1.2 MARR'S 2.5-D SKETCH
5.2 OCULAR INFORMATION
5.2.1 ACCOMMODATION
5.2.2 CONVERGENCE
5.3 STEREOSCOPIC INFORMATION
5.3.1 BINOCULAR DISPARITY
Corresponding Retinal Positions
The Horopter
5.3.2 THE CORRESPONDENCE PROBLEM
Random-Dot Stereograms
Autostereograms
Binocular Rivalry
5.3.3 COMPUTATIONAL THEORIES
The First Marr-Poggio Algorithm
Edge-based Algorithms
Filtering Algorithms
5.3.4 PHYSIOLOGICAL MECHANISMS
5.3.5 VERTICAL DISPARITY
5.3.6 DA VINCI STEREOPSIS
5.4 DYNAMIC INFORMATION
5.4.1 MOTION PARALLAX
5.4.2 OPTIC FLOW CAUSED BY A MOVING OBSERVER
Computational Theories
5.4.3 OPTIC FLOW CAUSED BY MOVING OBJECTS
5.4.3 ACCRETION AND DELETION OF TEXTURE
5.5 PICTORIAL INFORMATION
5.5.1 PERSPECTIVE PROJECTION
Alberti's Window
5.5.2 CONVERGENCE OF PARALLEL LINES
5.5.3 POSITION RELATIVE TO THE HORIZON OF A SURFACE
5.5.4 RELATIVE SIZE
5.5.5 FAMILIAR SIZE
5.5.6 TEXTURE GRADIENTS
5.5.7 EDGE INTERPRETATION
Vertex Classification.
Four Types Of Edges.
Edge Labels.
Physical Constraints.
Extensions And Generalizations.
5.5.8 SHADING INFORMATION
Perceiving Surface Orientation from Shading
Horn's Computational Analysis
Cast Shadows
5.5.9 AERIAL PERSPECTIVE
5.5.10 INTEGRATING INFORMATION SOURCES
Dominance
Compromise
Interaction
5.6 DEVELOPMENT OF DEPTH PERCEPTION
5.6.1 OCULAR INFORMATION
5.6.2 STEREOSCOPIC INFORMATION
5.6.3 DYNAMIC INFORMATION
5.6.4 PICTORIAL INFORMATION

Chapter 6 ORGANIZING OBJECTS AND GROUPS
The Problem of Perceptual Organization
The Experience Error
6.1 PERCEPTUAL GROUPING
6.1.1 THE CLASSICAL PRINCIPLES OF GROUPING
6.1.2 NEW PRINCIPLES OF GROUPING
Measuring Grouping Effects Objectively
6.1.3 IS GROUPING AN EARLY OR LATE PROCESS?
6.1.4 PAST EXPERIENCE
6.2 REGION ANALYSIS
6.2.1 UNIFORM CONNECTEDNESS
6.2.2 REGION SEGMENTATION
Boundary-based Approaches
Region-based Approaches
Evidence from Stabilized Images
Parts and Parsing
6.2.3 TEXTURE SEGREGATION
Discovering the Features of Texture
Texture Segregation as a Parallel Process
A Theory of Texture Segregation
6.3 FIGURE-GROUND ORGANIZATION
6.3.1 PRINCIPLES OF FIGURE/GROUND ORGANIZATION
6.3.2 ECOLOGICAL CONSIDERATIONS
6.3.3 EFFECTS OF MEANINGFULNESS
6.3.4 THE PROBLEM OF HOLES
Attention and Figure/Ground Organization
6.4 VISUAL INTERPOLATION
6.4.1 VISUAL COMPLETION
Theories of Amodal Completion
6.4.2 ILLUSORY CONTOURS
Relation to Visual Completion
Physiological Basis of Illusory Contours
6.4.3 PERCEIVED TRANSPARENCY
6.4.4 FIGURAL SCISSION
6.4.5 THE PRINCIPLE OF NON-ACCIDENTALNESS
6.5 MULTISTABILITY
6.5.1 CONNECTIONIST NETWORK MODELS
6.5.2 NEURAL FATIGUE
6.5.3 EYE FIXATIONS
6.5.4 THE ROLE OF INSTRUCTIONS
6.6 DEVELOPMENT OF PERCEPTUAL ORGANIZATION
6.6.1 THE HABITUATION PARADIGM
6.6.2 THE DEVELOPMENT OF GROUPING

Chapter 7 PERCEIVING OBJECT PROPERTIES AND PARTS
Constancy and Illusion
Modes of Perception Proximal and Distal
7.1 SIZE
7.1.1 SIZE CONSTANCY
The Size-Distance Relation
Demonstrations of Size Constancy
Departures from Constancy
Taking Account of Distance
Texture Occlusion
Relative Size
The Horizon Ratio
Development of Size Constancy
7.1.2 SIZE ILLUSIONS
The Moon Illusion
The Ponzo Illusion
Illusions of Relative Size
Occlusion Illusions
7.2 SHAPE
7.2.1 SHAPE CONSTANCY
Perspective Changes
Two-dimensional Figures
Three-dimensional Objects
Development of Shape Constancy
7.2.2 SHAPE ILLUSIONS
7.3 ORIENTATION
7.3.1 ORIENTATION CONSTANCY
Proprioceptive Information
7.3.2 ORIENTATION ILLUSIONS
Frames of Reference
Geometric Illusions
7.4 POSITION
7.4.1 PERCEPTION OF DIRECTION
7.4.2 POSITION CONSTANCY
Indirect Theories of Position Constancy
Direct Theories of Position Constancy
7.4.2 POSITION ILLUSIONS
7.5 PERCEPTUAL ADAPTATION
7.6 PARTS
7.6.1 EVIDENCE FOR PERCEPTION OF PARTS
Linguistic Evidence
Phenomenological Demonstrations
Perceptual Experiments
7.6.2 PART SEGMENTATION
Shape Primitives
Boundary Rules
7.6.3 GLOBAL AND LOCAL PROCESSING
Global Precedence
Configural Orientation Effects
Word, Object, and Configural Superiority Effects

Chapter 8 REPRESENTING SHAPE AND STRUCTURE
Object Shape and the 2.5-D Sketch
8.1 SHAPE EQUIVALENCE
8.1.1 DEFINING OBJECTIVE SHAPE
8.1.2 INVARIANT FEATURES
8.1.3 TRANSFORMATIONAL ALIGNMENT
8.1.4 OBJECT-CENTERED REFERENCE FRAMES
Geometric Coordinate Systems
Perceptual Reference Frames
Accounting for Failures of Shape Equivalence
Orientation and Shape
Heuristics in Reference Frame Selection
8.2 THEORIES OF SHAPE REPRESENTATION
8.2.1 TEMPLATES
Strengths
Weaknesses
8.2.2 FOURIER SPECTRA
Strengths
Weaknesses
8.2.3 FEATURES AND DIMENSIONS
Multidimensional Representations
Multifeatural Representations
Strengths
Weaknesses
8.2.4 STRUCTURAL DESCRIPTIONS
Shape Primitives
Strengths
Weaknesses
8.3 FIGURAL GOODNESS AND PRÄGNANZ
8.3.1 THEORIES OF FIGURAL GOODNESS
Classical Information Theory
Rotation and Reflection Subsets
Symmetry Subgroups
Local Symmetry Structure
8.3.2 STRUCTURAL INFORMATION THEORY
Primitive Codes
Removing Redundancies
Information Load
Applications to Perceptual Organization
Strengths
Weaknesses

Chapter 9 PERCEIVING FUNCTION AND CATEGORY
Two Approaches to Perceiving Function
9.1 THE PERCEPTION OF FUNCTION
9.1.1 DIRECT PERCEPTION OF AFFORDANCES
9.1.2 INDIRECT PERCEPTION BY CATEGORIZATION
Four Components of Categorization
Comparison Processes
Decision Processes
9.2 PHENOMENA OF PERCEPTUAL CATEGORIZATION
9.2.1 CATEGORICAL HIERARCHIES
Prototypes
Basic Level Categories
Entry Level Categories
9.2.2 PERSPECTIVE VIEWING CONDITIONS
Canonical Perspective
Priming Effects
Orientation Effects
9.2.3 PART STRUCTURE
9.2.4 CONTEXTUAL EFFECTS
9.2.5 VISUAL AGNOSIA
9.3 THEORIES OF OBJECT CATEGORIZATION
9.3.1 RECOGNITION BY COMPONENTS THEORY
Geons
Geon Relations
Stages of Object Categorization in RBC
A Neural Network Implementation
9.3.2 ACCOUNTING FOR EMPIRICAL PHENOMENA
Typicality Effects
Entry-Level Categories
Viewing Conditions
Part Structures
Contextual Effects
Visual Agnosia
Weaknesses
9.3.3 VIEWPOINT-SPECIFIC THEORIES
The Case for Multiple Views
Aspect Graphs
Alignment with 3-D Models
Alignment with 2-D View Combinations
Weaknesses
9.4 IDENTIFYING LETTERS AND WORDS
9.4.1 IDENTIFYING LETTERS
9.4.2 IDENTIFYING WORDS AND LETTERS WITHIN WORDS
9.4.3 THE INTERACTIVE ACTIVATION MODEL
Feature Level
Letter Level
Word Level
Word-to-Letter Feedback
Problems

Chapter 10 PERCEIVING MOTION AND EVENTS
10.1 IMAGE MOTION
10.1.1 THE COMPUTATIONAL PROBLEM OF MOTION
Space-Time Diagrams
10.1.2 CONTINUOUS MOTION
Adaptation and Aftereffects
Simultaneous Motion Contrast
The Autokinetic Effect
10.1.3 APPARENT MOTION
Early Gestalt Investigations
Motion Picture Technology
The Correspondence Problem of Apparent Motion
Short-Range vs. Long-Range Apparent Motion
The Aperture Problem
10.1.4 PHYSIOLOGICAL MECHANISMS
The Magno and Parvo Systems
Cortical Analysis of Motion
Neuropsychology of Motion Perception
10.1.5 COMPUTATIONAL THEORIES
Delay-and-Compare Networks
Edge-Based Models
Spatial-Frequency-Based Models
Integrating Local Motion
10.2 OBJECT MOTION
10.2.1 PERCEIVING OBJECT VELOCITY
Velocity Constancy
10.2.2 DEPTH AND MOTION
Rigid Motion in Depth
The Kinetic Depth Effect
The Rigidity Heuristic & the Correspondence Problem
The Stereo-Kinetic Effect
Perception of Non-Rigid Motion
10.2.3 LONG-RANGE APPARENT MOTION
Apparent Rotation
Curved Apparent Motion
Conditions for Long-Range Apparent Motion
10.2.4 DYNAMIC PERCEPTUAL ORGANIZATION
Grouping by Movement
Configural Motion
Induced Motion
Kinetic Completion and Illusory Figures
Anorthoscopic Perception
10.3 LOCOMOTION AND OPTIC FLOW
10.3.1 INDUCED MOTION OF THE SELF
Balance And Posture
10.3.2 PERCEIVING SELF-MOTION
Direction Of Self-Motion
Speed Of Self-Motion
Virtual Reality and Ecological Perception
10.4 UNDERSTANDING EVENTS
10.4.1 BIOLOGICAL MOTION
10.4.2 PERCEIVING CAUSATION
Perceiving Mass Relations
10.4.3 INTUITIVE PHYSICS
Recognizing Versus Generating Answers
Particle Versus Extended Body Motion

Chapter 11 VISUAL SELECTION EYE MOVEMENTS AND ATTENTION
11.1 EYE MOVEMENTS
Physiological Nystagmus
11.1.1 CLASSIFICATION OF SELECTIVE EYE MOVEMENTS
Saccadic Movements
Smooth Pursuit Movements
Vergence Movements
Vestibular Movements
Optokinetic Movements
11.1.2 THE PHYSIOLOGY OF THE OCULOMOTOR SYSTEM
11.1.3 SACCADIC EXPLORATION OF THE ENVIRONMENT
Transsaccadic Integration
11.2 VISUAL ATTENTION
Selective Attention
11.2.1 EARLY VERSUS LATE SELECTION
Auditory Attention
The Inattention Paradigm
Intentionally Ignored Information
11.2.2 COSTS AND BENEFITS OF ATTENTION
The Attentional Cuing Paradigm
Voluntary versus Involuntary Shifts of Attention
Three Components of Shifting Attention
11.2.3 THEORIES OF SPATIAL ATTENTION
The Spotlight Metaphor
The Zoom Lens Metaphor
Space-Based versus Object-Based Approaches
11.2.4 SELECTIVE ATTENTION TO PROPERTIES
The Stroop Effect
Integral versus Separable Dimensions
11.2.5 DISTRIBUTED VERSUS FOCUSSED ATTENTION
Visual Pop-Out
Search Asymmetry
11.2.6 FEATURE INTEGRATION THEORY
Conjunction Search
Texture Segregation
Illusory Conjunctions
Problems with Feature Integration Theory
Object Files
11.2.7 THE PHYSIOLOGY OF ATTENTION
Unilateral Neglect
Balint's Syndrome
Brain Imaging Studies
Electrophysiological Studies
11.2.8 ATTENTION AND EYE MOVEMENTS

Chapter 12 VISUAL MEMORY AND IMAGERY
12.1 VISUAL MEMORY
12.1.1 THREE MEMORY SYSTEMS
Five Characteristics of Memory Systems
12.1.2 ICONIC MEMORY
The Partial Report Procedure
Duration
Content
Maintenance
Loss
Masking
Persistence versus Processing
12.1.3 VISUAL SHORT-TERM MEMORY
Visual STM versus Iconic Memory
Visual STM versus Visual LTM
The Visuo-Spatial Scratch Pad
Transsaccadic Memory
12.1.4 LONG-TERM VISUAL MEMORY
Three Types of LTM
Visual Routines
Recall versus Recognition
How Good is Episodic Visual LTM?
Visual Imagery as a Mnemonic Device
Dual Coding Theory
Photographic Memory
Mnemonists
Neuropsychology of Visual Memory
12.1.5 MEMORY DYNAMICS
Tendencies toward Goodness
Effects of Verbal Labels
The Misinformation Effect
Representational Momentum
12.2 VISUAL IMAGERY
The History of Imagery Research
12.2.1 THE ANALOG/PROPOSITIONAL DEBATE
The Analog Position
The Propositional Position
12.2.2 MENTAL TRANSFORMATIONS
Mental Rotation
Other Transformations
12.2.3 IMAGE INSPECTION
Image Scanning
Image Size Effects
Mental Psychophysics
Reinterpreting Images
12.2.4 KOSSLYN'S MODEL OF IMAGERY
12.2.5 THE RELATION OF IMAGERY TO PERCEPTION
Behavioral Evidence
Neuropsychological Evidence
Brain Imaging Studies

Chapter 13 VISUAL AWARENESS
Unconscious Processes in Vision
The Explanatory Gap
13.1 PHILOSOPHICAL FOUNDATIONS
13.1.1 THE MIND-BODY PROBLEM
Dualism
Idealism
Materialism
Behaviorism
Functionalism
Supervenience
13.1.2 THE PROBLEM OF OTHER MINDS
Criteria for Consciousness
The Inverted Spectrum Argument
Phenomenological Criteria
Behavioral Criteria
Physiological Criteria
Correlational versus Causal Theories
13.2 THE NEUROPSYCHOLOGY OF VISUAL AWARENESS
13.2.1 SPLIT-BRAIN PATIENTS
13.2.2 BLINDSIGHT
The Case History of D.B.
Accurate Guessing without Visual Experience
The Two Visual Systems Hypothesis
Methodological Challenges
13.2.3 UNCONSCIOUS PROCESSING IN NEGLECT AND BALINT'S SYNDROME
13.2.4 UNCONSCIOUS FACE RECOGNITION IN PROSOPAGNOSIA
13.3 VISUAL AWARENESS IN NORMAL OBSERVERS
13.3.1 PERCEPTUAL DEFENSE
13.3.2 SUBLIMINAL PERCEPTION
Marcel's Experiments
Objective versus Subjective Thresholds of Awareness
Functional Correlates of Consciousness
13.3.3 INATTENTIONAL BLINDSIGHT
13.4 THEORIES OF CONSCIOUSNESS
The Activation Assumption
13.4.1 FUNCTIONAL ARCHITECTURE THEORIES
The STM Hypothesis
An Activation-based Conception of STM
The Attention Hypothesis
Working Memory Theories
The 2.5-D Sketch Theory of Consciousness
13.4.2 BIOLOGICAL THEORIES
Activation Thresholds
Duration Thresholds
The Cortical Hypothesis
The Crick/Koch Conjectures
ERTAS The Extended Reticular-Thalamic Activating System
Causal Theories of Consciousness An Analogy
13.4.3 CONSCIOUSNESS AND THE LIMITS OF SCIENCE
Relational Structure
The Isomorphism Constraint
Relational to Functionalism
Biology to the Rescue?

APPENDIX 1 PSYCHOPHYSICAL METHODS
A1.1 MEASURING THRESHOLDS
Method of Adjustment
Method of Limits
Method of Constant Stimuli
The Theoretical Status of Thresholds
A1.2 SIGNAL DETECTION THEORY
Response Bias
The Signal Detection Paradigm
The Theory of Signal Detectability
A1.3 DIFFERENCE THRESHOLDS
Just Noticeable Differences
Weber's Law
A1.4 PSYCHOPHYSICAL SCALING
Fechner's Law
Stevens' Law

APPENDIX 2 CONNECTIONIST MODELLING
A2.1 NETWORK BEHAVIOR
A2.1.1 UNIT BEHAVIOR
Combining Input Activation
Determining Output Activation
A2.1.2 SYSTEM ARCHITECTURE
Feedforward Networks
Feedback Networks
Symmetric Networks
Winner-Take-All Networks
A2.1.3 SYSTEMIC BEHAVIOR
Graceful Degradation
Settling into a Stable State
Soft Constraint Satisfaction
Pattern Completion
A2.2 CONNECTIONIST LEARNING ALGORITHMS
A2.2.1 BACK PROPAGATION
The Delta Rule
The Generalized Delta Rule
A2.2.2 GRADIENT DESCENT
Input Vector Space
Partitioning the Input Vector Space
State Space
Weight Space
Weight-Error Space
Gradient Descent
Local versus Global Minima

APPENDIX 3 COLOR TECHNOLOGY
Metamers
A3.1 ADDITIVE VERSUS SUBTRACTIVE COLOR MIXTURE
Adding versus Multiplying Spectra
Maxwell's Color Triangle
C.I.E. Color Space.
Subtractive Color Mixture Space?
A3.2 COLOR TELEVISION
A3.3 PAINTS AND DYES
Subtractive Combination of Paints
Additive Combination of Paints
A3.4 COLOR PHOTOGRAPHY
A3.5 COLOR PRINTING