1 Approach and scope |
|
1 | (15) |
|
|
1 | (3) |
|
1.1.1 Data, models, and theory |
|
|
1 | (2) |
|
1.1.2 From physiology to behavior and back via theory and models |
|
|
3 | (1) |
|
1.2 The problem of vision |
|
|
4 | (12) |
|
1.2.1 Visual tasks and subtasks |
|
|
5 | (2) |
|
1.2.2 Vision seen through visual encoding, selection, and decoding |
|
|
7 | (3) |
|
1.2.3 Visual encoding in retina and V1 |
|
|
10 | (2) |
|
1.2.4 Visual selection and V1's role in it |
|
|
12 | (2) |
|
1.2.5 Visual decoding and its associated brain areas |
|
|
14 | (2) |
2 A very brief introduction of what is known about vision experimentally |
|
16 | (51) |
|
2.1 Neurons, neural circuits, and brain regions |
|
|
16 | (6) |
|
2.1.1 Neurons, somas, dendrites, axons, and action potentials |
|
|
16 | (1) |
|
2.1.2 A simple neuron model |
|
|
17 | (1) |
|
2.1.3 Random processes of action potential generation in neurons |
|
|
18 | (1) |
|
2.1.4 Synaptic connections, neural circuits, and brain areas |
|
|
18 | (1) |
|
2.1.5 Visual processing areas along the visual pathway |
|
|
19 | (3) |
|
|
22 | (17) |
|
2.2.1 Receptive fields of retinal ganglion cells |
|
|
22 | (3) |
|
2.2.2 Sensitivity to sinusoidal gratings, and contrast sensitivity curves |
|
|
25 | (5) |
|
2.2.3 Responses to spatiotemporal inputs |
|
|
30 | (4) |
|
|
34 | (1) |
|
2.2.5 Color processing in the retina |
|
|
35 | (2) |
|
2.2.6 Spatial sampling in the retina |
|
|
37 | (1) |
|
2.2.7 LGN on the pathway from the retina to V1 |
|
|
38 | (1) |
|
|
39 | (15) |
|
2.3.1 The retinotopic map |
|
|
39 | (1) |
|
2.3.2 The receptive fields in V1-the feature detectors |
|
|
40 | (1) |
|
2.3.3 Orientation selectivity, bar and edge detectors |
|
|
41 | (1) |
|
2.3.4 Spatial frequency tuning and multiscale coding |
|
|
42 | (1) |
|
2.3.5 Temporal and motion direction selectivity |
|
|
43 | (3) |
|
2.3.6 Ocular dominance and disparity selectivity |
|
|
46 | (2) |
|
2.3.7 Color selectivity of V1 neurons |
|
|
48 | (1) |
|
|
48 | (4) |
|
2.3.9 The influences on a V1 neuron's response from contextual stimuli outside the receptive field |
|
|
52 | (2) |
|
|
54 | (6) |
|
2.4.1 Two processing streams |
|
|
54 | (1) |
|
|
55 | (2) |
|
|
57 | (2) |
|
|
59 | (1) |
|
2.4.5 IT and temporal cortical areas for object recognition |
|
|
60 | (1) |
|
2.5 Eye movements, their associated brain regions, and links with attention |
|
|
60 | (3) |
|
2.5.1 Close link between eye movements and attention |
|
|
62 | (1) |
|
2.6 Top-down attention and neural responses |
|
|
63 | (2) |
|
2.7 Behavioral studies on vision |
|
|
65 | (2) |
3 The efficient coding principle |
|
67 | (110) |
|
3.1 A brief introduction to information theory |
|
|
68 | (9) |
|
3.1.1 Measuring information |
|
|
68 | (2) |
|
3.1.2 Information transmission, information channels, and mutual information |
|
|
70 | (4) |
|
3.1.3 Information redundancy, representation efficiency, and error correction |
|
|
74 | (3) |
|
3.2 Formulation of the efficient coding principle |
|
|
77 | (6) |
|
3.2.1 An optimization problem |
|
|
77 | (2) |
|
|
79 | (4) |
|
3.3 Efficient neural sampling in the retina |
|
|
83 | (7) |
|
3.3.1 Contrast sampling in a fly's compound eye |
|
|
83 | (2) |
|
3.3.2 Spatial sampling by receptor distribution on the retina |
|
|
85 | (4) |
|
3.3.3 Optimal color sampling by the cones |
|
|
89 | (1) |
|
3.4 Efficient coding by visual receptive field transforms |
|
|
90 | (6) |
|
3.4.1 The general analytical solution for efficient coding of Gaussian signals |
|
|
91 | (5) |
|
3.5 Case study: stereo coding in V1 as an efficient transform of inputs in the dimension of ocularity |
|
|
96 | (24) |
|
3.5.1 Principal component analysis K0 |
|
|
98 | (4) |
|
|
102 | (3) |
|
3.5.3 Contrast enhancement, decorrelation, and whitening in the high S/N regime |
|
|
105 | (1) |
|
3.5.4 Many equivalent solutions of optimal encoding |
|
|
106 | (2) |
|
3.5.5 Smoothing and output correlation in the low S/N region |
|
|
108 | (2) |
|
3.5.6 A special, most local, class of optimal coding |
|
|
110 | (1) |
|
3.5.7 Adaptation of the optimal code to the statistics of the input environment |
|
|
110 | (7) |
|
3.5.8 A psychophysical test of the adaptation of the efficient stereo coding |
|
|
117 | (3) |
|
3.5.9 How might one test the predictions physiologically? |
|
|
120 | (1) |
|
3.6 The efficient receptive field transforms in space, color, time, and scale in the retina and V1 |
|
|
120 | (50) |
|
3.6.1 Efficient spatial coding in the retina |
|
|
123 | (11) |
|
3.6.2 Efficient coding in time |
|
|
134 | (4) |
|
3.6.3 Efficient coding in color |
|
|
138 | (4) |
|
3.6.4 Coupling space and color coding in the retina |
|
|
142 | (5) |
|
3.6.5 Spatial coding in V1 |
|
|
147 | (7) |
|
3.6.6 Coupling the spatial and color coding in V1 |
|
|
154 | (7) |
|
3.6.7 Coupling spatial coding with stereo coding in V1-coding disparity |
|
|
161 | (3) |
|
3.6.8 Coupling space and time coding in the retina and V1 |
|
|
164 | (3) |
|
3.6.9 V1 neurons tuned simultaneously to multiple feature dimensions |
|
|
167 | (3) |
|
3.7 The efficient code, and the related sparse code, in low noise limit by numerical simulations |
|
|
170 | (3) |
|
|
171 | (2) |
|
3.8 How to get efficient codes by developmental rules and unsupervised learning |
|
|
173 | (4) |
|
3.8.1 Learning for a single encoding neuron |
|
|
174 | (1) |
|
3.8.2 Learning simultaneously for multiple encoding neurons |
|
|
175 | (2) |
4 V1 and information coding |
|
177 | (12) |
|
4.1 Pursuit of efficient coding in V1 by reducing higher order redundancy |
|
|
177 | (9) |
|
4.1.1 Higher order statistics contain much of the meaningful information about visual objects |
|
|
178 | (2) |
|
4.1.2 Characterizing higher order statistics |
|
|
180 | (3) |
|
4.1.3 Efforts to understand V1 neural properties from the perspective of reducing higher order redundancy |
|
|
183 | (2) |
|
4.1.4 Higher order redundancy in natural images is only a very small fraction of the total redundancy |
|
|
185 | (1) |
|
4.2 Problems in understanding V1 solely based on efficient coding |
|
|
186 | (1) |
|
4.3 Multiscale and overcomplete representation in V1 is useful for invariant object recognition from responses of selected neural subpopulations |
|
|
187 | (2) |
|
4.3.1 Information selection, amount, and meaning |
|
|
188 | (1) |
5 The V1 hypothesis-creating a bottom-up saliency map for preattentive selection and segmentation |
|
189 | (126) |
|
5.1 Visual selection and visual saliency |
|
|
189 | (12) |
|
5.1.1 Visual selection-top-down and bottom-up selections |
|
|
189 | (6) |
|
5.1.2 A brief overview of visual search and segmentation-behavioral studies of saliency |
|
|
195 | (2) |
|
5.1.3 Saliency regardless of visual input features |
|
|
197 | (3) |
|
5.1.4 A quick review of what we should expect about saliencies and a saliency map |
|
|
200 | (1) |
|
5.2 The V1 saliency hypothesis |
|
|
201 | (8) |
|
5.2.1 Detailed formulation of the V1 saliency hypothesis |
|
|
202 | (2) |
|
5.2.2 Intracortical interactions in V1 as mechanisms to compute saliency |
|
|
204 | (2) |
|
5.2.3 Reading out the saliency map |
|
|
206 | (1) |
|
5.2.4 Statistical and operational definitions of saliency |
|
|
207 | (1) |
|
5.2.5 Overcomplete representation in V1 for the role of saliency |
|
|
208 | (1) |
|
5.3 A hallmark of the saliency map in V1-attention capture by an ocular singleton which is barely distinctive to perception |
|
|
209 | (6) |
|
5.3.1 Food for thought: looking (acting) before or without seeing |
|
|
215 | (1) |
|
5.4 Testing and understanding the V1 saliency map in a V1 model |
|
|
215 | (37) |
|
5.4.1 The V1 model: its neural elements, connections, and desired behavior |
|
|
216 | (6) |
|
5.4.2 Calibration of the V1 model to biological reality |
|
|
222 | (3) |
|
5.4.3 Computational requirements on the dynamic behavior of the model |
|
|
225 | (2) |
|
5.4.4 Applying the V1 model to visual search and visual segmentation |
|
|
227 | (20) |
|
5.4.5 Other effects of the saliency mechanisms-figure-ground segmentation and the medial axis effect |
|
|
247 | (3) |
|
5.4.6 Input contrast dependence of the contextual influences |
|
|
250 | (1) |
|
5.4.7 Reflections from the V1 model |
|
|
250 | (2) |
|
5.5 Additional psychophysical tests of the V1 saliency hypothesis |
|
|
252 | (17) |
|
5.5.1 The feature-blind "auction"-maximum rather than summation over features |
|
|
252 | (5) |
|
5.5.2 The fingerprints of colinear facilitation in V1 |
|
|
257 | (3) |
|
5.5.3 The fingerprint of V1's conjunctive cells |
|
|
260 | (6) |
|
5.5.4 A zero-parameter quantitative prediction and its experimental test |
|
|
266 | (3) |
|
5.5.5 Reflections-from behavior back to physiology via the V1 saliency hypothesis |
|
|
269 | (1) |
|
5.6 The roles of V1 and other cortical areas in visual selection |
|
|
269 | (10) |
|
5.6.1 Using visual depth feature to probe contributions of extrastriate cortex to atten- tional control |
|
|
271 | (4) |
|
5.6.2 Salient but indistinguishable inputs activate early visual cortical areas but not the parietal and frontal areas |
|
|
275 | (4) |
|
5.7 V1's role beyond saliency-selection versus decoding, periphery versus central vision |
|
|
279 | (6) |
|
5.7.1 Implications for the functional roles of visual cortical areas based on their repre- sentations of the visual field |
|
|
281 | (1) |
|
5.7.2 Saliency, visual segmentation, and visual recognition |
|
|
282 | (3) |
|
5.8 Nonlinear V1 neural dynamics for saliency and preattentive segmentation |
|
|
285 | (28) |
|
5.8.1 A minimal model of the primary visual cortex for saliency computation |
|
|
286 | (13) |
|
5.8.2 Dynamic analysis of the V1 model and constraints on the neural connections |
|
|
299 | (13) |
|
5.8.3 Extensions and generalizations |
|
|
312 | (1) |
|
5.9 Appendix: parameters in the V1 model |
|
|
313 | (2) |
6 Visual recognition as decoding |
|
315 | (49) |
|
6.1 Definition of visual decoding |
|
|
315 | (2) |
|
6.2 Some notable observations about visual recognition |
|
|
317 | (9) |
|
6.2.1 Recognition is after an initial selection or segmentation |
|
|
317 | (1) |
|
|
318 | (1) |
|
6.2.3 Is decoding the shape of an object in the attentional spotlight a default routine? |
|
|
319 | (2) |
|
6.2.4 Recognition by imagination or input synthesis |
|
|
321 | (2) |
|
6.2.5 Visual perception can be ambiguous or unambiguous |
|
|
323 | (2) |
|
6.2.6 Neural substrates for visual decoding |
|
|
325 | (1) |
|
6.3 Visual decoding from neural responses |
|
|
326 | (21) |
|
6.3.1 Example: decoding motion direction from MT neural responses |
|
|
327 | (3) |
|
6.3.2 Example: discriminating two inputs based on photoreceptor responses |
|
|
330 | (2) |
|
6.3.3 Example: discrimination by decoding the V1 neural responses |
|
|
332 | (2) |
|
6.3.4 Example: light wavelength discrimination by decoding from cone responses |
|
|
334 | (4) |
|
6.3.5 Perception, including illusion, of a visual feature value by neural population de- coding |
|
|
338 | (7) |
|
6.3.6 Poisson-like neural noise and increasing perceptual performance for stronger visual inputs |
|
|
345 | (1) |
|
6.3.7 Low efficiency of sensory information utilization by the central visual system |
|
|
345 | (1) |
|
6.3.8 Transduction and central inefficiencies in the framework of encoding, attentional selection, and decoding |
|
|
346 | (1) |
|
6.4 Bayesian inference and the influence of prior belief in visual decoding |
|
|
347 | (14) |
|
6.4.1 The Bayesian framework |
|
|
348 | (1) |
|
6.4.2 Bayesian visual inference is highly complex unless the number and the dimen sions of possible percepts are restricted |
|
|
349 | (1) |
|
6.4.3 Behavioral evidence for Bayesian visual inference |
|
|
350 | (11) |
|
6.5 The initial visual recognition, feedforward mechanisms, and recurrent neural connections |
|
|
361 | (3) |
|
6.5.1 The fast speed of coarse initial recognition by the primate visual system |
|
|
361 | (1) |
|
6.5.2 Object detection and recognition by models of hierarchical feedforward networks |
|
|
361 | (2) |
|
6.5.3 Combining feedforward and feedback intercortical mechanisms, and recurrent intracortical mechanisms, for object inference |
|
|
363 | (1) |
7 Epilogue |
|
364 | (3) |
|
7.1 Our ignorance of vision viewed from the perspective of vision as encoding, selection, and decoding |
|
|
364 | (1) |
|
|
365 | (2) |
References |
|
367 | (13) |
Index |
|
380 | |