Atjaunināt sīkdatņu piekrišanu

E-grāmata: Inference and Learning from Data: Volume 1: Foundations

(École Polytechnique Fédérale de Lausanne)
  • Formāts: PDF+DRM
  • Izdošanas datums: 22-Dec-2022
  • Izdevniecība: Cambridge University Press
  • Valoda: eng
  • ISBN-13: 9781009218139
  • Formāts - PDF+DRM
  • Cena: 101,11 €*
  • * ši ir gala cena, t.i., netiek piemērotas nekādas papildus atlaides
  • Ielikt grozā
  • Pievienot vēlmju sarakstam
  • Šī e-grāmata paredzēta tikai personīgai lietošanai. E-grāmatas nav iespējams atgriezt un nauda par iegādātajām e-grāmatām netiek atmaksāta.
  • Formāts: PDF+DRM
  • Izdošanas datums: 22-Dec-2022
  • Izdevniecība: Cambridge University Press
  • Valoda: eng
  • ISBN-13: 9781009218139

DRM restrictions

  • Kopēšana (kopēt/ievietot):

    nav atļauts

  • Drukāšana:

    nav atļauts

  • Lietošana:

    Digitālo tiesību pārvaldība (Digital Rights Management (DRM))
    Izdevējs ir piegādājis šo grāmatu šifrētā veidā, kas nozīmē, ka jums ir jāinstalē bezmaksas programmatūra, lai to atbloķētu un lasītu. Lai lasītu šo e-grāmatu, jums ir jāizveido Adobe ID. Vairāk informācijas šeit. E-grāmatu var lasīt un lejupielādēt līdz 6 ierīcēm (vienam lietotājam ar vienu un to pašu Adobe ID).

    Nepieciešamā programmatūra
    Lai lasītu šo e-grāmatu mobilajā ierīcē (tālrunī vai planšetdatorā), jums būs jāinstalē šī bezmaksas lietotne: PocketBook Reader (iOS / Android)

    Lai lejupielādētu un lasītu šo e-grāmatu datorā vai Mac datorā, jums ir nepieciešamid Adobe Digital Editions (šī ir bezmaksas lietotne, kas īpaši izstrādāta e-grāmatām. Tā nav tas pats, kas Adobe Reader, kas, iespējams, jau ir jūsu datorā.)

    Jūs nevarat lasīt šo e-grāmatu, izmantojot Amazon Kindle.

Written in an engaging and rigorous style by a world authority in the field, this is an accessible and comprehensive introduction to core topics in inference and learning. With downloadable Matlab code and solutions for instructors, this is the ideal introduction for students of data science, machine learning, and engineering.

This extraordinary three-volume work, written in an engaging and rigorous style by a world authority in the field, provides an accessible, comprehensive introduction to the full spectrum of mathematical and statistical techniques underpinning contemporary methods in data-driven learning and inference. This first volume, Foundations, introduces core topics in inference and learning, such as matrix theory, linear algebra, random variables, convex optimization and stochastic optimization, and prepares students for studying their practical application in later volumes. A consistent structure and pedagogy is employed throughout this volume to reinforce student understanding, with over 600 end-of-chapter problems (including solutions for instructors), 100 figures, 180 solved examples, datasets and downloadable Matlab code. Supported by sister volumes Inference and Learning, and unique in its scale and depth, this textbook sequence is ideal for early-career researchers and graduate students across many courses in signal processing, machine learning, statistical analysis, data science and inference.

Recenzijas

'Inference and Learning from Data is a uniquely comprehensive introduction to the signal processing foundations of modern data science. Lucidly written, with a carefully balanced choice of topics, this textbook is an indispensable resource for both graduate students and data science practitioners, a piece of lasting value.' Helmut Bölcskei, ETH Zurich 'This textbook provides a lucid and magisterial treatment of methods for inference and learning from data, aided by hundreds of solved examples, computer simulations, and over 1000 problems. The material ranges from fundamentals to recent advances in statistical learning theory; variational inference; neural, convolutional, and Bayesian networks; and several other topics. It is aimed at students and practitioners, and can be used for several different introductory and advanced courses.' Thomas Kailath, Stanford University 'A tour de force comprehensive three-volume set for the fast-developing areas of data science, machine learning, and statistical signal processing. With masterful clarity and depth, Sayed covers, connects, and integrates background fundamentals and classical and emerging methods in inference and learning. The books are rich in worked-out examples, exercises, and links to data sets. Commentaries with historical background and contexts for the topics covered in each chapter are a special feature.' Mostafa Kaveh, University of Minnesota 'This is the first of a three-volume series covering from fundamentals to the many various methods in inference and learning from data. Professor Sayed is a prolific author of award-winning books and research papers who has himself contributed significantly to many of the topics included in the series. With his encyclopedic knowledge, his careful attention to detail, and in a very approachable style, this first volume covers the basics of matrix theory, probability and stochastic processes, convex and non-convex optimization, gradient-descent, convergence analysis, and several other advanced topics that will be needed for volume II (Inference) and volume III (Learning). This series, and in particular this volume, will be a must-have for educators, students, researchers, and technologists alike who are pursuing a systematic study, want a quick refresh, or may use it as a helpful reference to learn about these fundamentals.' Jose Moura, Carnegie Mellon University 'Volume I of Inference and Learning from Data provides a foundational treatment of one of the most topical aspects of contemporary signal and information processing, written by one of the most talented expositors in the field. It is a valuable resource both as a textbook for students wishing to enter the field and as a reference work for practicing engineers.' Vincent Poor, Princeton University 'Inference and Learning from Data, Vol. I: Foundations offers an insightful and well-integrated primer with just the right balance of everything that new graduate students need to put their research on a solid footing. It covers foundations in a modern way - emphasizing the most useful concepts, including proofs, and timely topics which are often missing from introductory graduate texts. All in one beautifully written textbook. An impressive feat! I highly recommend it.' Nikolaos Sidiropoulos, University of Virginia 'This exceptional encyclopedic work on learning from data will be the bible of the field for many years to come. Totaling more than 3000 pages, this three-volume book covers in an exhaustive and timely manner the topic of data science, which has become critically important to many areas and lies at the basis of modern signal processing, machine learning, artificial intelligence, and their numerous applications. Written by an authority in the field, the book is really unique in scale and breadth, and it will be an invaluable source of information for students, researchers, and practitioners alike.' Peter Stoica, Uppsala University 'Very meticulous, thorough, and timely. This volume is largely focused on optimization, which is so important in the modern-day world of data science, signal processing, and machine learning. The book is classical and modern at the same time - many classical topics are nicely linked to modern topics of current interest. All the necessary mathematical background is covered. Professor Sayed is one of the foremost researchers and educators in the field and the writing style is unhurried and clear with many examples, truly reflecting the towering scholar that he is. This volume is so complete that it can be used for self-study, as a classroom text, and as a timeless research reference.' P. P. Vaidyanathan, Caltech 'The book series is timely and indispensable. It is a unique companion for graduate students and early-career researchers. The three volumes provide an extraordinary breadth and depth of techniques and tools, and encapsulate the experience and expertise of a world-class expert in the field. The pedagogically crafted text is written lucidly, yet never compromises rigor. Theoretical concepts are enhanced with illustrative figures, well-thought problems, intuitive examples, datasets, and MATLAB codes that reinforce readers' learning.' Abdelhak Zoubir, TU Darmstadt

Papildus informācija

Discover core topics in inference and learning with the first volume of this extraordinary three-volume set.
Volume I Foundations
Preface xxvii
P.1 Emphasis on Foundations xxvii
P.2 Glimpse of History xxix
P.3 Organization of the Text xxxi
P.4 How to Use the Text xxxiv
P.5 Simulation Datasets xxxvii
P.6 Acknowledgments xi
Notation xiv
1 Matrix Theory
1(58)
1.1 Symmetric Matrices
1(4)
1.2 Positive-Definite Matrices
5(2)
1.3 Range Spaces and Nullspaces
7(4)
1.4 Schur Complements
11(3)
1.5 Cholesky Factorization
14(4)
1.6 QR Decomposition
18(2)
1.7 Singular Value Decomposition
20(2)
1.8 Square-Root Matrices
22(2)
1.9 Kronecker Products
24(6)
1.10 Vector and Matrix Norms
30(7)
1.11 Perturbation Bounds on Eigenvalues
37(1)
1.12 Stochastic Matrices
38(1)
1.13 Complex-Valued Matrices
39(2)
1.14 Commentaries and Discussion
41(9)
Problems
47(3)
1.A Proof of Spectral Theorem
50(2)
1.B Constructive Proof of SVD
52(7)
References
53(6)
2 Vector Differentiation
59(9)
2.1 Gradient Vectors
59(3)
2.2 Hessian Matrices
62(1)
2.3 Matrix Differentiation
63(2)
2.4 Commentaries and Discussion
65(3)
Problems
65(2)
References
67(1)
3 Random Variables
68(64)
3.1 Probability Density Functions
68(3)
3.2 Mean and Variance
71(6)
3.3 Dependent Random Variables
77(16)
3.4 Random Vectors
93(3)
3.5 Properties of Covariance Matrices
96(1)
3.6 Illustrative Applications
97(9)
3.7 Complex-Valued Variables
106(3)
3.8 Commentaries and Discussion
109(10)
Problems
112(7)
3.A Convergence of Random Variables
119(3)
3.B Concentration Inequalities
122(10)
References
128(4)
4 Gaussian Distribution
132(35)
4.1 Scalar Gaussian Variables
132(2)
4.2 Vector Gaussian Variables
134(4)
4.3 Useful Gaussian Manipulations
138(6)
4.4 Jointly Distributed Gaussian Variables
144(6)
4.5 Gaussian Processes
150(5)
4.6 Circular Gaussian Distribution
155(2)
4.7 Commentaries and Discussion
157(10)
Problems
160(5)
References
165(2)
5 Exponential Distributions
167(29)
5.1 Definition
167(2)
5.2 Special Cases
169(9)
5.3 Useful Properties
178(5)
5.4 Conjugate Priors
183(4)
5.5 Commentaries and Discussion
187(5)
Problems
189(3)
5.A Derivation of Properties
192(4)
References
195(1)
6 Entropy and Divergence
196(44)
6.1 Information and Entropy
196(8)
6.2 Kullback Leibler Divergence
204(5)
6.3 Maximum Entropy Distribution
209(2)
6.4 Moment Matching
211(2)
6.5 Fisher Information Matrix
213(4)
6.6 Natural Gradients
217(10)
6.7 Evidence Lower Bound
227(4)
6.8 Commentaries and Discussion
231(9)
Problems
234(3)
References
237(3)
7 Random Processes
240(21)
7.1 Stationary Processes
240(5)
7.2 Power Spectral Density
245(7)
7.3 Spectral Factorization
252(3)
7.4 Commentaries and Discussion
255(6)
Problems
257(2)
References
259(2)
8 Convex Functions
261(41)
8.1 Convex Sets
261(2)
8.2 Convexity
263(2)
8.3 Strict Convexity
265(1)
8.4 Strong Convexity
266(2)
8.5 Hessian Matrix Conditions
268(4)
8.6 Sub-gradient Vectors
272(7)
8.7 Jensen Inequality
279(2)
8.8 Conjugate Functions
281(4)
8.9 Bregman Divergence
285(5)
8.10 Commentaries and Discussion
290(12)
Problems
293(6)
References
299(3)
9 Convex Optimization
302(28)
9.1 Convex Optimization Problems
302(8)
9.2 Equality Constraints
310(2)
9.3 Motivating the KKT Conditions
312(3)
9.4 Projection onto Convex Sets
315(7)
9.5 Commentaries and Discussion
322(8)
Problems
323(5)
References
328(2)
10 Lipschitz Conditions
330(11)
10.1 Mean-Value Theorem
330(2)
10.2 δ-Smooth Functions
332(5)
10.3 Commentaries and Discussion
337(4)
Problems
338(2)
References
340(1)
11 Proximal Operator
341(34)
11.1 Definition and Properties
341(6)
11.2 Proximal Point Algorithm
347(2)
11.3 Proximal Gradient Algorithm
349(5)
11.4 Convergence Results
354(2)
11.5 Douglas-Rachford Algorithm
356(2)
11.6 Commentaries and Discussion
358(8)
Problems
362(4)
11.A Convergence under Convexity
366(3)
11.B Convergence under Strong Convexity
369(6)
References
372(3)
12 Gradient-Descent Method
375(66)
12.1 Empirical and Stochastic Risks
375(4)
12.2 Conditions on Risk Function
379(2)
12.3 Constant Step Sizes
381(11)
12.4 Iteration-Dependent Step-Sizes
392(10)
12.5 Coordinate-Descent Method
402(11)
12.6 Alternating Projection Algorithm
413(5)
12.7 Commentaries and Discussion
418(15)
Problems
425(8)
12.A Zeroth-Order Optimization
433(8)
References
436(5)
13 Conjugate Gradient Method
441(30)
13.1 Linear Systems of Equations
441(13)
13.2 Nonlinear Optimization
454(5)
13.3 Convergence Analysis
459(6)
13.4 Commentaries and Discussion
465(6)
Problems
466(3)
References
469(2)
14 Subgradient Method
471(36)
14.1 Subgradient Algorithm
471(4)
14.2 Conditions on Risk Function
475(4)
14.3 Convergence Behavior
479(4)
14.4 Pocket Variable
483(3)
14.5 Exponential Smoothing
486(3)
14.6 Iteration-Dependent Step Sizes
489(4)
14.7 Coordinate-Descent Algorithms
493(3)
14.8 Commentaries and Discussion
496(5)
Problems
498(3)
14.A Deterministic Inequality Recursion
501(6)
References
505(2)
15 Proximal and Mirror-Descent Methods
507(40)
15.1 Proximal Gradient Method
507(8)
15.2 Projection Gradient Method
515(4)
15.3 Mirror-Descent Method
519(18)
15.4 Comparison of Convergence Rates
537(2)
15.5 Commentaries and Discussion
539(8)
Problems
541(3)
References
544(3)
16 Stochastic Optimization
547(52)
16.1 Stochastic Gradient Algorithm
548(17)
16.2 Stochastic Subgradient Algorithm
565(4)
16.3 Stochastic Proximal Gradient Algorithm
569(5)
16.4 Gradient Noise
574(2)
16.5 Regret Analysis
576(6)
16.6 Commentaries and Discussion
582(8)
Problems
586(4)
16.A Switching Expectation and Differentiation
590(9)
References
595(4)
17 Adaptive Gradient Methods
599(33)
17.1 Motivation
599(4)
17.2 AdaGrad Algorithm
603(5)
17.3 RMSprop Algorithm
608(2)
17.4 ADAM Algorithm
610(4)
17.5 Momentum Acceleration Methods
614(5)
17.6 Federated Learning
619(7)
17.7 Commentaries and Discussion
626(6)
Problems
630(2)
17 A Regret Analysis for ADAM
632(10)
References
640(2)
18 Gradient Noise
642(41)
18.1 Motivation
642(3)
18.2 Smooth Risk Functions
645(3)
18.3 Gradient Noise for Smooth Risks
648(12)
18.4 Nonsmooth Risk Functions
660(5)
18.5 Gradient Noise for Nonsmooth Risks
665(8)
18.6 Commentaries and Discussion
673(4)
Problems
675(2)
18.A Averaging over Mini-Batches
677(2)
18.B Auxiliary Variance Result
679(4)
References
681(2)
19 Convergence Analysis I: Stochastic Gradient Algorithms
683(47)
19.1 Problem Setting
683(3)
19.2 Convergence under Uniform Sampling
686(5)
19.3 Convergence of Mini-Batch Implementation
691(1)
19.4 Convergence under Vanishing Step Sizes
692(6)
19.5 Convergence under Random Reshuffling
698(3)
19.6 Convergence under Importance Sampling
701(6)
19.7 Convergence of Stochastic Conjugate Gradient
707(5)
19.8 Commentaries and Discussion
712(8)
Problems
716(4)
19.A Stochastic Inequality Recursion
720(2)
19.B Proof of Theorem 19.5
722(8)
References
727(3)
20 Convergence Analysis II: Stochastic Subgradient Algorithms
730(26)
20.1 Problem Setting
730(5)
20.2 Convergence under Uniform Sampling
735(3)
20.3 Convergence with Pocket Variables
738(2)
20.4 Convergence with Exponential Smoothing
740(5)
20.5 Convergence of Mini-Batch Implementation
745(2)
20.6 Convergence under Vanishing Step Sizes
747(3)
20.7 Commentaries and Discussion
750(6)
Problems
753(1)
References
754(2)
21 Convergence Analysis III: Stochastic Proximal Algorithms
756(23)
21.1 Problem Setting
756(5)
21.2 Convergence under Uniform Sampling
761(4)
21.3 Convergence of Mini-Batch Implementation
765(1)
21.4 Convergence under Vanishing Step Sizes
766(3)
21.5 Stochastic Projection Gradient
769(2)
21.6 Mirror-Descent Algorithm
771(3)
21.7 Commentaries and Discussion
774(5)
Problems
775(1)
References
776(3)
22 Variance-Reduced Methods I: Uniform Sampling
779(37)
22.1 Problem Setting
779(3)
22.2 Naive Stochastic Gradient Algorithm
782(3)
22.3 Stochastic Average-Gradient Algorithm (SAGA)
785(8)
22.4 Stochastic Variance-Reduced Gradient Algorithm (SVRG)
793(6)
22.5 Nonsmooth Risk Functions
799(7)
22.6 Commentaries and Discussion
806(4)
Problems
808(2)
22.A Proof of Theorem 22.2
810(3)
22.B Proof of Theorem 22.3
813(3)
References
815(1)
23 Variance-Reduced Methods II: Random Reshuffling
816(36)
23.1 Amortized Variance-Reduced Gradient Algorithm (AVRG)
816(2)
23.2 Evolution of Memory Variables
818(4)
23.3 Convergence of SAGA
822(5)
23.4 Convergence of AVRG
827(3)
23.5 Convergence of SVRG
830(1)
23.6 Nonsmooth Risk Functions
831(1)
23.7 Commentaries and Discussion
832(2)
Problems
833(1)
23.A Proof of Lemma 23.3
834(4)
23.B Proof of Lemma 23.4
838(4)
23.C Proof of Theorem 23.1
842(3)
23.D Proof of Lemma 23.5
845(4)
23.E Proof of Theorem 23.2
849(3)
References
851(1)
24 Nonconvex Optimization
852(50)
24.1 First- and Second-Order Stationarity
852(8)
24.2 Stochastic Gradient Optimization
860(5)
24.3 Convergence Behavior
865(7)
24.4 Commentaries and Discussion
872(4)
Problems
874(2)
24.A Descent in the Large Gradient Regime
876(1)
24.B Introducing a Short-Term Model
877(11)
24.C Descent Away from Strict Saddle Points
888(9)
24.D Second-Order Convergence Guarantee
897(5)
References
900(2)
25 Decentralized Optimization I: Primal Methods
902(67)
25.1 Graph Topology
903(6)
25.2 Weight Matrices
909(4)
25.3 Aggregate and Local Risks
913(5)
25.4 Incremental, Consensus, and Diffusion
918(17)
25.5 Formal Derivation as Primal Methods
935(5)
25.6 Commentaries and Discussion
940(7)
Problems
943(4)
25.A Proof of Lemma 25.1
947(2)
25.B Proof of Property (25.71)
949(1)
25.C Convergence of Primal Algorithms
949(20)
References
965(4)
26 Decentralized Optimization II: Primal-Dual Methods
969(40)
26.1 Motivation
969(1)
26.2 EXTRA Algorithm
970(2)
26.3 EXACT Diffusion Algorithm
972(3)
26.4 Distributed Inexact Gradient Algorithm
975(3)
26.5 Augmented Decentralized Gradient Method
978(1)
26.6 ATC Tracking Method
979(4)
26.7 Unified Decentralized Algorithm
983(2)
26.8 Convergence Performance
985(2)
26.9 Dual Method
987(3)
26.10 Decentralized Nonconvex Optimization
990(5)
26.11 Commentaries and Discussion
995(5)
Problems
998(2)
26.A Convergence of Primal-Dual Algorithms
1000(9)
References
1006(3)
Author Index 1009(24)
Subject Index 1033
Ali H. Sayed is Professor and Dean of Engineering at École Polytechnique Fédérale de Lausanne (EPFL), Switzerland. He has also served as Distinguished Professor and Chairman of Electrical Engineering at the University of California, Los Angeles, USA, and as President of the IEEE Signal Processing Society. He is a member of the US National Academy of Engineering (NAE) and The World Academy of Sciences (TWAS), and a recipient of the 2022 IEEE Fourier Award and the 2020 IEEE Norbert Wiener Society Award. He is a Fellow of the IEEE.