Contributors |
|
xv | |
About the Editors |
|
xxi | |
Foreword |
|
xxv | |
|
Foreword |
|
xxix | |
|
Preface |
|
xxxi | |
|
Chapter 1 Introduction. Software Architecture for Cloud and Big Data: An Open Quest for the Architecturally Significant Requirements |
|
|
1 | (10) |
|
|
|
|
|
|
1.1 A Perspective into Software Architecture for Cloud and Big Data |
|
|
1 | (1) |
|
1.2 Cloud Architecturally Significant Requirements and Their Design Implications |
|
|
2 | (5) |
|
1.2.1 Dynamism and Elasticity as Cloud Architecturally Significant Requirements |
|
|
3 | (1) |
|
1.2.2 Multitenancy as Cloud Architecturally Significant Requirement |
|
|
4 | (1) |
|
1.2.3 Service Level Agreements (SLAs) Constraints as Cloud Architecturally Significant Requirement |
|
|
4 | (1) |
|
1.2.4 Cloud Marketplaces as Architecturally Significant Requirement |
|
|
5 | (1) |
|
1.2.5 Seeking Value as Cloud Architecturally Significant Requirement |
|
|
6 | (1) |
|
1.3 Big Data Management as Cloud Architecturally Significant Requirement |
|
|
7 | (4) |
|
1.3.1 Big Data Analytics Enabled by the Cloud and Its Architecturally Significant Requirements |
|
|
8 | (1) |
|
1.3.2 Architecturally Significant Requirements in Realm of Competing Big Data Technologies |
|
|
8 | (1) |
|
|
9 | (2) |
|
PART 1 CONCEPTS AND MODELS |
|
|
11 | (92) |
|
Chapter 2 Hyperscalability -- The Changing Face of Software Architecture |
|
|
13 | (20) |
|
|
|
13 | (1) |
|
2.2 Hyperscalable Systems |
|
|
14 | (6) |
|
|
14 | (1) |
|
|
15 | (2) |
|
|
17 | (2) |
|
|
19 | (1) |
|
2.3 Principles of Hyperscalable Systems |
|
|
20 | (9) |
|
2.3.1 Automate and Optimize to Control Costs |
|
|
21 | (2) |
|
2.3.2 Simple Solutions Promote Scalability |
|
|
23 | (2) |
|
2.3.3 Utilize Stateless Services |
|
|
25 | (1) |
|
2.3.4 Observability is Fundamental to Success at Hyperscale |
|
|
26 | (3) |
|
|
29 | (1) |
|
|
30 | (3) |
|
|
30 | (3) |
|
Chapter 3 Architecting to Deliver Value From a Big Data and Hybrid Cloud Architecture |
|
|
33 | (16) |
|
|
|
|
|
33 | (1) |
|
3.2 Supporting the Analytics Lifecycle |
|
|
33 | (2) |
|
3.3 The Role of Data Lakes |
|
|
35 | (1) |
|
3.4 Key Design Features That Make a Data Lake Successful |
|
|
36 | (1) |
|
3.5 Architecture Example -- Context Management in the IoT |
|
|
37 | (1) |
|
3.6 Big Data Origins and Characteristics |
|
|
38 | (1) |
|
3.7 The Systems That Capture and Process Big Data |
|
|
39 | (2) |
|
3.8 Operating Across Organizational Silos |
|
|
41 | (1) |
|
3.9 Architecture Example -- Local Processing of Big Data |
|
|
42 | (1) |
|
3.10 Architecture Example -- Creating a Multichannel View |
|
|
43 | (2) |
|
3.11 Application Independent Data |
|
|
45 | (1) |
|
3.12 Metadata and Governance |
|
|
45 | (1) |
|
|
46 | (1) |
|
3.14 Outlook and Future Directions |
|
|
47 | (2) |
|
|
48 | (1) |
|
Chapter 4 Domain-Driven Design of Big Data Systems Based on a Reference Architecture |
|
|
49 | (20) |
|
|
|
|
|
49 | (1) |
|
4.2 Domain-Driven Design Approach |
|
|
50 | (3) |
|
|
53 | (1) |
|
4.4 Feature Model of Big Data Systems |
|
|
54 | (6) |
|
|
56 | (1) |
|
4.4.2 Information Management |
|
|
56 | (1) |
|
4.4.3 Interface and Visualization |
|
|
56 | (1) |
|
|
56 | (1) |
|
|
57 | (1) |
|
|
57 | (1) |
|
4.4.7 Feature Constraints |
|
|
57 | (3) |
|
4.5 Deriving the Application Architectures and Example |
|
|
60 | (6) |
|
|
60 | (1) |
|
4.5.2 Design Rule Modeling |
|
|
60 | (1) |
|
4.5.3 Associating Design Decisions With Features |
|
|
61 | (1) |
|
4.5.4 Generation of the Application Architecture and the Deployment Diagram |
|
|
62 | (1) |
|
4.5.5 Deriving Big Data Architectures of Existing Systems |
|
|
63 | (3) |
|
|
66 | (3) |
|
|
67 | (2) |
|
Chapter 5 An Architectural Model-Based Approach to Quality-Aware DevOps in Cloud Applications |
|
|
69 | (22) |
|
|
|
|
|
|
|
69 | (1) |
|
5.2 A Cloud-Based Software Application |
|
|
70 | (1) |
|
5.3 Differences in Architectural Models Among Development and Operations |
|
|
71 | (1) |
|
5.4 The iObserve Approach |
|
|
72 | (3) |
|
5.5 Addressing the Differences in Architectural Models |
|
|
75 | (5) |
|
5.5.1 The iObserve Megamodel |
|
|
75 | (2) |
|
5.5.2 Descriptive and Prescriptive Architectural Models in iObserve |
|
|
77 | (1) |
|
5.5.3 Static and Dynamic Content in Architectural Models |
|
|
78 | (2) |
|
5.6 Applying iObserve to CoCoME |
|
|
80 | (4) |
|
5.6.1 Applying the iObserve Megamodel |
|
|
81 | (1) |
|
5.6.2 Applying Descriptive and Prescriptive Architectural Models |
|
|
82 | (1) |
|
5.6.3 Applying Live Visualization |
|
|
82 | (2) |
|
|
84 | (1) |
|
|
85 | (2) |
|
|
87 | (4) |
|
|
87 | (4) |
|
Chapter 6 Bridging Ecology and Cloud: Transposing Ecological Perspective to Enable Better Cloud Autoscaling |
|
|
91 | (12) |
|
|
|
|
91 | (1) |
|
|
92 | (1) |
|
|
93 | (1) |
|
6.4 Transposing Ecological Principles, Theories and Models to Cloud Ecosystem |
|
|
94 | (3) |
|
6.5 Ecology-Inspired Self-Aware Pattern |
|
|
97 | (2) |
|
6.6 Opportunities and Challenges |
|
|
99 | (1) |
|
|
100 | (1) |
|
|
100 | (3) |
|
|
101 | (1) |
|
|
102 | (1) |
|
PART 2 ANALYZING AND EVALUATING |
|
|
103 | (78) |
|
Chapter 7 Evaluating Web PKIs |
|
|
105 | (22) |
|
|
|
|
105 | (1) |
|
|
106 | (2) |
|
7.3 Desired Features and Security Concerns |
|
|
108 | (2) |
|
|
110 | (10) |
|
|
110 | (1) |
|
7.4.2 Difference Observation |
|
|
110 | (4) |
|
|
114 | (1) |
|
7.4.4 Certificate Management Transparency |
|
|
115 | (5) |
|
|
120 | (3) |
|
7.5.1 Property Perspective |
|
|
120 | (2) |
|
|
122 | (1) |
|
|
123 | (4) |
|
|
123 | (4) |
|
Chapter 8 Performance Isolation in Cloud-Based Big Data Architectures |
|
|
127 | (20) |
|
|
|
|
127 | (1) |
|
|
128 | (2) |
|
|
128 | (1) |
|
8.2.2 Big Data Architecture |
|
|
129 | (1) |
|
8.3 Case Study and Problem Statement |
|
|
130 | (2) |
|
|
130 | (2) |
|
|
132 | (1) |
|
8.4 Performance Monitoring in Cloud-Based Systems |
|
|
132 | (2) |
|
8.5 Application Framework for Performance Isolation |
|
|
134 | (3) |
|
8.6 Evaluation of the Framework |
|
|
137 | (6) |
|
|
140 | (3) |
|
|
143 | (1) |
|
|
143 | (1) |
|
|
144 | (3) |
|
|
145 | (2) |
|
Chapter 9 From Legacy to Cloud: Risks and Benefits in Software Cloud Migration |
|
|
147 | (20) |
|
|
|
|
147 | (1) |
|
|
147 | (7) |
|
|
148 | (1) |
|
|
148 | (3) |
|
|
151 | (1) |
|
9.2.4 Data Analysis Method |
|
|
152 | (2) |
|
|
154 | (8) |
|
9.3.1 Overview of Primary Studies and Quality Evaluation |
|
|
154 | (2) |
|
|
156 | (3) |
|
|
159 | (1) |
|
9.3.4 Models and Frameworks for Cloud Migration |
|
|
160 | (2) |
|
|
162 | (2) |
|
9.4.1 Findings and Lessons Learned |
|
|
162 | (1) |
|
9.4.2 Threats to Validity |
|
|
163 | (1) |
|
|
164 | (3) |
|
|
164 | (3) |
|
Chapter 10 Big Data: A Practitioners Perspective |
|
|
167 | (14) |
|
|
|
|
10.1 Big Data Is a New Paradigm -- Differences With Traditional Data Warehouse, Pitfalls and Consideration |
|
|
167 | (2) |
|
10.1.1 Differences With Traditional Data Warehouse |
|
|
167 | (1) |
|
|
168 | (1) |
|
|
169 | (1) |
|
10.2 Product Considerations for Big Data -- Use of Open Source Products for Big Data, Pitfalls and Considerations |
|
|
169 | (2) |
|
10.2.1 The Use of Open Source Product for Big Data |
|
|
169 | (1) |
|
|
170 | (1) |
|
|
171 | (1) |
|
10.3 Use of Cloud for hosting Big Data -- Why to Use Cloud, Pitfalls and Consideration |
|
|
171 | (1) |
|
|
171 | (1) |
|
|
172 | (1) |
|
|
172 | (1) |
|
10.4 Big Data Implementation -- Architecture Definition, Processing Framework and Migration Pattern From Data Warehouse to Big Data |
|
|
172 | (6) |
|
10.4.1 Patterns for Transitioning From Data Warehouse to Big Data |
|
|
175 | (3) |
|
|
178 | (3) |
|
|
179 | (2) |
|
|
181 | (140) |
|
Chapter 11 A Taxonomy and Survey of Stream Processing Systems |
|
|
183 | (24) |
|
|
|
|
|
|
183 | (2) |
|
11.2 Stream Processing Platforms: A Brief Background |
|
|
185 | (3) |
|
11.2.1 Requirements of Stream Processing Platforms/Engines |
|
|
186 | (1) |
|
11.2.2 Generic Model of Modern Stream Processing Platforms/Engines |
|
|
187 | (1) |
|
|
188 | (9) |
|
11.3.1 Functional Aspects |
|
|
189 | (6) |
|
11.3.2 Nonfunctional Aspects |
|
|
195 | (2) |
|
11.4 A Survey of Stream Processing Platforms |
|
|
197 | (3) |
|
11.4.1 Data Stream Management Systems |
|
|
198 | (1) |
|
11.4.2 Complex Event Processing Systems |
|
|
198 | (1) |
|
11.4.3 Stream Processing Platforms/Engines |
|
|
199 | (1) |
|
11.5 Comparison Study of the Stream Processing Platforms |
|
|
200 | (2) |
|
|
200 | (1) |
|
11.5.2 Messaging & Distribution |
|
|
201 | (1) |
|
11.5.3 Data Processing/Stream Processors |
|
|
201 | (1) |
|
|
202 | (1) |
|
11.6 Conclusions and Future Directions |
|
|
202 | (5) |
|
|
204 | (3) |
|
Chapter 12 Architecting Cloud Services for the Digital Me in a Privacy-Aware Environment |
|
|
207 | (20) |
|
|
|
|
|
|
|
207 | (1) |
|
|
208 | (2) |
|
|
210 | (2) |
|
12.3.1 Service Composition |
|
|
211 | (1) |
|
12.3.2 Technology Abstraction |
|
|
211 | (1) |
|
12.3.3 Service and Data Integration |
|
|
211 | (1) |
|
12.3.4 Trusted Use of Personal Data |
|
|
212 | (1) |
|
|
212 | (1) |
|
12.5 System-of-Systems Approach |
|
|
213 | (6) |
|
12.5.1 Persistence Service |
|
|
214 | (2) |
|
12.5.2 DataConversion Service |
|
|
216 | (1) |
|
|
217 | (1) |
|
|
217 | (1) |
|
12.5.5 PersonalData Service |
|
|
218 | (1) |
|
|
219 | (2) |
|
|
221 | (2) |
|
12.7.1 Service Composition |
|
|
222 | (1) |
|
12.7.2 Technology Abstraction |
|
|
222 | (1) |
|
12.7.3 Service and Data Integration |
|
|
222 | (1) |
|
12.7.4 Trusted Use of Personal Data |
|
|
222 | (1) |
|
|
223 | (1) |
|
|
223 | (4) |
|
|
224 | (3) |
|
Chapter 13 Reengineering Data-Centric Information Systems for the Cloud -- A Method and Architectural Patterns Promoting Multitenancy |
|
|
227 | (26) |
|
|
|
|
|
|
227 | (1) |
|
13.2 Context and Problem: Multitenancy in Cloud Computing |
|
|
228 | (2) |
|
13.3 Solution Overview: Reengineering Method and Process |
|
|
230 | (2) |
|
13.4 Solution Detail 1: Architectural Patterns in the Method |
|
|
232 | (5) |
|
13.4.1 Architectural Reengineering Steps for the Cloud (Architectural Refactoring) |
|
|
233 | (1) |
|
13.4.2 Multitenancy Requirements and Patterns for Cloud Environments |
|
|
234 | (1) |
|
13.4.3 The Multitenancy Capable Model |
|
|
235 | (1) |
|
13.4.4 The Multitenancy Capable Controller |
|
|
235 | (1) |
|
13.4.5 The Multitenancy Capable View |
|
|
236 | (1) |
|
13.5 Solution Detail 2: Testing and Code Reviews |
|
|
237 | (3) |
|
13.5.1 Testing for Multitenancy Defects |
|
|
237 | (2) |
|
13.5.2 Code Review for Multitenancy Defects |
|
|
239 | (1) |
|
|
240 | (1) |
|
13.6 Case Study (Implementation) |
|
|
240 | (6) |
|
13.6.1 Multitenancy Transformation Without Patterns |
|
|
241 | (2) |
|
13.6.2 Multitenancy Transformation With Patterns |
|
|
243 | (1) |
|
|
243 | (3) |
|
|
246 | (1) |
|
|
247 | (1) |
|
13.9 Summary and Conclusions |
|
|
247 | (6) |
|
Appendix 13.A Architectural Refactoring (AR) Reference |
|
|
248 | (1) |
|
|
249 | (4) |
|
Chapter 14 Exploring the Evolution of Big Data Technologies |
|
|
253 | (32) |
|
|
|
|
|
|
253 | (1) |
|
14.2 Big Data in Our Daily Lives |
|
|
254 | (2) |
|
14.3 Data Intensive Computing |
|
|
256 | (7) |
|
14.3.1 Big Compute Versus Big Data |
|
|
256 | (2) |
|
14.3.2 Data Intensive Applications |
|
|
258 | (1) |
|
14.3.3 Data Intensive Frameworks |
|
|
258 | (1) |
|
|
259 | (4) |
|
|
263 | (4) |
|
|
263 | (3) |
|
|
266 | (1) |
|
|
267 | (4) |
|
14.5.1 Resilient Distributed Datasets |
|
|
268 | (1) |
|
14.5.2 Data Flow and Programming With Spark |
|
|
269 | (1) |
|
14.5.3 Spark Processing Engines |
|
|
270 | (1) |
|
14.5.4 Hadoop Ecosystem Taxonomy |
|
|
271 | (1) |
|
14.6 The Role of Cloud Computing |
|
|
271 | (3) |
|
14.7 The Future of Big Data Platforms |
|
|
274 | (6) |
|
14.7.1 Big Data Applications |
|
|
274 | (1) |
|
14.7.2 Big Data Frameworks and Hardware |
|
|
275 | (4) |
|
14.7.3 Big Data on the Road to Exascale |
|
|
279 | (1) |
|
|
280 | (5) |
|
|
281 | (4) |
|
Chapter 15 A Taxonomy and Survey of Fault-Tolerant Workflow Management Systems in Cloud and Distributed Computing Environments |
|
|
285 | (36) |
|
|
|
|
|
|
285 | (1) |
|
|
286 | (2) |
|
15.2.1 Workflow Management Systems |
|
|
286 | (1) |
|
15.2.2 Workflow Scheduling |
|
|
287 | (1) |
|
15.3 Introduction to Fault-Tolerance |
|
|
288 | (2) |
|
15.3.1 Necessity for Fault-Tolerance in Distributed Systems |
|
|
290 | (1) |
|
|
290 | (1) |
|
15.5 Taxonomy of Fault-Tolerant Scheduling Algorithms |
|
|
291 | (12) |
|
|
292 | (3) |
|
|
295 | (2) |
|
|
297 | (3) |
|
|
300 | (1) |
|
|
301 | (1) |
|
15.5.6 User-Defined Exception Handling |
|
|
301 | (1) |
|
|
301 | (1) |
|
|
301 | (1) |
|
|
302 | (1) |
|
15.5.10 Trust-Based Scheduling Algorithms |
|
|
302 | (1) |
|
15.6 Modeling of Failures in Workflow Management Systems |
|
|
303 | (1) |
|
15.7 Metrics Used to Quantify Fault-Tolerance |
|
|
304 | (1) |
|
15.8 Survey of Workflow Management Systems and Frameworks |
|
|
305 | (8) |
|
|
305 | (3) |
|
|
308 | (1) |
|
|
309 | (1) |
|
|
310 | (1) |
|
|
310 | (1) |
|
15.8.6 Cloudbus Workflow Management System |
|
|
310 | (1) |
|
|
311 | (1) |
|
15.8.8 The e-Science Central (e-SC) |
|
|
311 | (1) |
|
|
311 | (1) |
|
15.8.10 Big Data Workflow Frameworks: MapReduce, Hadoop, and Spark |
|
|
312 | (1) |
|
15.8.11 Other Workflow Management Systems |
|
|
313 | (1) |
|
15.9 Tools and Support Systems |
|
|
313 | (2) |
|
15.9.1 Data Management Tools |
|
|
313 | (1) |
|
15.9.2 Security and Fault-Tolerance Management Tools |
|
|
314 | (1) |
|
15.9.3 Cloud Development Tools |
|
|
314 | (1) |
|
|
314 | (1) |
|
|
315 | (6) |
|
|
315 | (6) |
|
PART 4 RESOURCE MANAGEMENT |
|
|
321 | (68) |
|
Chapter 16 The HARNESS Platform: A Hardware- and Network-Enhanced Software System for Cloud Computing |
|
|
323 | (30) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
323 | (1) |
|
|
324 | (1) |
|
|
325 | (2) |
|
16.4 Managing Heterogeneity |
|
|
327 | (9) |
|
16.4.1 Hierarchical Resource Management |
|
|
328 | (1) |
|
16.4.2 Agnostic Resource Management |
|
|
329 | (2) |
|
16.4.3 Ranking Allocation Requests |
|
|
331 | (3) |
|
|
334 | (2) |
|
16.5 Prototype Description |
|
|
336 | (4) |
|
16.5.1 The Platform Layer |
|
|
336 | (2) |
|
16.5.2 The Infrastructure Layer |
|
|
338 | (1) |
|
16.5.3 The Virtual Execution Layer |
|
|
339 | (1) |
|
|
340 | (9) |
|
16.6.1 Executing HPC Applications on the Cloud |
|
|
340 | (5) |
|
16.6.2 Resource Scheduling with Network Constraints |
|
|
345 | (4) |
|
|
349 | (4) |
|
|
350 | (1) |
|
|
350 | (1) |
|
|
351 | (2) |
|
Chapter 17 Auditable Version Control Systems in Untrusted Public Clouds |
|
|
353 | (14) |
|
|
|
|
17.1 Motivation and Contributions |
|
|
353 | (2) |
|
17.2 Background Knowledge |
|
|
355 | (2) |
|
17.2.1 Data Organization in Version Control Systems |
|
|
355 | (2) |
|
17.2.2 Remote Data Integrity Checking (RDIC) |
|
|
357 | (1) |
|
17.3 System and Adversarial Model |
|
|
357 | (1) |
|
17.4 Auditable Version Control Systems |
|
|
358 | (4) |
|
17.4.1 Definition of AVCS |
|
|
358 | (1) |
|
17.4.2 An AVCS Construction |
|
|
359 | (3) |
|
|
362 | (1) |
|
17.6 Other RDIC Approaches for Version Control Systems |
|
|
363 | (1) |
|
|
363 | (2) |
|
17.7.1 Theoretical Evaluation |
|
|
363 | (1) |
|
17.7.2 Experimental Evaluation |
|
|
364 | (1) |
|
|
365 | (2) |
|
|
365 | (2) |
|
Chapter 18 Scientific Workflow Management System for Clouds |
|
|
367 | (22) |
|
|
|
|
367 | (1) |
|
|
368 | (2) |
|
18.3 Workflow Management Systems for Clouds |
|
|
370 | (2) |
|
18.4 Cloudbus Workflow Management System |
|
|
372 | (2) |
|
18.5 Cloud-Based Extensions to the Workflow Engine |
|
|
374 | (5) |
|
18.6 Performance Evaluation |
|
|
379 | (6) |
|
|
379 | (1) |
|
|
379 | (1) |
|
18.6.3 Setup of Experimental Infrastructure |
|
|
380 | (1) |
|
|
381 | (1) |
|
|
382 | (3) |
|
18.7 Summary and Conclusions |
|
|
385 | (4) |
|
|
386 | (3) |
|
|
389 | (14) |
|
Chapter 19 Outlook and Future Directions |
|
|
391 | (12) |
|
|
|
|
|
|
19.1 New or Advanced Applications |
|
|
391 | (3) |
|
19.2 Advanced Supporting Technologies |
|
|
394 | (1) |
|
19.3 Architecturally Significant Requirements |
|
|
395 | (2) |
|
19.4 Challenges for the Architecting Process |
|
|
397 | (4) |
|
|
401 | (2) |
|
|
402 | (1) |
Glossary |
|
403 | (4) |
Author Index |
|
407 | (18) |
Subject Index |
|
425 | |