List of Contributors |
|
xv | |
About the Editors |
|
xvii | |
Foreword |
|
xix | |
|
Preface |
|
xxi | |
|
Acknowledgments |
|
xxiii | |
Section 1 Introduction To Big Data |
|
|
Chapter 1 An Introduction to Big Data |
|
|
3 | (11) |
|
|
3 | (1) |
|
How Different Is Big Data? |
|
|
4 | (1) |
|
More on Big Data: Types and Sources |
|
|
4 | (1) |
|
|
5 | (1) |
|
|
5 | (1) |
|
|
5 | (1) |
|
|
5 | (2) |
|
|
6 | (1) |
|
|
6 | (1) |
|
|
6 | (1) |
|
|
7 | (1) |
|
|
7 | (1) |
|
Big Data in the Big World |
|
|
7 | (2) |
|
|
7 | (1) |
|
Advantages and Applications |
|
|
7 | (2) |
|
Analytical Capabilities of Big Data |
|
|
9 | (1) |
|
|
9 | (1) |
|
Greater Risk Intelligence |
|
|
9 | (1) |
|
Satisfying Business Needs |
|
|
9 | (1) |
|
Predictive Modeling and Optimization |
|
|
10 | (1) |
|
|
10 | (1) |
|
Identifying Business Use Cases |
|
|
10 | (1) |
|
Video and Voice Analytics |
|
|
10 | (1) |
|
|
10 | (1) |
|
An Overview of Big Data Solutions |
|
|
11 | (1) |
|
|
11 | (1) |
|
IBM InfoSphere BigInsights |
|
|
11 | (1) |
|
Big Data on Amazon Web Services |
|
|
11 | (1) |
|
|
11 | (1) |
|
|
12 | (1) |
|
|
12 | (2) |
|
Chapter 2 Drilling into the Big Data Gold Mine: Data Fusion and High-Performance Analytics for Intelligence Professionals |
|
|
14 | (9) |
|
|
14 | (1) |
|
The Age of Big Data and High-Performance Analytics |
|
|
14 | (1) |
|
|
15 | (4) |
|
Building the Complete Intelligence Picture |
|
|
16 | (3) |
|
|
19 | (1) |
|
Scenario 1: Fusion and Michigan State Police |
|
|
19 | (1) |
|
Scenario 2: National Security and Intelligence Solution in the Middle East |
|
|
19 | (1) |
|
|
20 | (1) |
|
|
20 | (3) |
Section 2 Core Concepts And Application Scenarios |
|
|
Chapter 3 Harnessing the Power of Big Data to Counter International Terrorism |
|
|
23 | (16) |
|
|
23 | (1) |
|
|
24 | (9) |
|
|
24 | (1) |
|
|
25 | (1) |
|
|
26 | (1) |
|
|
27 | (1) |
|
|
27 | (2) |
|
|
29 | (1) |
|
|
30 | (1) |
|
|
30 | (2) |
|
|
32 | (1) |
|
Changing Threat Landscape |
|
|
33 | (1) |
|
|
34 | (2) |
|
|
36 | (1) |
|
|
37 | (2) |
|
Chapter 4 Big Data and Law Enforcement: Advances, Implications, and Lessons from an Active Shooter Case Study |
|
|
39 | (16) |
|
The Intersection of Big Data and Law Enforcement |
|
|
39 | (2) |
|
Case Example and Workshop Overview |
|
|
41 | (2) |
|
|
43 | (2) |
|
|
43 | (1) |
|
Interacting with the Public |
|
|
44 | (1) |
|
|
44 | (1) |
|
Twitter as a Social Media Source of Big Data |
|
|
45 | (1) |
|
Social Media Data Analyzed for the Workshop |
|
|
45 | (1) |
|
Tools and Capabilities Prototypes During the Workshop |
|
|
46 | (5) |
|
|
46 | (1) |
|
Dynamic Classification of Tweet Content |
|
|
46 | (1) |
|
Content-Based Image Retrieval |
|
|
47 | (1) |
|
Maximizing Geographic Information |
|
|
48 | (1) |
|
|
49 | (1) |
|
Influence and Reach of Messaging |
|
|
49 | (1) |
|
|
50 | (1) |
|
Law Enforcement Feedback for the Sessions |
|
|
51 | (1) |
|
|
51 | (1) |
|
|
52 | (1) |
|
|
52 | (3) |
|
Chapter 5 Interpretation and Insider Threat: Rereading the Anthrax Mailings of 2001 Through a "Big Data" Lens |
|
|
55 | (13) |
|
|
55 | (2) |
|
|
57 | (1) |
|
The Advancement of Big Data Analytics After 2001 |
|
|
58 | (1) |
|
|
59 | (2) |
|
Potential for Stylometric and Sentiment Analysis |
|
|
61 | (2) |
|
Potential for Further Pattern Analysis and Visualization |
|
|
63 | (1) |
|
Final Words: Interpretation and Insider Threat |
|
|
64 | (1) |
|
|
65 | (3) |
|
Chapter 6 Critical Infrastructure Protection by Harnessing Big Data |
|
|
68 | (13) |
|
|
68 | (1) |
|
|
68 | (1) |
|
Understanding the Strategic Landscape into Which Big Data Must Be Applied |
|
|
69 | (4) |
|
What Is Meant by an Overarching Architecture? |
|
|
73 | (3) |
|
|
73 | (3) |
|
|
76 | (1) |
|
Strategic Community Architecture Framework |
|
|
77 | (3) |
|
|
80 | (1) |
|
|
80 | (1) |
|
Chapter 7 Military and Big Data Revolution |
|
|
81 | (27) |
|
|
81 | (1) |
|
|
82 | (1) |
|
Simple to Complex Use Cases |
|
|
83 | (4) |
|
|
87 | (2) |
|
|
88 | (1) |
|
Correlation of Data Over Space and Time |
|
|
88 | (1) |
|
More on the Digital Version of the Real World (See the World as Events) |
|
|
89 | (2) |
|
Quality of Data, Metadata, and Content |
|
|
90 | (1) |
|
Real-Time Big Data Systems |
|
|
91 | (4) |
|
Application Principles and Constraints |
|
|
91 | (2) |
|
|
93 | (2) |
|
Implementing the Real-Time Big Data System |
|
|
95 | (7) |
|
Batch Processing (into the "Batch Layer") |
|
|
95 | (1) |
|
Processing Layer (into the "Batch Layer") |
|
|
95 | (1) |
|
|
96 | (1) |
|
Data Stream Processing (into the "Streaming Layer") |
|
|
97 | (1) |
|
Alerts and Notifications (into the "Publishing Layer") |
|
|
98 | (1) |
|
Filtering Processing Fitting in Memory (into the "Streaming Layer") |
|
|
98 | (1) |
|
Machine Learning and Filtering (into the "Batch and Streaming Layers") |
|
|
99 | (1) |
|
Online Clustering (into the "Streaming Layer") |
|
|
100 | (1) |
|
|
100 | (1) |
|
|
101 | (1) |
|
Insight into Deep Data Analytics Tools and Real-Time Big Data Systems |
|
|
102 | (2) |
|
|
103 | (1) |
|
|
103 | (1) |
|
Adding Flexibility and Adaptation |
|
|
104 | (1) |
|
Very Short Loop and Battlefield Big Data Datacenters |
|
|
104 | (1) |
|
|
104 | (2) |
|
|
106 | (2) |
|
Chapter 8 Cybercrime: Attack Motivations and Implications for Big Data and National Security |
|
|
108 | (23) |
|
|
108 | (2) |
|
Defining Cybercrime and Cyberterrorism |
|
|
110 | (1) |
|
Attack Classification and Parameters |
|
|
111 | (2) |
|
Who Perpetrates These Attacks? |
|
|
113 | (2) |
|
|
113 | (1) |
|
|
114 | (1) |
|
|
114 | (1) |
|
|
114 | (1) |
|
|
115 | (1) |
|
Tools Used to Facilitate Attacks |
|
|
115 | (2) |
|
|
117 | (1) |
|
Attack Motivations Taxonomy |
|
|
118 | (4) |
|
|
118 | (2) |
|
|
120 | (1) |
|
|
120 | (1) |
|
|
120 | (1) |
|
Informational/Promotional |
|
|
121 | (1) |
|
|
121 | (1) |
|
|
121 | (1) |
|
|
122 | (1) |
|
Detecting Motivations in Open-Source Information |
|
|
122 | (1) |
|
|
123 | (1) |
|
|
123 | (8) |
Section 3 Methods And Technological Solutions |
|
|
Chapter 9 Requirements and Challenges for Big Data Architectures |
|
|
131 | (9) |
|
What Are the Challenges Involved in Big Data Processing? |
|
|
131 | (1) |
|
|
131 | (1) |
|
Technological Underpinning |
|
|
132 | (2) |
|
|
132 | (2) |
|
Planning for a Big Data Platform |
|
|
134 | (5) |
|
Infrastructure Requirements |
|
|
134 | (3) |
|
Capacity Planning Considerations |
|
|
137 | (1) |
|
Cloud Computing Considerations |
|
|
137 | (2) |
|
|
139 | (1) |
|
|
139 | (1) |
|
Chapter 10 Tools and Technologies for the Implementation of Big Data |
|
|
140 | (15) |
|
|
140 | (1) |
|
|
141 | (1) |
|
Representation, Storage, and Data Management |
|
|
141 | (1) |
|
|
142 | (2) |
|
|
142 | (1) |
|
Association Rule Learning |
|
|
143 | (1) |
|
|
143 | (1) |
|
|
143 | (1) |
|
|
143 | (1) |
|
Natural Language Processing and Text Analysis |
|
|
143 | (1) |
|
|
144 | (1) |
|
|
144 | (1) |
|
|
144 | (1) |
|
|
144 | (1) |
|
|
145 | (1) |
|
|
145 | (1) |
|
|
145 | (1) |
|
|
145 | (1) |
|
|
146 | (1) |
|
Project Initiation and Launch |
|
|
146 | (4) |
|
Information Technology Project Reference Class |
|
|
148 | (1) |
|
|
149 | (1) |
|
User Factors and Change Management |
|
|
149 | (1) |
|
Data Sources and Analytics |
|
|
150 | (1) |
|
|
150 | (1) |
|
|
150 | (1) |
|
Analytics Philosophy: Analysis or Synthesis |
|
|
151 | (1) |
|
Governance and Compliance |
|
|
152 | (1) |
|
Data Protection Requirements and Privacy |
|
|
152 | (1) |
|
|
153 | (2) |
|
Chapter 11 Mining Social Media: Architecture, Tools, and Approaches to Detecting Criminal Activity |
|
|
155 | (18) |
|
|
155 | (2) |
|
Mining of Social Networks for Crime |
|
|
157 | (1) |
|
|
158 | (1) |
|
|
158 | (1) |
|
|
158 | (1) |
|
|
159 | (1) |
|
|
159 | (1) |
|
General Architecture and Various Components of Text Mining |
|
|
159 | (6) |
|
|
159 | (1) |
|
|
160 | (1) |
|
|
160 | (1) |
|
|
161 | (1) |
|
|
161 | (1) |
|
|
161 | (1) |
|
|
162 | (1) |
|
|
163 | (1) |
|
|
163 | (1) |
|
|
163 | (1) |
|
|
163 | (1) |
|
|
163 | (2) |
|
Automatic Extraction of BNs from Text |
|
|
165 | (1) |
|
Dependence Relation Extraction from Text |
|
|
165 | (1) |
|
|
166 | (1) |
|
|
166 | (3) |
|
Probability Information Extraction |
|
|
166 | (1) |
|
Aggregation of Structural and Probabilistic Data |
|
|
166 | (1) |
|
|
167 | (1) |
|
|
167 | (2) |
|
Example of BN Application to Crime Detection: Covert Networks |
|
|
169 | (1) |
|
|
169 | (1) |
|
|
170 | (3) |
|
Chapter 12 Making Sense of Unstructured Natural Language Information |
|
|
173 | (11) |
|
|
173 | (1) |
|
Big Data and Unstructured Data |
|
|
174 | (1) |
|
Aspects of Uncertainty in Sense Making |
|
|
175 | (1) |
|
Situation Awareness and Intelligence |
|
|
176 | (1) |
|
Situation Awareness: Short Timelines, Small Footprint |
|
|
176 | (1) |
|
Intelligence: Long(er) Timelines, Larger Footprint |
|
|
176 | (1) |
|
Processing Natural Language Data |
|
|
177 | (1) |
|
Structuring Natural Language Data |
|
|
178 | (1) |
|
Two Significant Weaknesses |
|
|
179 | (1) |
|
Ignoring Lexical Clues on Credibility and Reliability |
|
|
179 | (1) |
|
Out of Context, Out of Mind |
|
|
180 | (1) |
|
An Alternative Representation for Flexibility |
|
|
180 | (2) |
|
|
182 | (1) |
|
|
182 | (2) |
|
Chapter 13 Literature Mining and Ontology Mapping Applied to Big Data |
|
|
184 | (25) |
|
|
184 | (1) |
|
|
185 | (2) |
|
Parameter Optimized Latent Semantic Analysis |
|
|
186 | (1) |
|
Improving the Semantic Meaning of the POLSA Framework |
|
|
186 | (1) |
|
|
186 | (1) |
|
ARIANA: Adaptive Robust Integrative Analysis for Finding Novel Associations |
|
|
187 | (1) |
|
Conceptual Framework of ARIANA |
|
|
187 | (6) |
|
|
189 | (1) |
|
Data Stratification and POLSA |
|
|
189 | (1) |
|
|
190 | (2) |
|
|
192 | (1) |
|
Visualization and Interface |
|
|
192 | (1) |
|
Implementation of ARIANA for Biomedical Applications |
|
|
193 | (8) |
|
|
194 | (1) |
|
|
195 | (1) |
|
Data Stratification and POLSA |
|
|
195 | (3) |
|
Parameter Optimized Latent Semantic Analysis |
|
|
198 | (1) |
|
|
198 | (2) |
|
Reverse Ontology Mapping and I&V |
|
|
200 | (1) |
|
|
201 | (1) |
|
Case Study I: KD: Lethal Drug Interaction |
|
|
201 | (1) |
|
Case Study II: Data Repurposing: AD Study |
|
|
202 | (1) |
|
|
202 | (2) |
|
|
204 | (1) |
|
|
205 | (1) |
|
|
205 | (4) |
|
Chapter 14 Big Data Concerns in Autonomous Al Systems |
|
|
209 | (20) |
|
|
209 | (1) |
|
Artificially Intelligent System Memory Management |
|
|
210 | (2) |
|
|
210 | (1) |
|
Short-term Artificial Memories |
|
|
211 | (1) |
|
Long-term Artificial Memories |
|
|
211 | (1) |
|
Artificial Memory Processing and Encoding |
|
|
212 | (6) |
|
Short-term Artificial Memory Processing |
|
|
212 | (4) |
|
Long-term Artificial Memory Processing |
|
|
216 | (1) |
|
Implicit Biographical Memory Recall/Reconstruction Using Spectral Decomposition Mapping |
|
|
217 | (1) |
|
|
218 | (3) |
|
Adaptation of Constructivist Learning Concepts for Big Data in an AIS |
|
|
220 | (1) |
|
Practical Solutions for Secure Knowledge Development in Big Data Environments |
|
|
221 | (3) |
|
Practical Big Data Security Solutions |
|
|
221 | (2) |
|
Optimization of Sociopolitical-Economic Systems and Sentiment Analysis |
|
|
223 | (1) |
|
|
224 | (1) |
|
|
225 | (4) |
Section 4 Legal And Social Challenges |
|
|
Chapter 15 The Legal Challenges of Big Data Application in Law Enforcement |
|
|
229 | (9) |
|
|
229 | (1) |
|
|
229 | (1) |
|
|
230 | (1) |
|
|
230 | (6) |
|
|
231 | (2) |
|
Purpose Limitation and Further Processing |
|
|
233 | (1) |
|
Public Trust and Confidence |
|
|
234 | (2) |
|
|
236 | (1) |
|
How Far Should Big Data Principles Such as "Do Not Track" and "Do Not Collect" Be Applicable to LEAs, Either in Qualified Format or at All? |
|
|
236 | (1) |
|
|
237 | (1) |
|
Chapter 16 Big Data and the Italian Legal Framework: Opportunities for Police Forces |
|
|
238 | (12) |
|
|
238 | (1) |
|
|
239 | (3) |
|
Directive 95/46/EC and Revision Process Started in 2012 |
|
|
239 | (2) |
|
|
241 | (1) |
|
The Italian Legal Framework |
|
|
242 | (3) |
|
Authority for Personal Data Protection |
|
|
242 | (1) |
|
|
242 | (1) |
|
Focus on Italian Police Forces |
|
|
243 | (1) |
|
Police Data Processing and Privacy |
|
|
244 | (1) |
|
Opportunities and Constraints for Police Forces and Intelligence |
|
|
245 | (3) |
|
|
248 | (2) |
|
Chapter 17 Accounting for Cultural Influences in Big Data Analytics |
|
|
250 | (11) |
|
|
250 | (1) |
|
Considerations from Cross-Cultural Psychology for Big Data Analytics |
|
|
251 | (1) |
|
Cultural Dependence in the Supply and Demand Sides of Big Data Analytics |
|
|
252 | (4) |
|
Cultural Dependence on the Supply Side (Data Creation) |
|
|
253 | (1) |
|
Cultural Dependence on the Demand Side (Data Interpretation) |
|
|
254 | (2) |
|
(Mis)Matches among Producer, Production, Interpreter, and Interpretation Contexts |
|
|
256 | (1) |
|
Integrating Cultural Intelligence into Big Data Analytics: Some Recommendations |
|
|
257 | (1) |
|
|
258 | (1) |
|
|
259 | (2) |
|
Chapter 18 Making Sense of the Noise: An ABC Approach to Big Data and Security |
|
|
261 | (14) |
|
How Humans Naturally Deal with Big Data |
|
|
261 | (1) |
|
The Three Stages of Data Processing Explained |
|
|
262 | (3) |
|
|
263 | (1) |
|
|
263 | (1) |
|
|
264 | (1) |
|
The Public Order Policing Model and the Common Operational Picture |
|
|
265 | (2) |
|
Applications to Big Data and Security |
|
|
267 | (3) |
|
Level 1: Reflexive Response |
|
|
268 | (1) |
|
Level 2: Pre-attentive Response |
|
|
268 | (1) |
|
Level 3: Attentive Response and the Focused, Intellectual Management of Data |
|
|
269 | (1) |
|
Application to Big Data and National Security |
|
|
270 | (2) |
|
A Final Caveat from the FBI Bulletin |
|
|
272 | (1) |
|
|
272 | (3) |
Glossary |
|
275 | (4) |
Index |
|
279 | |