Atjaunināt sīkdatņu piekrišanu

Practice of Crowdsourcing [Mīkstie vāki]

Series edited by ,
Citas grāmatas par šo tēmu:
  • Mīkstie vāki
  • Cena: 102,83 €
  • Grāmatu piegādes laiks ir 3-4 nedēļas, ja grāmata ir uz vietas izdevniecības noliktavā. Ja izdevējam nepieciešams publicēt jaunu tirāžu, grāmatas piegāde var aizkavēties.
  • Daudzums:
  • Ielikt grozā
  • Piegādes laiks - 4-6 nedēļas
  • Pievienot vēlmju sarakstam
Practice of CrowdsourcingPalielināt
Citas grāmatas par šo tēmu:
Permanent link: https://www.kriso.lv/db/9781681735238.html
Keywords:

Many data-intensive applications that use machine learning or artificial intelligence techniques depend on humans providing the initial dataset, enabling algorithms to process the rest or for other humans to evaluate the performance of such algorithms.

Not only can labeled data for training and evaluation be collected faster, cheaper, and easier than ever before, but we now see the emergence of hybrid human-machine software that combines computations performed by humans and machines in conjunction. There are, however, real-world practical issues with the adoption of human computation and crowdsourcing. Building systems and data processing pipelines that require crowd computing remains difficult. In this book, we present practical considerations for designing and implementing tasks that require the use of humans and machines in combination with the goal of producing high-quality labels.

Preface viii
Acknowledgments xix
1 Introduction
1(14)
1.1 Human Computers
1(1)
1.2 Basic Concepts
2(3)
1.3 Examples
5(5)
1.3.1 Query Classification
5(3)
1.3.2 Flip a Coin
8(2)
1.4 Some Generic Observations
10(1)
1.5 A Note on Platforms
11(1)
1.6 The Importance of Labels
12(1)
1.7 Scope and Structure
13(2)
2 Designing and Developing Microtasks
15(22)
2.1 Microtask Development Flow
15(1)
2.2 Programming HITs
16(2)
2.3 Asking Questions
18(1)
2.4 Collecting Responses
19(2)
2.5 Interface Design
21(3)
2.6 Cognitive Biases and Effects
24(1)
2.7 Content Aspects
25(1)
2.7.1 Presentation
25(1)
2.7.2 Data Familiarity
25(1)
2.7.3 Metadata and Internationalization
26(1)
2.8 Task Clarity
26(1)
2.9 Task Complexity
27(1)
2.10 Sensitive Data
28(1)
2.11 Examples
29(6)
2.11.1 Binary Relevance Assessment
29(1)
2.11.2 Graded Relevance Assessment
30(1)
2.11.3 Web Page Relevance Assessment
31(1)
2.11.4 Ranked Comparison
32(1)
2.11.5 Survey Style
33(1)
2.11.6 User Study
34(1)
2.12 Summary
35(2)
3 Quality Assurance
37(16)
3.1 Quality Framework
37(1)
3.2 Quality Control Overview
38(2)
3.3 Recommendations from Platforms
40(1)
3.4 Worker Qualification
40(2)
3.5 Reliability and Validity
42(4)
3.5.1 Inter-rater Reliability
43(1)
3.5.2 Internal Consistency
44(1)
3.5.3 Discussion
45(1)
3.6 HIT Debugging
46(4)
3.7 Summary
50(3)
4 Algorithms and Techniques for Quality Control
53(12)
4.1 Framework
53(1)
4.2 Voting
54(1)
4.3 Attention Monitoring
55(1)
4.4 Honey Pots
55(2)
4.5 Workers Reviewing Work
57(1)
4.6 Justification
58(1)
4.7 Aggregation Methods
59(1)
4.8 Behavioral Data
60(1)
4.9 Expertise and Routing
61(1)
4.10 Summary
61(4)
5 The Human Side of Human Computation
65(8)
5.1 Overview
66(1)
5.2 Demographics
66(1)
5.3 Incentives
67(1)
5.4 Worker Experience
68(2)
5.5 Worker Feedback
70(1)
5.5.1 Operational
70(1)
5.5.2 General communication
70(1)
5.5.3 HIT Assessment
71(1)
5.6 Legal and Ethics
71(1)
5.7 Summary
72(1)
6 Putting All Things Together
73(16)
6.1 The State of the Practice
73(1)
6.2 Wetware Programming
74(4)
6.2.1 What to Measure
74(1)
6.2.2 Program Structure and Design Patterns
74(1)
6.2.3 Development Process
75(3)
6.3 Testing and Debugging
78(1)
6.4 Work Quality Control
79(4)
6.4.1 Instrumentation
80(1)
6.4.2 Algorithms
80(1)
6.4.3 Behavioral Data
81(1)
6.4.4 Incentives
81(1)
6.4.5 Discussion
82(1)
6.5 Managing Construction
83(1)
6.6 Operational Considerations
84(1)
6.7 Summary of Practices
85(1)
6.8 Summary
86(3)
7 Systems and Data Pipelines
89(10)
7.1 Evaluation
89(2)
7.2 Machine Translation
91(1)
7.3 Handwritting Recognition and Transcription
91(1)
7.4 Taxonomy Creation
92(1)
7.5 Data Analysis
92(1)
7.6 News Near-Duplicate Detection
93(1)
7.7 Entity Resolution
94(1)
7.8 Classification
95(1)
7.9 Image and Speech
95(1)
7.10 Information Extraction
95(1)
7.11 RABJ
96(1)
7.12 Workflows
96(1)
7.13 Summary
97(2)
8 Looking Ahead
99(6)
8.1 Crowds and Social Networks
99(1)
8.2 Interactive and Real-Time Crowdsourcing
100(1)
8.3 Programming Languages
101(1)
8.4 Databases and Crowd-powered Algorithms
102(1)
8.5 Fairness, Bias, and Reproducibility
102(1)
8.6 An Incomplete List of Requirements for Infrastructure
102(2)
8.7 Summary
104(1)
Bibliography 105(24)
Author's Biography 129