Musical robotics is a multi- and trans-disciplinary research area involving a wide range of different domains that contribute to its development, including: computer science, multimodal interfaces and processing, artificial intelligence, electronics, robotics, mechatronics and more. A musical robot requires many different complex systems to work together; integrating musical representation, techniques, expressions, detailed analysis and controls, for both playing and listening. The development of interactive multimodal systems provides advancements which enable enhanced human-machine interaction and novel possibilities for embodied robotic platforms. This volume is focused on this highly exciting interdisciplinary field. This book consists of 14 chapters highlighting different aspects of musical activities and interactions, discussing cutting edge research related to interactive multimodal systems and their integration with robots to further enhance musical understanding, interpretation, performance, education and enjoyment. It is dichotomized into two sections: Section I focuses on understanding elements of musical performance and expression while Section II concentrates on musical robots and automated instruments. Musical Robots and Interactive Multimodal Systems provides an introduction and foundation for researchers, students and practitioners to key achievements and current research trends on interactive multimodal systems and musical robotics.
|
Chapter 1 Musical Robots and Interactive Multimodal Systems: An Introduction |
|
|
1 | (12) |
|
|
|
|
|
Section I Understanding Elements of Musical Performance and Expression |
|
|
|
Chapter 2 Sound-Action Chunks in Music |
|
|
13 | (14) |
|
|
|
Chapter 3 Automatic Music Transcription: From Monophonic to Polyphonic |
|
|
27 | (20) |
|
|
|
|
|
|
|
Chapter 4 Multimodal Analysis of Expressive Gesture in Music Performance |
|
|
47 | (20) |
|
|
|
|
|
Chapter 5 Input Devices and Music Interaction |
|
|
67 | (18) |
|
|
|
|
|
|
|
|
|
Chapter 6 Capturing Bowing Gesture: Interpreting Individual Technique |
|
|
85 | (20) |
|
|
|
Chapter 7 Interactive Multimedia for Technology-Enhanced Learning with Multimodal Feedback |
|
|
105 | (22) |
|
|
|
Chapter 8 Online Gesture Analysis and Control of Audio Processing |
|
|
127 | (16) |
|
|
|
|
|
|
|
|
|
|
|
Section II Musical Robots and Automated Instruments |
|
|
|
Chapter 9 Automated Piano: Techniques for Accurate Expression of Piano Playing |
|
|
143 | (22) |
|
|
|
Chapter 10 McBlare: A Robotic Bagpipe Player |
|
|
165 | (14) |
|
|
|
|
|
|
|
Chapter 11 Violin Playing Robot and Kansei |
|
|
179 | (16) |
|
|
|
Chapter 12 Wind Instrument Playing Humanoid Robots |
|
|
195 | (20) |
|
|
|
|
|
Chapter 13 Multimodal Techniques for Human / Robot Interaction |
|
|
215 | (18) |
|
|
|
Chapter 14 Interactive Improvisation with a Robotic Marimba Player |
|
|
233 | (20) |
|
|
|
|
|
Chapter 15 Interactive Musical System for Multimodal Musician-Humanoid Interaction |
|
|
253 | (16) |
|
|
|
|
|
|
| Author Index |
|
269 | |
Jorge Solis received his B.S. degree in Electronics Systems from the Monterrey Institute of Technology (Mexico) in 1998 and the Ph.D. degree from the Scuola Superiore SantAnna, Perceptual Robotics Laboratory (Italy) in 2004. He is an Assistant Professor of Robotics at Waseda University and Visiting Researcher at the Humanoid Robotics Institute, Waseda University. Prof. Solis is the author and co-author of over 130 technical papers for International Journals and Conferences. He is currently a co-chair of the IEEE-RAS TC on Biorobotics and Roboethics. He received the Finalist Awards on Entertainment Robots and Systems at the IROS2007 and IROS2009 (sponsored by the New Technology Foundation). His research involves the Humanoid Robots; Medical Robots and Systems; Education Robots; Rehabilitation Robots, Haptics Interfaces, Sound/ Gesture/Vision Recognition Systems and Tele-operation Systems.
Kia Ng received his B.Sc. degree in Computational Science and Music, and Ph.D. in Computer Science from the University of Leeds (UK). Kia is director of the Interdisciplinary Centre for Scientific Research in Music (ICSRiM) and Senior Lecturer at the University of Leeds. His research interests include interactive multimedia, gesture analysis, computer vision and computer music. He has contributed to a range of large-scale research projects, including i-Maestro on technology-enhanced learning, CASPAR, and AXMEDIS on cross-media, with over 20M Euro funding. Kia has more than 150 publications and presented talks in over 30 countries including keynotes and invited lectures in Canada, China, France, Germany, Japan, UK, and USA. He has appeared on television and radio interviews such as BBC, Sky TV, and has beenfeatured in the New York Times, Guardian, Financial Times, and others. Kia has also organised over 15 international events including conferences, exhibitions and a convention. Kia is a chartered scientist, a fellow of the Royal Society of Arts and a fellow of the Institute of Directors. Web: www.kcng.org
Kia Ng received his B.Sc. degree in Computational Science and Music, and Ph.D. in Computer Science from the University of Leeds (UK). Kia is director of the Interdisciplinary Centre for Scientific Research in Music (ICSRiM) and Senior Lecturer at the University of Leeds. His research interests include interactive multimedia, gesture analysis, computer vision and computer music. He has contributed to a range of large-scale research projects, including i-Maestro on technology-enhanced learning, CASPAR, and AXMEDIS on cross-media, with over 20M Euro funding. Kia has more than 150 publications and presented talks in over 30 countries including keynotes and invited lectures in Canada, China, France, Germany, Japan, UK, and USA. He has appeared on television and radio interviews such as BBC, Sky TV, and has been featured in the New York Times, Guardian, Financial Times, and others. Kia has also organised over 15 international events including conferences, exhibitions and a convention. Kia is a chartered scientist, a fellow of the Royal Society of Arts and a fellow of the Institute of Directors. Web: www.kcng.org
Kia Ng received his B.Sc. degree in Computational Science and Music, and Ph.D. in Computer Science from the University of Leeds (UK). Kia is director of the Interdisciplinary Centre for Scientific Research in Music (ICSRiM) and Senior Lecturer at the University of Leeds. His research interests include interactive multimedia, gesture analysis, computer vision and computer music. He has contributed to a range of large-scale research projects, including i-Maestro on technology-enhanced learning, CASPAR, and AXMEDIS on cross-media, with over 20M Euro funding. Kia has more than 150 publications and presented talks in over 30 countries including keynotes and invited lectures in Canada, China, France, Germany, Japan, UK, and USA. He has appeared on television and radio interviews such as BBC, Sky TV, and has been featured in the New York Times, Guardian, Financial Times, and others. Kia has also organised over 15 international events including conferences, exhibitions and a convention. Kia is a chartered scientist, a fellow of the Royal Society of Arts and a fellow of the Institute of Directors. Web: www.kcng.org
Kia Ng received his B.Sc. degree in Computational Science and Music, and Ph.D. in Computer Science from the University of Leeds (UK). Kia is director of the Interdisciplinary Centre for Scientific Research in Music (ICSRiM) and Senior Lecturer at the University of Leeds. His research interests include interactive multimedia, gesture analysis, computer vision and computer music. He has contributed to a range of large-scale research projects, including i-Maestro on technology-enhanced learning, CASPAR, and AXMEDIS on cross-media, with over 20M Euro funding. Kia has more than 150 publications and presented talks in over 30 countries including keynotes and invited lectures in Canada, China, France, Germany, Japan, UK, and USA. He has appeared on television and radio interviews such as BBC, Sky TV, and has been featured in the New York Times, Guardian, Financial Times, and others. Kia has also organised over 15 international events including conferences, exhibitions and a convention. Kia is a chartered scientist, a fellow of the Royal Society of Arts and a fellow of the Institute of Directors. Web: www.kcng.org
Kia Ng received his B.Sc. degree in Computational Science and Music, and Ph.D. in Computer Science from the University of Leeds (UK). Kia is director of the Interdisciplinary Centre for Scientific Research in Music (ICSRiM) and Senior Lecturer at the University of Leeds. His research interests include interactive multimedia, gesture analysis, computer vision and computer music. He has contributed to a range of large-scale research projects, including i-Maestro on technology-enhanced learning, CASPAR, and AXMEDIS on cross-media, with over 20M Euro funding. Kia has more than 150 publications and presented talks in over 30 countries including keynotes and invited lectures in Canada, China, France, Germany, Japan, UK, and USA. He has appeared on television and radio interviews such as BBC, Sky TV, and has been featured in the New York Times, Guardian, Financial Times, and others. Kia has also organised over 15 international events including conferences, exhibitions and a convention. Kia is a chartered scientist, a fellow of the Royal Society of Arts and a fellow of the Institute of Directors. Web: www.kcng.org
