EPEC - Engineering Predictability with Embodied Cognition (completed)

• Charles P. Martin, Kai Olav Ellefsen, and Jim Torresen. 2017. Deep models for ensemble touch-screen improvisation. Proceedings of Audio Mostly ’17.
• Charles P. Martin and Jim Torresen. 2017. Exploring social mobile music with tiny touch-screen performances. Proceedings of the 14th sound and music computing conference, Aalto University, pp. 175–180.
• Charles P. Martin and Jim Torresen. 2017. MicroJam: An app for sharing tiny touch-screen performances. Proceedings of the international conference on new interfaces for musical expression, Aalborg University Copenhagen, pp. 495–496.
• Nygaard, Tønnes Frostad; Tørresen, Jim; Glette, Kyrre Multi-objective Evolution of Fast and Stable Gaits on a Physical Quadruped Robotic Platform. Proceedings of the 2016 IEEE Symposium Series on Computational Intelligence (SSCI)
• Tørresen, Jim; Iversen, Andreas Høyer; Greisiger, Ralf Data from Past Patients used to Streamline Adjustment of Levels for Cochlear Implant for New Patients. Proceedings of the 2016 IEEE Symposium Series on Computational Intelligence (SSCI)

• Nygaard, Tønnes; Martin, Charles Patrick; Tørresen, Jim; Glette, Kyrre & Howard, David (2021). Real-world embodied AI through a morphologically adaptive quadruped robot. Nature Machine Intelligence. doi: 10.1038/s42256-021-00320-3. Full text in Research Archive Show summary

  Robots are traditionally bound by a fixed morphology during their operational lifetime, which is limited to adapting only their control strategies. Here we present the first quadrupedal robot that can morphologically adapt to different environmental conditions in outdoor, unstructured environments. Our solution is rooted in embodied AI and comprises two components: (1) a robot that permits in situ morphological adaptation and (2) an adaptation algorithm that transitions between the most energy-efficient morphologies on the basis of the currently sensed terrain. We first build a model that describes how the robot morphology affects performance on selected terrains. We then test continuous adaptation on realistic outdoor terrain while allowing the robot to constantly update its model. We show that the robot exploits its training to effectively transition between different morphological configurations, exhibiting substantial performance improvements over a non-adaptive approach. The demonstrated benefits of real-world morphological adaptation demonstrate the potential for a new embodied way of incorporating adaptation into future robotic designs.

• Nygaard, Tønnes; Martin, Charles Patrick; Howard, David; Tørresen, Jim & Glette, Kyrre (2021). Environmental Adaptation of Robot Morphology and Control Through Real-world Evolution. Evolutionary Computation. ISSN 1063-6560. doi: 10.1162/evco_a_00291. Full text in Research Archive Show summary

  Robots operating in the real world will experience a range of different environments and tasks. It is essential for the robot to have the ability to adapt to its surroundings to work efficiently in changing conditions. Evolutionary robotics aims to solve this by optimizing both the control and body (morphology) of a robot, allowing adaptation to internal, as well as external factors. Most work in this field has been done in physics simulators, which are relatively simple and not able to replicate the richness of interactions found in the real world. Solutions that rely on the complex interplay between control, body, and environment are therefore rarely found. In this paper, we rely solely on real-world evaluations and apply evolutionary search to yield combinations of morphology and control for our mechanically self-reconfiguring quadruped robot. We evolve solutions on two distinct physical surfaces and analyze the results in terms of both control and morphology. We then transition to two previously unseen surfaces to demonstrate the generality of our method. We find that the evolutionary search finds high-performing and diverse morphology-controller configurations by adapting both control and body to the different properties of the physical environments. We additionally find that morphology and control vary with statistical significance between the environments. Moreover, we observe that our method allows for morphology and control parameters to transfer to previously-unseen terrains, demonstrating the generality of our approach.

• Nordmoen, Jørgen; Nygaard, Tønnes Frostad; Samuelsen, Eivind & Glette, Kyrre (2021). On Restricting Real-Valued Genotypes in Evolutionary Algorithms. In Castillo, Pedro A. & Jiménez Laredo, Juan Luiz (Ed.), Applications of Evolutionary Computation. Springer. ISSN 978-3-030-72699-7. p. 3–16. doi: 10.1007/978-3-030-72699-7_1. Full text in Research Archive
• Nygaard, Tønnes; Howard, David & Glette, Kyrre (2020). Real world morphological evolution is feasible. In Coello Coello, Carlos A. (Eds.), GECCO '20: Proceedings of the 2020 Genetic and Evolutionary Computation Conference Companion. Association for Computing Machinery (ACM). ISSN 978-1-4503-7127-8. p. 1392–1394. doi: 10.1145/3377929.3398095. Show summary

  Evolutionary algorithms offer great promise for the automatic design of robot bodies, tailoring them to specific environments or tasks. Most research is done on simplified models or virtual robots in physics simulators, which do not capture the natural noise and richness of the real world. Very few of these virtual robots are built as physical robots, and the few that are will rarely be further improved in the actual environment they operate in, limiting the effectiveness of the automatic design process. We utilize our shape-shifting quadruped robot, which allows us to optimize the design in its real-world environment. The robot is able to change the length of its legs during operation, and is robust enough for complex experiments and tasks. We have co-evolved control and morphology in several different scenarios, and have seen that the algorithm is able to exploit the dynamic morphology solely through real-world experiments.

• Martin, Charles Patrick; Glette, Kyrre; Nygaard, Tønnes & Tørresen, Jim (2020). Understanding Musical Predictions With an Embodied Interface for Musical Machine Learning. Frontiers in Artificial Intelligence. ISSN 2624-8212. 3(6). doi: 10.3389/frai.2020.00006. Show summary

  Machine-learning models of music often exist outside the worlds of musical performance practice and abstracted from the physical gestures of musicians. In this work, we consider how a recurrent neural network (RNN) model of simple music gestures may be integrated into a physical instrument so that predictions are sonically and physically entwined with the performer's actions. We introduce EMPI, an embodied musical prediction interface that simplifies musical interaction and prediction to just one dimension of continuous input and output. The predictive model is a mixture density RNN trained to estimate the performer's next physical input action and the time at which this will occur. Predictions are represented sonically through synthesized audio, and physically with a motorized output indicator. We use EMPI to investigate how performers understand and exploit different predictive models to make music through a controlled study of performances with different models and levels of physical feedback. We show that while performers often favor a model trained on human-sourced data, they find different musical affordances in models trained on synthetic, and even random, data. Physical representation of predictions seemed to affect the length of performances. This work contributes new understandings of how musicians use generative ML models in real-time performance backed up by experimental evidence. We argue that a constrained musical interface can expose the affordances of embodied predictive interactions.

• Martin, Charles Patrick & Gardner, Henry (2019). Free-Improvised Rehearsal-as-Research for Musical HCI. In Holland, Simon; Mudd, Tom; Wilkie-McKenna, Katie; McPherson, Andrew & Wanderley, Marcelo M. (Ed.), New Directions in Music and Human-Computer Interaction. Springer. ISSN 978-3-319-92068-9. p. 269–284. doi: https:/doi.org/10.1007/978-3-319-92069-6_17. Full text in Research Archive Show summary

  The formal evaluation of new interfaces for musical expression (NIMEs) in their use by ensembles of musicians is a challenging problem in human-computer interaction (HCI). NIMEs are designed to support creative expressions that are often improvised and unexpected. In the collaborative setting of a musical ensemble, interactions are complex and it can be almost impossible to directly evaluate the impact of interface variations. The evaluation environment also needs to be carefully considered. In the wild, concert pressures and practicalities limit experimental control. In the laboratory, studies may not sufficiently reflect real-world usage to make their conclusions relevant. To address some of these issues, we propose a methodology of rehearsal-as-research to study free improvisation by ensembles of NIME performers. In this methodology, evaluation sessions are structured to mirror established practices for improvisation training and performance development. Such sessions can allow controlled, order-balanced studies with extensive data collection in the style of factorial HCI experiments while preserving the artistic setting of a rehearsal. Experiment design, questionnaires, and objective measures such as session duration will be discussed along with two case studies.

• Martin, Charles Patrick & Tørresen, Jim (2019). Data Driven Analysis of Tiny Touchscreen Performance with MicroJam. Computer Music Journal. ISSN 0148-9267. 43(4). Full text in Research Archive Show summary

  The widespread adoption of mobile devices, such as smartphones and tablets, has made touchscreens a common interface for musical performance. While new mobile music instrument have been investigated from design and user experience perspectives, there is little examination of the performers’ musical outputs. In this work, we introduce a constrained touchscreen performance app, MicroJam, designed to enable collaboration between performers, and engage in a data-driven analysis of more than 1600 performances using the app. MicroJam constrains performances to five seconds, and emphasises frequent and casual music making through a social media-inspired interface. Performers collaborate by replying to performances, adding new musical layers that are played back at the same time. Our analysis shows that users tend to focus on the centre and diagonals of the touchscreen area, and tend to swirl or swipe rather than tap. We also observe that while long swipes dominate the visual appearance of performances, the majority of interactions are short with limited expressive possibilities. Our findings enhance our understanding of how users perform in touchscreen apps and could be applied in future app designs for social musical interaction.

• Nygaard, Tønnes Frostad; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Evolving Robots on Easy Mode: Towards a Variable Complexity Controller for Quadrupeds. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 11454 LNCS, p. 616–632. doi: 10.1007/978-3-030-16692-2_41. Full text in Research Archive
• Wallace, Benedikte & Martin, Charles Patrick (2019). Comparing models for harmony prediction in an interactive audio looper. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 11453 LNCS, p. 173–187. doi: 10.1007/978-3-030-16667-0_12.
• Teigen, Bjørn Ivar; Ellefsen, Kai Olav & Tørresen, Jim (2019). A Categorization of Reinforcement Learning Exploration Techniques Which Facilitates Combination of Different Methods, Proceedings of the 9th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics. IEEE (Institute of Electrical and Electronics Engineers). ISSN 978-1-5386-8129-9. p. 189–194. doi: 10.1109/DEVLRN.2019.8850685.
• Nordmoen, Jørgen Halvorsen; Nygaard, Tønnes Frostad; Ellefsen, Kai Olav & Glette, Kyrre (2019). Evolved embodied phase coordination enables robust quadruped robot locomotion. In López-Ibáñez, Manuel (Eds.), GECCO '19: Proceedings of the Genetic and Evolutionary Computation Conference. Association for Computing Machinery (ACM). ISSN 978-1-4503-6111-8. p. 133–141. doi: 10.1145/3321707.3321762. Full text in Research Archive
• Martin, Charles Patrick & Tørresen, Jim (2019). An Interactive Musical Prediction System with Mixture Density Recurrent Neural Networks. In Queiroz, Marcelo & Xambo Sedo, Anna (Ed.), Proceedings of the International Conference on New Interfaces for Musical Expression. Universidade Federal do Rio Grande do Sul. ISSN 2220-4792. p. 260–265. Full text in Research Archive
• Næss, Torgrim Rudland & Martin, Charles Patrick (2019). A Physical Intelligent Instrument using Recurrent Neural Networks. In Queiroz, Marcelo & Xambo Sedo, Anna (Ed.), Proceedings of the International Conference on New Interfaces for Musical Expression. Universidade Federal do Rio Grande do Sul. ISSN 2220-4792. p. 79–82. Full text in Research Archive
• Faitas, Andrei; Baumann, Synne Engdahl; Næss, Torgrim Rudland; Tørresen, Jim & Martin, Charles Patrick (2019). Generating Convincing Harmony Parts with Simple Long Short-Term Memory Networks. In Queiroz, Marcelo & Xambo Sedo, Anna (Ed.), Proceedings of the International Conference on New Interfaces for Musical Expression. Universidade Federal do Rio Grande do Sul. ISSN 2220-4792. Full text in Research Archive
• Ellefsen, Kai Olav; Huizinga, Joost & Tørresen, Jim (2019). Guiding Neuroevolution with Structural Objectives. Evolutionary Computation. ISSN 1063-6560. 28(1), p. 115–140. doi: 10.1162/evco_a_00250. Full text in Research Archive Show summary

  The structure and performance of neural networks are intimately connected, and by use of evolutionary algorithms, neural network structures optimally adapted to a given task can be explored. Guiding such neuroevolution with additional objectives related to network structure has been shown to improve performance in some cases, especially when modular neural networks are beneficial. However, apart from objectives aiming to make networks more modular, such structural objectives have not been widely explored. We propose two new structural objectives and test their ability to guide evolving neural networks on two problems which can benefit from decomposition into subtasks. The first structural objective guides evolution to align neural networks with a user-recommended decomposition pattern. Intuitively, this should be a powerful guiding target for problems where human users can easily identify a structure. The second structural objective guides evolution towards a population with a high diversity in decomposition patterns. This results in exploration of many different ways to decompose a problem, allowing evolution to find good decompositions faster. Tests on our target problems reveal that both methods perform well on a problem with a very clear and decomposable structure. However, on a problem where the optimal decomposition is less obvious, the structural diversity objective is found to outcompete other structural objectives—and this technique can even increase performance on problems without any decomposable structure at all.

• Ellefsen, Kai Olav & Tørresen, Jim (2019). Self-adapting Goals Allow Transfer of Predictive Models to New Tasks, Nordic Artificial Intelligence Research and Development: Third Symposium of the Norwegian AI Society, NAIS 2019. Springer. ISSN 978-3-030-35664-4. p. 28–39. doi: https:/doi.org/10.1007/978-3-030-35664-4_3.
• Nygaard, Tønnes Frostad; Nordmoen, Jørgen Halvorsen; Ellefsen, Kai Olav; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Experiences from Real-World Evolution with DyRET: Dynamic Robot for Embodied Testing, Nordic Artificial Intelligence Research and Development: Third Symposium of the Norwegian AI Society, NAIS 2019. Springer. ISSN 978-3-030-35664-4. p. 58–68. doi: https:/doi.org/10.1007/978-3-030-35664-4_6.
• Becker, Artur; Herrebrøden, Henrik; Gonzalez Sanchez, Victor Evaristo; Nymoen, Kristian; Dal Sasso Freitas, Carla Maria & Tørresen, Jim [Show all 7 contributors for this article] (2019). Functional Data Analysis of Rowing Technique Using Motion Capture Data. In Coleman, Grisha (Eds.), Proceedings of the 6th International Conference on Movement and Computing. ACM Publications. ISSN 978-1-4503-7654-9. doi: 10.1145/3347122.3347135. Show summary

  We present an approach to analyzing the motion capture data of rowers using bivariate functional principal component analysis (bfPCA). The method has been applied on data from six elite rowers rowing on an ergometer. The analyses of the upper and lower body coordination during the rowing cycle revealed significant differ- ences between the rowers, even though the data was normalized to account for differences in body dimensions. We make an argument for the use of bfPCA and other functional data analysis methods for the quantitative evaluation and description of technique in sports.

• Weber, Aline; Martin, Charles Patrick; Tørresen, Jim & da Silva, Bruno Castro (2019). Identifying Reusable Early-Life Options, Proceedings of the 9th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics. IEEE (Institute of Electrical and Electronics Engineers). ISSN 978-1-5386-8129-9. doi: 10.1109/DEVLRN.2019.8850725.
• Nygaard, Tønnes Frostad; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Self-Modifying Morphology Experiments with DyRET: Dynamic Robot for Embodied Testing. IEEE International Conference on Robotics and Automation (ICRA). ISSN 1050-4729. 2019-May, p. 9446–9452. doi: 10.1109/ICRA.2019.8793663. Full text in Research Archive
• Miseikis, Justinas; Brijacak, Inka; Yahyanejad, Saeed; Glette, Kyrre; Elle, Ole Jacob & Tørresen, Jim (2019). Two-Stage Transfer Learning for Heterogeneous Robot Detection and 3D Joint Position Estimation in a 2D Camera Image Using CNN. IEEE International Conference on Robotics and Automation (ICRA). ISSN 1050-4729. 2019-May, p. 8883–8889. doi: 10.1109/ICRA.2019.8794077. Full text in Research Archive
• Garcia, Rafael; Falcao, Alexandre Xavier; Telea, Alexandru C.; Castro da Silva, Bruno; Tørresen, Jim & Comba, Joao Luiz Dihl (2019). A Methodology for Neural Network Architectural Tuning Using Activation Occurrence Maps. In Jayne, Chrisina & Somogyvári, Zoltán (Ed.), 2019 International Joint Conference on Neural Networks (IJCNN) . IEEE (Institute of Electrical and Electronics Engineers). ISSN 978-1-7281-1985-4. doi: 10.1109/IJCNN.2019.8852223.
• Weber, Aline; Alegre, Lucas N.; Tørresen, Jim & Castro da Silva, Bruno (2019). Parameterized Melody Generation with Autoencoders and Temporally-Consistent Noise. In Visi, Federico (Eds.), Music Proceedings of the International Conference on New Interfaces for Musical Expression. Universidade Federal do Rio Grande do Sul. ISSN 2220-4792. p. 174–179. Full text in Research Archive
• Martin, Charles Patrick; Jensenius, Alexander Refsum & Tørresen, Jim (2018). Composing an ensemble standstill work for Myo and Bela. In Dahl, Luke; Bowman, Doug & Martin, Tom (Ed.), Proceedings of the International Conference On New Interfaces For Musical Expression. Virginia Tech. ISSN 2220-4792. p. 196–197. Full text in Research Archive Show summary

  This paper describes the process of developing a standstill performance work using the Myo gesture control armband and the Bela embedded computing platform. The combination of Myo and Bela allows a portable and extensible version of the standstill performance concept while introducing muscle tension as an additional control parameter. We describe the technical details of our setup and introduce Myo-to-Bela and Myo-to-OSC software bridges that assist with prototyping compositions using the Myo controller.

• Garcia, Rafael; Telea, Alexandru C; Castro da Silva, Bruno; Tørresen, Jim & Dihl Comba, Joao Luiz (2018). A task-and-technique centered survey on visual analytics for deep learning model engineering. Computers & graphics. ISSN 0097-8493. 77, p. 30–49. doi: 10.1016/j.cag.2018.09.018.
• Nygaard, Tønnes Frostad; Martin, Charles Patrick; Samuelsen, Eivind; Tørresen, Jim & Glette, Kyrre (2018). Real-world evolution adapts robot morphology and control to hardware limitations. In aguirre, hernan (Eds.), GECCO '18: Proceedings of the Genetic and Evolutionary Computation Conference. Association for Computing Machinery (ACM). ISSN 978-1-4503-5618-3. p. 125–132. doi: 10.1145/3205455.3205567. Full text in Research Archive Show summary

  For robots to handle the numerous factors that can affect them in the real world, they must adapt to changes and unexpected events. Evolutionary robotics tries to solve some of these issues by automatically optimizing a robot for a specific environment. Most of the research in this field, however, uses simplified representations of the robotic system in software simulations. The large gap between performance in simulation and the real world makes it challenging to transfer the resulting robots to the real world. In this paper, we apply real world multi-objective evolutionary optimization to optimize both control and morphology of a four-legged mammal-inspired robot. We change the supply voltage of the system, reducing the available torque and speed of all joints, and study how this affects both the fitness, as well as the morphology and control of the solutions. In addition to demonstrating that this real-world evolutionary scheme for morphology and control is indeed feasible with relatively few evaluations, we show that evolution under the different hardware limitations results in comparable performance for low and moderate speeds, and that the search achieves this by adapting both the control and the morphology of the robot.

• Nordmoen, Jørgen Halvorsen; Ellefsen, Kai Olav & Glette, Kyrre (2018). Combining MAP-Elites and Incremental Evolution to Generate Gaits for a Mammalian Quadruped Robot. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 10784 LNCS, p. 719–733. doi: 10.1007/978-3-319-77538-8_48. Full text in Research Archive

View all works in Cristin

• Ellefsen, Kai Olav & Rohlfing, Katharina J. (2019). Proceedings of the 9th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics. IEEE (Institute of Electrical and Electronics Engineers). ISBN 978-1-5386-8129-9. 340 p.

View all works in Cristin

• Glette, Kyrre (2020). Evolutionary algorithms for intelligent robots.
• Tørresen, Jim (2019). Kunstig intelligens – hvem, hva og hvor. (Eng. Artificial Intelligence – who, what and where).
• Tørresen, Jim (2019). Making Robots Adaptive and Preferable to Humans.
• Tørresen, Jim (2019). Intelligent Robots and Systems in Real-World Environment.
• Nygaard, Tønnes Frostad & Glette, Kyrre (2019). Her er universitetets mest avanserte, selvlærende robot. [Business/trade/industry journal]. Apollon.
• Glette, Kyrre (2019). Kunstig intelligens for tilpasningsdyktige roboter.
• Glette, Kyrre; Nygaard, Tønnes Frostad & Vogt, Yngve (2019). Her er universitetets nest selvlærende robot. [Business/trade/industry journal]. Teknisk ukeblad.
• Tørresen, Jim (2019). Intelligent and Adaptive Robots in Real-World Environment. Show summary

  240862 259293 247697 288285 262762

• Tørresen, Jim (2019). Design and Control of Robots for Real-World Environment.
• Tørresen, Jim (2019). Artificial Intelligence and Applications in Health and Care .
• Tørresen, Jim (2019). Sensing Human State with Application in Older People Care and Mental Health Treatment.
• Tørresen, Jim (2019). Supporting Older People with Robots for Independent Living. Show summary

  247697 288285 262762

• Ellefsen, Kai Olav & Tørresen, Jim (2019). Evolutionary Robotics: Automatic design of robot bodies and control.
• Tørresen, Jim (2019). Hva er kunstig intelligens?
• Tørresen, Jim (2019). Future and Ethical Perspectives of Robotics and AI.
• Nygaard, Tønnes Frostad; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Self-Modifying Morphology Experiments with DyRET: Dynamic Robot for Embodied Testing.
• Teigen, Bjørn Ivar; Ellefsen, Kai Olav & Tørresen, Jim (2019). A Categorization of Reinforcement Learning Exploration Techniques Which Facilitates Combination of Different Methods.
• Ellefsen, Kai Olav & Tørresen, Jim (2019). Self-Adapting Goals Allow Transfer of Predictive Models to New Tasks.
• Nordmoen, Jørgen Halvorsen; Nygaard, Tønnes Frostad; Ellefsen, Kai Olav & Glette, Kyrre (2019). Evolved embodied phase coordination enables robust quadruped robot locomotion .
• Nygaard, Tønnes Frostad; Nordmoen, Jørgen Halvorsen; Ellefsen, Kai Olav; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Experiences from Real-World Evolution with DyRET: Dynamic Robot for Embodied Testing.
• Ellefsen, Kai Olav; Huizinga, Joost & Tørresen, Jim (2019). Guiding Neuroevolution with Structural Objectives.
• Becker, Artur; Herrebrøden, Henrik; Gonzalez Sanchez, Victor Evaristo; Nymoen, Kristian; Dal Sasso Freitas, Carla Maria & Tørresen, Jim [Show all 7 contributors for this article] (2019). Functional Data Analysis of Rowing Technique Using Motion Capture Data. Show summary

  We present an approach to analyzing the motion capture data ofrowers using bivariate functional principal component analysis(bfPCA). The method has been applied on data from six elite rowersrowing on an ergometer. The analyses of the upper and lower bodycoordination during the rowing cycle revealed significant differences between the rowers, even though the data was normalized toaccount for differences in body dimensions. We make an argumentfor the use of bfPCA and other functional data analysis methods forthe quantitative evaluation and description of technique in sports.

• Tørresen, Jim; Glette, Kyrre & Ellefsen, Kai Olav (2019). Adaptive Robot Body and Control for Real-World Environments.
• Tørresen, Jim; Glette, Kyrre & Ellefsen, Kai Olav (2019). Intelligent, Adaptive Robots in Real-World Scenarios.
• Ellefsen, Kai Olav (2019). Hva Kan Roboter Lære av Biologisk Liv?
• Nordmoen, Jørgen Halvorsen & Fadelli, Ingrid (2019). A new method to enable robust locomotion in a quadruped robot. [Internet]. TechXplore.
• Næss, Torgrim Rudland; Tørresen, Jim & Martin, Charles Patrick (2019). A Physical Intelligent Instrument using Recurrent Neural Networks.
• Martin, Charles Patrick & Tørresen, Jim (2019). An Interactive Musical Prediction System with Mixture Density Recurrent Neural Networks.
• Martin, Charles Patrick & Torresen, Jim (2019). An Interactive Music Prediction System with Mixture Density Recurrent Neural Networks.
• Martin, Charles Patrick; Næss, Torgrim Rudland; Faitas, Andrei & Baumann, Synne Engdahl (2019). Session on Musical Prediction and Generation with Deep Learning.
• Nygaard, Tønnes Frostad; Nordmoen, Jørgen Halvorsen; Martin, Charles Patrick; Tørresen, Jim & Glette, Kyrre (2019). Lessons Learned from Real-World Experiments with DyRET: the Dynamic Robot for Embodied Testing.
• Glette, Kyrre (2019). Kunstig intelligens for tilpasningsdyktige roboter .
• Faitas, Andrei; Baumann, Synne Engdahl; Torresen, Jim & Martin, Charles Patrick (2019). Generating Convincing Harmony Parts with Simple Long Short-Term Memory Networks.
• Miseikis, Justinas; Brijacak, Inka; Yahyanejad, Saeed; Glette, Kyrre; Elle, Ole Jacob & Tørresen, Jim (2019). Two-Stage Transfer Learning for Heterogeneous Robot Detection and 3D Joint Position Estimation in a 2D Camera Image Using CNN.
• Miura, Jun & Tørresen, Jim (2019). Intelligent Robot Technologies for Care and Lifestyle Support .
• Comba, Joao Luiz Dihl & Tørresen, Jim (2019). Visual Data Analysis of Unstructured and Big Data.
• Rohlfing, Katharina J. & Tørresen, Jim (2019). Explainability: an interactive view.
• Tørresen, Jim (2018). Remote Lab and Applications for High Performance and Embedded Architectures.
• Tørresen, Jim (2018). Artificial Intelligence – State-of-the-art.
• Tørresen, Jim (2018). Kunstig Intelligens – Lærende og tilpasningsdyktig teknologi.
• Tørresen, Jim (2018). Roboter kommer nærmere – skal vi glede eller grue oss?
• Martin, Charles Patrick; Jensenius, Alexander Refsum & Tørresen, Jim (2018). Composing an ensemble standstill work for Myo and Bela. Show summary

  This paper describes the process of developing a standstill performance work using the Myo gesture control armband and the Bela embedded computing platform. The combination of Myo and Bela allows a portable and extensible version of the standstill performance concept while introducing muscle tension as an additional control parameter. We describe the technical details of our setup and introduce Myo-to-Bela and Myo-to-OSC software bridges that assist with prototyping compositions using the Myo controller.

• Martin, Charles Patrick; Xambó, Anna; Visi, Federico; Morreale, Fabio & Jensenius, Alexander Refsum (2018). Stillness under Tension. Show summary

  Stillness Under Tension​ is an ensemble standstill work for Myo gesture control armband and Bela embedded music platform. Humans are incapable of standing completely still due to breathing and other involuntary micromotions. This work explores the expressive space of standing still through an inverse action-sound mapping: less movement leads to more sound. Four performers stand as still as possible on stage, each wearing a Myo armband connected to a Bela embedded sound processing platform. The Myo is used to measure the performers movement, and the muscle activity in their forearm which they can use--both voluntarily and involuntarily--to control a synthesised sound world. Each performer uses one Myo and Bela in a musical space defined by their physical position and posture while standing still.

• Jensenius, Alexander Refsum; Martin, Charles Patrick; Bjerkestrand, Kari Anne Vadstensvik & Johnson, Victoria (2018). Stillness under Tension.
• Ellefsen, Kai Olav & Tørresen, Jim (2018). Evolutionary Robotics: Automatic design of robot controllers and bodies.
• Ellefsen, Kai Olav (2018). Evolusjonær Robotikk: Automatisk design og kontroll av roboter.
• Søyseth, Vegard Dønnem; Nygaard, Tønnes Frostad; Martin, Charles Patrick; Uddin, Md Zia & Ellefsen, Kai Olav (2018). ROBIN-Stand ved Cutting Edge 2018.
• (2018). Real-World Evolution Adapts Robot Morphology and Control to Hardware Limitations.
• Nygaard, Tønnes Frostad; Søyseth, Vegard Dønnem; Nordmoen, Jørgen Halvorsen & Glette, Kyrre (2018). Stand with the DyRET robot.
• Nygaard, Tønnes Frostad & Khattar, Shivani (2018). How a shape-shifting robot is learning from its mistakes. [TV]. NBC news.
• Nygaard, Tønnes Frostad & Simon, Matt (2018). The shape-shifting robot that evolves by falling down. [Business/trade/industry journal]. Wired Science.
• Nygaard, Tønnes Frostad & Papadopoulos, Loukia (2018). New Evolving Robot Teaches Itself to Walk Through Trial and Error. [Business/trade/industry journal]. Interesting Engineering.
• Martin, Charles Patrick (2018). MicroJam. Show summary

  MicroJam is a mobile app for sharing tiny touch-screen performances. Mobile applications that streamline creativity and social interaction have enabled a very broad audience to develop their own creative practices. While these apps have been very successful in visual arts (particularly photography), the idea of social music-making has not had such a broad impact. MicroJam includes several novel performance concepts intended to engage the casual music maker and inspired by current trends in social creativity support tools. Touch-screen performances are limited to 5-seconds, instrument settings are posed as sonic "filters", and past performances are arranged as a timeline with replies and layers. These features of MicroJam encourage users not only to perform music more frequently, but to engage with others in impromptu ensemble music making.

• Nygaard, Tønnes Frostad & Nordmoen, Jørgen Halvorsen (2018). Dynamic Robot for Embodied Testing, Open Source Material.
• Nygaard, Tønnes Frostad & Dormehl, Luke (2018). This robot taught itself how to walk and it’s as clumsy as a newborn deer. [Business/trade/industry journal]. Digital Trends.
• Nygaard, Tønnes Frostad & Gonzales, Robbie (2018). How a Flock of Drones Developed Collective Intelligence. [Business/trade/industry journal]. Wired Science.
• Nygaard, Tønnes Frostad & Simon, Matt (2018). How Roboticists Are Copying Nature To Make Fantastical Machines. [Internet]. Wired Science.
• Martin, Charles Patrick (2018). Predictive Music Systems for Interactive Performance. Show summary

  Automatic music generation is a compelling task where much recent progress has been made with deep learning models. But how these models can be integrated into interactive music systems; how can they encourage or enhance the music making of human users? Musical performance requires prediction to operate instruments, and perform in groups. Predictive models can help interactive systems to understand their temporal context, and ensemble behaviour. Deep learning can allow data-driven models with a long memory of past states. This process could be termed "predictive musical interaction", where a predictive model is embedded in a musical interface, assisting users by predicting unknown states of musical processes. I’ll discuss a framework for predictive musical interaction including examples from our lab, and consider how this work could be applied more broadly in HCI and robotics. This talk will cover material from this paper: https://arxiv.org/abs/1801.10492

• Martin, Charles Patrick; Glette, Kyrre & Tørresen, Jim (2018). Creative Prediction with Neural Networks. Show summary

  The goal of this tutorial is to apply predictive machine learning models to creative data. The focus of the tutorial will be recurrent neural networks (RNNs), deep learning models that can be used to generate sequential and temporal data. RNNs can be applied to many kinds of creative data including text and music. They can learn the long-range structure from a corpus of data and “create” new sequences by predicting one element at a time. When embedded in a creative interface, they can be used for “predictive interaction” where a human collaborates with, influences, and is influenced by a generative neural network. We will walk through the fundamental steps for training creative RNNs using live-coded demonstrations with Python code in Jupyter Notebooks. These steps are: collecting and cleaning data, building and training an RNN, and developing predictive interactions. We will also have live demonstrations and interactive live-hacking of our creative RNN systems! You’re welcome to bring a laptop with python to the tutorial and load up our code examples, or to follow along with us on the screen!

• Garcia Ceja, Enrique Alejandro; Ellefsen, Kai Olav; Martin, Charles Patrick & Tørresen, Jim (2018). Prediction, Interaction, and User Behaviour. Show summary

  The goal of this tutorial is to apply predictive machine learning models to human behaviour through a human computer interface. We will introduce participants to the key stages for developing predictive interaction in user-facing technologies: collecting and identifying data, applying machine learning models, and developing predictive interactions. Many of us are aware of recent advances in deep neural networks (DNNs) and other machine learning (ML) techniques; however, it is not always clear how we can apply these techniques in interactive and real-time applications. Apart from well-known examples such as image classification and speech recognition, what else can predictive ML models be used for? How can these computational intelligence techniques be deployed to help users? In this tutorial, we will show that ML models can be applied to many interactive applications to enhance users’ experience and engagement. We will demonstrate how sensor and user interaction data can be collected and investigated, modelled using classical ML and DNNs, and where predictions of these models can feed back into an interface. We will walk through these processes using live-coded demonstrations with Python code in Jupyter Notebooks so participants will be able to see our investigations live and take the example code home to apply in their own projects. Our demonstrations will be motivated from examples from our own research in creativity support tools, robotics, and modelling user behaviour. In creativity, we will show how streams of interaction data from a creative musical interface can be modelled with deep recurrent neural networks (RNNs). From this data, we can predict users’ future interactions, or the potential interactions of other users. This enables us to “fill in” parts of a tablet-based musical ensemble when other users are not available, or to continue a user’s composition with potential musical parts. In user behaviour, we will show how smartphone sensor data can be used to infer user contextual information such as physical activities. This contextual information can be used to trigger interactions in smart home or internet of things (IoT) environments, to help tune interactive applications to user’s needs, or to help track health data.

• Tørresen, Jim (2018). Intelligent Systems for Medical and Healthcare Applications.
• Tørresen, Jim (2018). Når etikk betyr alt. Dagens næringsliv. ISSN 0803-9372.
• Tørresen, Jim; Garcia Ceja, Enrique Alejandro; Ellefsen, Kai Olav & Martin, Charles Patrick (2018). Equipping Systems with Forecasting Capabilities .
• Martin, Charles Patrick (2018). Creative Prediction with Neural Networks.
• Tørresen, Jim (2018). Ethical Robots and Autonomous Systems.
• Tørresen, Jim (2018). Kunstig intelligens – hvem, hva og hvor.
• Tørresen, Jim (2018). Artificial Intelligence Applied for Real-World Systems.
• Næss, Torgrim Rudland; Martin, Charles Patrick & Tørresen, Jim (2019). A Physical Intelligent Instrument using Recurrent Neural Networks. Universitetet i Oslo.
• Tørresen, Jim; Teigen, Bjørn Ivar & Ellefsen, Kai Olav (2018). An Active Learning Perspective on Exploration in Reinforcement Learning. Universitetet i Oslo.
• Brustad, Henrik & Martin, Charles Patrick (2018). Digital Audio Generation with Neural Networks. Universitetet i Oslo.
• Fjeld, Matias Hermanrud & Tørresen, Jim (2018). 3D Spatial Navigation in Octrees with Reinforcement Learning. Universitetet i Oslo.
• Wallace, Benedikte & Martin, Charles Patrick (2018). Predictive songwriting with concatenative accompaniment. Universitetet i Oslo.

View all works in Cristin