UBISS 2024 - Workshops & Instructors

Workshop A: UBIQUITOUS ARTIFICIAL INTELLIGENCE: MACHINE LEARNING ON IOT DEVICES

Maximum number of participants to be enrolled to the workshop: 20

Artificial intelligence (AI), generative models (GenAI), and machine learning (ML) are becoming ubiquitous! These technologies are at the core of intelligent systems and are required to implement intelligent environments, autonomous robots and drones, and smart mobile devices. In this workshop, we focus on the Question: How can we implement intelligent functions on small networked devices? We will cover how to design the system architecture, ways to acquire and process data with sensors, the choice of ML algorithms, the design of deep learning architectures, how to train systems, and how to deploy models. In particular, we will teach how to implement technical solutions and what trade-offs need to be navigated.

Topics to be covered include:

Overview of AI and ML technologies for use on IoT devices,
System architectures for networked embedded intelligent systems,
Data acquisition and data processing for ML on mobile and embedded systems,
Deep learning architectures for resource-constrained devices,
Training ML models for and on mobile and embedded devices,
Deploying models on IoT devices to implement smart function.

We will look into the technical aspects of implementing such systems, covering:

Calling APIs in the cloud (e.g. Open AIs APIs or Microsoft’s Azure cognitive services),
Deploying models on a server and connecting remotely,
Running models on the edge devices,
Training (very small) models on embedded processors (e.g. TinyML, TensorFlow_lite),
Available libraries for resource-constrained devices.

In practical exercises and tutorials, we will look at different ways to use Machine Learning (ML) on embedded processors and mobile devices. In particular, we will use Python and Micro-Python.
Each student will build an embedded device that reacts intelligently to its environment and to interactions. The system will be centered on an ‘Arduino Nano RP2040 Connect’ or a Raspberry Pi. Students will add sensors and actuators. Ideas include a device that makes a happy sound when it meets a person it has seen before, a ball that lights up when you move it very gently in a secret pattern, and a smart tag that learns its usual environment and will send a message when it is taken away. Every participant decides on their connected devices and defines their project in dialog with the lecturers.

Students will learn how to design, implement, and test intelligent IoT systems using ML. This includes:

Choosing an embedded platform for an intelligent system,
Defining a system architecture for an intelligent system based on given requirements,
Designing and conducting sensor-based data acquisition for IoT devices for ML,
Implementing an ML model and running it on an embedded device using Python,
An overview of different ML libraries and frameworks,
Understanding trade-offs in the design of resource-constrained ML systems.

Our primary target audience are doctoral students and master students with a background in computer science, data science or related subjects. Basic skills in programming are required to participate in this workshop. BSc students, postdocs, and professionals wanting to deepen their skills and knowledge in ML and intelligent systems are also welcome.

Instructors:

Professor Albrect Schmidt, LMU Munich, Germany
Assistant Professor Sven Mayer, LMU Munich, Germany.

Albrect Schmidt is a full professor of computer science at Ludwig-Maximilians-Universität (LMU) in Munich. His research and teaching interests include intelligent user interfaces, interactive systems, ubiquitous computing, multimodal user interfaces, digital media technologies, and media informatics. He studied computer science at the University of Ulm and Manchester Metropolitan University. In 2003, he earned a PhD in computer science from Lancaster University with a focus on “Ubiquitous Computing – Computing in Context.” In the past 25 years, he has published more than 400 papers, with over 29,000 citations of his work. Over 30 students have completed their PhD under his supervision and mentorship, and many of them are now professors themselves. Albrecht served as a technical program co-chair for ACM CHI in 2014, as a conference co-chair for ACM UIST in 2018, and as conference co-chair for ACM CHI 2023 in Hamburg. He is also a member of the editorial board for the ACM TOCHI journal and co-founder of the ACM conferences TEI and Automotive User Interfaces. In 2018, he was inducted into the ACM SIGCHI Academy and, in 2020, he was elected to the German academy of science, Leopoldina. Furthermore, he co-founded an IoT company in 2018 and is enthusiastic about exploring how AI and ML are transforming software and systems engineering.
Website

Sven Mayer is an assistant professor (Jun.-Prof.) of computer science at LMU Munich (Germany). His research sits at the intersection between Human-Computer Interaction and Artificial Intelligence, where he focuses on the next generation of computing systems. He uses artificial intelligence to design, build, and evaluate future human-centered interfaces. In particular, he envisions enabling humans to outperform their performance in collaboration with the machine. His research directly translates into this service work, for instance, in 2023, he served as the General Chair of the Conference on Hybrid Human-Artificial Intelligence. He is also a member of the editorial board for the ACM TOCHI journal and he is a ACM CHI Steering Committee member.He focuses on areas such as augmented and virtual reality, mobile scenarios, and robotics. Before his faculty appointment at LMU in 2020, he was a postdoctoral researcher at Carnegie Mellon University (2019 – 2020) and received his doctoral degree in 2019 at the University of Stuttgart.
Website

Workshop B: CARE-FULL! MORE-THAN-HUMAN URBAN FUTURES

Maximum number of participants to be enrolled to the workshop: 20

With the increasing awareness of the detrimental effects of anthropocentrism, interests in how we might understand and engage with the more-than-human have been growing substantially across various fields of research and practice, including interaction design. This workshop will explore different approaches to interaction design that can help us move away from human-centric views particularly evident in the current smart cities and urban informatics discourses. Recognising our complex entanglements with planetary ecosystems and in search of justice for humans and other-than-humans, we will work with and across diverse disciplines such as HCI, STS, geography, critical data studies, planning, and design, to reconsider concepts and practices that can help us re-imagine more-than-human futures for urban environments.

Topics to be covered will include:

Critiques of and alternative approaches to smart cities, digital twins, urban informatics;
More-than-human design;
Care and co-creation;
Multispecies justice.

The workshop will include lectorials, hands-on studio sessions, and a dérive (drift) outside, in parallel with the development of group projects, and will end with final presentations. Participants will engage in a hands-on project that leads to tangible results, aligning with the workshop’s focus on co-creative practices for cohabitation and more-than-human urban futures. Students will form small interdisciplinary teams and work on questions and challenges related to the workshop’s theme. Examples of possible project directions include but certainly not limited to the following:

Ways to listen to an/or amplify the different voices of urban forests;
How matters of inclusivity and bias can be addressed in urban governance context, through the lens of care;
Designing a digital twin for an other-than-human habitat such as a beehive, considering ecological and technological interplay;
Creative exploration of human and multispecies subjectivities in relation to generative Artificial Intelligence;
Augmented or immersive media that can help draw attention to the – often hidden – human-induced damages and threats to multispecies lives in urban environments.

Prospective learning outcomes include:

Understanding of core concepts, theories and approaches related to more-than-human design;
Practical experience through co-creative experimentation with these concepts;
Collaborating with and receiving feedback from participants with diverse professional and cultural knowledge, practices, and experiences;
Ability to critically engage with more-than-human perspectives with a focus on urban interaction design.

Preferred student profile:

Interest or experience in design, transdisciplinary, and creative practice-based research;
A passion for exploring new and radical approaches to urban challenges;
Optional: Programming skills for those engaging in any technical prototype development.

Instructors:

Prof. Marcus Foth, Queensland University of Technology, Australia
Associate Professor Jaz Hee-jeong Choi, Amsterdam University of Applied Sciences, Netherlands
Dr. Ekaterina Gilman, University of Oulu, Finland

Marcus Foth is a Professor of Urban Informatics in the School of Design and a Chief Investigator in the QUT Digital Media Research Centre (DMRC), Faculty of Creative Industries, Education, and Social Justice, Queensland University of Technology, Brisbane, Australia. For more than two decades, Marcus has led ubiquitous computing and interaction design research into interactive digital media, screen, mobile and smart city applications. Marcus founded the Urban Informatics Research Lab in 2006 and the QUT Design Lab in 2016. He is a member of the QUT More-than-Human Futures research group. Marcus has published more than 280 peer-reviewed publications. He is a Fellow of the Australian Computer Society and the Queensland Academy of Arts and Sciences, a Distinguished Member of the international Association for Computing Machinery (ACM), and currently serves on Australia’s national College of Experts. In his spare time Marcus is a wombassador and hobby beekeeper.
Website

Jaz Hee-jeong Choi is an Associate Professor in Civic Interaction Design at the Amsterdam University of Applied Sciences. Their transdisciplinary research and practise situate ‘care’ at the core of transformational encounters in different settings ranging from cities as complex cyberphysical networks to forests as moving creatures. They build on this to explore how radical transformation can materialise care-fully through creative-critical engagements. Jaz’s work is often experimental, multisensory, playful, and participatory, and starts from the margins to understand and create just futures. Their current research, practice, and engagement focus on the dynamics of creative practice as feral care: they explore 1) its potential to arouse societal transformation in different cultural and more-than-human contexts, and 2) experimental and co-creative ways to form relational spaces to make sense of indeterminacy, plurality, and entanglements towards change. Jaz loves spicy food.
Website

Ekaterina Gilman Ekaterina Gilman, D.Sc. (Tech.), is a postdoctoral researcher supported by the Academy of Finland at the Center for Ubiquitous Computing. Her research interests include data analytics, context modelling and reasoning, and machine learning in ubiquitous computing, IoT, and data-intensive systems. She has also worked as a visiting researcher in Northern Finland Biobank Borealis and Centre for Health and Technology, Oulu, Finland. She has been involved as PI and researcher in several research projects funded by European Regional Development Fund, Business Finland, Academy of Finland, EU Horizon 2020 research and innovation programme. Ekaterina actively participates in the research community as a reviewer for international journals, organiser and program committee member of scientific workshops and conferences. Ekaterina likes playing piano.
Website

Workshop C: DESIGNING TANGIBLE AI INTERACTIONS

Maximum number of participants to be enrolled to the workshop: 20

Artificial intelligence (AI) is increasingly integrated into our daily lives, and it is important to design interactions between humans and AI that are intuitive, meaningful, and experiential. The emerging research field and development trend of human-centered AI aims at providing concepts and methods for making AI understandable, while maintaining human oversight. This workshop will investigate how AI can be seamlessly and effectively incorporated into physical objects and environments.
Workshop participants will gain insights into creating and investigating human-AI interactions that utilise the concepts from:
• models of human-AI interaction,
• embodied and multimodal interactions,
• tangible user interfaces and tangible explainable AI,
• AI applications as socio-technical systems.

The aim is to understand how AI systems in people’s everyday lives may offer users more intuitive experiences in human-AI interactions. The workshop will also contain critical and multi-perspective reflections to the concepts and solutions.

The workshop consists of a mixture of lectures, reading and discussion sessions, demonstrations, and collaborative hands-on design and prototyping activities. Each day of the workshop will have a variety of these learning modes so it will never get boring!

This hands-on group work project focuses on a tangible AI interaction concept selected from the instructors’ proposals or invented by the groups themselves. This project work will produce a design of an AI system or application concept that people (users) may interact with through a tangible device/object. Such a concept could combine a tangible device prototype design with other modalities such as conversational interaction (e.g. using an LLM). Thematically the project works concepts could address, for example:
• AI assistants for personal wellness,
• smart home systems,
• educational AI-powered devices for children,
• public user interfaces to AI-driven systems in the urban environment.

A “minimum viable user study” will be required to support the design and/or evaluation of the concept. The students will present the concept, the user study results, and their critical reflections of the outcome in the result seminar in the last day of the workshop.

Our primary target audience are doctoral students who want to learn about the design of tangible forms of AI. However, also MSc and BSc candidates are welcome to apply, likewise postdocs, and industry and public sector professionals wanting to deepen their skills and knowledge in tangible AI interactions. We welcome students with various disciplinary backgrounds, such as Human-Computer Interaction, human-centered AI, design, machine learning, and related fields to join and have fun while doing and learning.

Instructors:

Professor Kaisa Väänänen, Tampere University, Finland
Professor Jonna Häkkilä, University of Lapland, Finland
Dr. Ashley Colley, University of Lapland, Finland

Kaisa Väänänen is a full professor of user experience (Human-Computer Interaction, software engineering) in Tampere University, Finland. Kaisa leads the research group of Human-Centered Technology (IHTE) in the unit of Computing Sciences. She has extensive teaching and supervision experience as well as leadership of study programmes. Kaisa has 25 years of experience in research related to human- computer interaction both in university and industry. In 1995-2004, she worked at Nokia Inc, in leading positions of user needs research and strategic consumer insights. Kaisa’s research interests cover user experience and human-centered design, with emphasis on design research of digital services for advancing sustainability and human-centered AI. Kaisa is an author of 180+ peer-reviewed academic publications. She is very active in the international research community, and frequently takes part in organizing conferences related to user experience and human-computer interaction, such as the flagship conference of Human-Computer Interaction, ACM CHI 2023.
Website

Jonna Häkkilä is professor for Industrial Design at the University of Lapland, Finland (2014-), and docent in Human Computer Interaction (HCI) at the University of Oulu, Finland. She leads the user experience research group LUX at U. Lapland. Previously, she was Director of User Experience (UX) at the Center for Internet Excellence U. Oulu (2012-2014), and research leader at Nokia Research Center (2007-2011). She received her PhD degree in computer science from the University of Oulu, Finland, in 2007. She has published 150+ peer reviewed scientific papers on HCI, focusing on mobile and ubiquitous computing and user centric design. She has received research grants as PI from Horizon 2020, Academy of Finland, Business Finland, and Interreg. She works at the cross-section of technology and design, and her group’s works have been awarded at ISWC and exhibited at Milan Design week and Wanted Design, New York. Her current research interests include wearable technologies, unobtrusive interaction with technology, and in utilizing design methods for creating and assessing future technology visions.
Website

Ashley Colley is a university researcher at the University of Lapland. He has more than 25 years industry and research experience, e.g. 10 years as a Principal Designer at Nokia, and as co-founder of the successful health tech. start-up Oura Health (www.ouraring.com). He is the inventor of more than 30 patents/patent applications. He is an active member of the international Human Computer Interaction (HCI) research community and has published over 100 peer-reviewed publications.
Website

Workshop D: VIRTUALLY REAL? THE ART AND SCIENCE OF DESIGNING IMPACTFUL (OR EVEN TRANSFORMATIVE?) VIRTUAL EXPERIENCES

Maximum number of participants to be enrolled to the workshop: 20

Are you ready to dive into the realm of Virtual Reality (VR) and unlock its full potential? Join us for a journey where you’ll harness VR technology, including state-of-the-art head-mounted displays and the Unity game engine, to craft VR experiences that resonate and leave a lasting impact. This workshop is more than just learning the tools; it’s about connecting with global peers in a project-based learning environment to ideate, create, and present VR experiences that push boundaries. Prepare to document your process, share your insights, and unveil your creations in a showcase that celebrates innovation and collaboration.

The virtual reality landscape is evolving rapidly, with advancements in technology making VR more accessible, affordable, and powerful than ever before. However, the true art of VR lies beyond just the technology. It requires a deep understanding of the user – their perception, cognition, emotions, and behavior within a computer-simulated environment. This workshop is designed to bridge the gap between technology and user experience, focusing on how to design VR experiences that are not just technologically sound but also psychologically impactful and emotionally resonant. We will explore the intricate balance between artistic creativity and scientific principles, aiming to create experiences that are transformative, not just in how they look or feel, but in how they connect with and affect the user. Join us to explore the frontiers of VR, where technology meets human experience, and where your creations can leave a lasting imprint on the virtual landscape.

This workshop offers a comprehensive exploration of the various facets necessary to design VR experiences that are not only impactful but potentially transformative. The core of the workshop revolves around project-based learning where participants will form diverse teams to conceptualize, develop, and refine a VR prototype. Our multifaceted approach intertwines theoretical lectures with practical demonstrations and hands-on activities.

Through this comprehensive approach, participants will not only develop technical prowess in VR development but also cultivate a critical understanding of how to create VR experiences that are meaningful, impactful, and potentially transformative. The main learning objectives of this workshop are multifaceted:

Understanding and effectively using core VR concepts and technologies;
Gaining familiarity with current and future applications of VR;
Developing a nuanced understanding of how to (not) design for VR users (also known as “immersants”), focusing on creating immersive and impactful experiences;
Engaging in an agile VR development process, fostering an environment of project-based learning and teamwork.

Reflecting on and applying theoretical foundations such as transformative experience design frameworks, immersion/presence frameworks, and the reality-virtuality continuum.
As designing impactful VR experiences utilizes knowledge and skills from a variety of disciplines such as (but not limited to) computer science, psychology/cognitive science, human factors/HCI, user experience design, philosophy, new media, arts, and game design etc., we invite students from any of those and other disciplines that could contribute to designing impactful experiences to apply, and aim to create teams with complementary skill levels to maximize your learning, team experience, and impact of the VR project. We would like to note that this is an intensive and hands-on workshop requiring ideation and development of a VR project, which has many technical components. To ensure your success in the workshop and that everyone can support their project team, it is essential that each student comes in with sufficient technical background, specifically with at least some basic knowledge of the VR/game engine Unity we’ll be using.

Instructors:

Professor Bernhard Riecke, Simon Fraser University, Canada
Dr. Matti Pouke, University of Oulu, Finland
Dr. Paula Alavesa, University of Oulu, Finland

Bernhard Riecke is full professor at the School of Interactive Arts and Technology at Simon Fraser University where he directs the iSpace Lab (an acronym for immersive Spatial Perception Action/Art Cognition and Embodiment). He joined SFU in 2008, after researching for a decade in the Virtual Reality Group of the Max Planck Institute for Biological Cybernetics in Tübingen, Germany and working as a post-doctoral researcher at the Max Planck Institute, Vanderbilt University, and UC Santa Barbara. His key research areas include human spatial cognition/orientation/updating/navigation; Enabling robust and effortless spatial orientation in VR and telepresence; Self-motion perception, illusions (“vection”), interfaces, and simulation; and investigating and designing for transformative positive experiences in VR which he touches on in his TEDx talk “Could Virtual Reality make us more human?”.
Website

Matti Pouke is a postdoctoral researcher in the Perception Engineering research group at the Center for Ubiquitous Computing, University of Oulu, Finland. In his doctoral research he studied the transformation and visualization of real human activity in virtual environments. Since then he has been involved in many research and development projects involving the use of virtual environments in public urban contexts, including smart buildings, architectural visualizations, and libraries. His current research focuses on human perception in multi-scale virtual reality experiences as well as Presence and Plausibility. He is a member of ACM and IEEE.
Website

Paula Alavesa is a university lecturer in the Perception Engineering research and Cyber Security and Informatics (CSI) groups at the Center for Ubiquitous Computing, University of Oulu, Finland. Her doctoral thesis (2018) focused on playful appropriations of hybrid spaces where virtual and physical environments were combined into urban pervasive games. She has several years of experience in applied constructive research with location-based, augmented, and virtual reality technologies. Her current research continues the overarching theme of combining digital and physical realms using XR technologies and games or gamification approach as the mediator. She is a member of IEEE, ACM, SIGCHI and Digital Games Research Association (DiGRA).
Website