PULSE VOL.4 (May 2023)
The Pomposa Project: E-wearables
- Art & Tech Spark Curiosity in Everyone -
- Yui Shikakura & Enrico Bertelli
Abstract
The Pomposa E-Wearables project was carried out at PGVIM in August 2022, and it involved seven workshops and performances aimed at inspiring creativity in young people, disabled youth, university students, and elderly people. The STEAM approach was applied in the project to encourage everyone’s creativity.
One of the project’s main objectives was to allow the participants to prioritize the design of the wearable structure over its codes, and this was achieved by creating most of the code through discussions with the students. Additionally, the participants were given the opportunity to learn how the devices worked in order to understand the technology better. Although they did not code their wearables, it was crucial for them to comprehend how the technology functioned.
The limitations of the technology, such as not crossing over conductive threads, served as a catalyst for creativity, and the participants were able to create a variety of musical instruments, showcasing their expertise in decorative arts. During the showcase on the last day, the participants gave a short performance, highlighting their creativity and learning.
The project fostered a positive attitude towards learning new things, promoting transformative competencies, especially “ Creating new value.” The process of learning and creation developed critical thinking and creativity, which helped the students find different approaches to solving problems and collaborate with others to find solutions to complex issues.
The Pomposa project enabled everyone to be creative, leverage their artistic passion, and discover sides of music-making that were unexpected and enjoyable. The project broke stereotypes and allowed the participants to collaborate, tackle challenges, and share their ideas with others. The STEAM approach made creation a fun learning experience, and the attitude to celebrate new ideas and transform them into their own became an important competence for living in an unforeseen future.
Introduction
What if you could compose a song just by holding a squid-shaped plushie, play an arpeggio by tilting your head back and forth, around and around ? Throughout August 2022, we delivered the Pomposa E-Wearables project at PGVIM in Bangkok, Thailand, making these unique music-making moments a reality. All of the instruments came to be on the very day, and the designers we invited had no prior experience with such technology. This is just one of the possibilities unlocked by the STEAM approach, where music is on the driving seat of STEM skill development. Pomposa allowed us to bring together artists, disabled youth, the elderly, and university students, through a unique pathway between technology, creativity and performance.
In this paper, we set out to analyse Pomposa’s key learnings and share best practices in the use of technology for people with disabilities, for others to build upon.
The students and participants with their works
Background
The STEAM approach was introduced by Yakman (2008) [1] as a mixture of Science, Technology, Engineering, and Mathematics (STEM), approached via the Arts (A). Students’ imagination is fostered through art-making, sparking curiosity and creativity, all of which boost STEM skill development. The gamified outcome entices students to learn proactively; they are able to connect new knowledge to their personal experiences via this organic, cross-disciplinary approach.
Research led by the OECD [2] (1997-2003) aimed at defining and selecting the key competencies for the future of society and education (DeSeCo). As a result, The Education for 2030 was published in 2016 [3].
The publication aimed to steer schools in the formative steps for students who are to become young adults in the 2030s. The goal is to foresee skills needed for jobs which are yet to be created, technologies not yet invented, and to solve problems that are yet to be formulated. The publication identifies the “transformative competencies” and underlines the importance for young people to be innovative, responsible and aware:
- Creating new value
- Reconciling tensions and dilemmas
- Taking responsibility
“To navigate through such uncertainty, students will need to develop curiosity, imagination, resilience and self-regulation; they will need to respect and appreciate the ideas, perspectives and values of others; and they will need to cope with failure and rejection and move forward in the face of adversity.” [4]
In England, we have led Conductive Music CIC (or CM), a team of educational specialists delivering music-led STEAM workshops to children from challenging backgrounds since 2012. We set out to inspire children to create digital instruments and find their musical voice through composition, collaboration, and performance in a supportive all-ability-friendly learning environment.
Evaluation
CM’s critical evaluation and monitoring systems are key to our continued success and growth, strengthening good practices and highlighting areas for development. We provide a safe and stimulating learning environment for children from challenging backgrounds, fostering their abilities and celebrating their individual experiences.
Four of our staff members hold PhDs in Music and Education and are experts in quantitative and qualitative analysis. Our work follows creative evaluation tasks generated by music leaders, project leaders’ reflections and group evaluations. Perspectives from participants are gathered in online forms, with full anonymised logs of student attendance to monitor engagement. Data and analytic insights are gathered from Social Media advertising and our websites. We regularly host PhD researchers from the Digital Humanities Department of Canberra University, Australia, for data and trend analysis.
Our data collection forms utilise a combination of summative and formative assessment methods to ensure that participants from challenging backgrounds effectively absorb and comprehend the content. These forms adhere to access and inclusion principles, accommodating English as Additional Language students and disabled students.
Since January 2021, we have collected 22,223 feedback forms from students, teachers, and other educational specialists who observed our sessions. We are proud to keep a high level of enjoyment and content appreciation, with students grading us 9/10 on average, and teachers confirming our tutors’ competence at 9.7/10. We firmly believe that our projects can develop skills and boost the resilience necessary to adapt to and survive the rapidly changing world surrounding our students.
The Pomposa project
Pomposa was designed based on CM’s existing expertise in delivering in- person STEAM experiences for disabled students. We strove to provide participants with practical skills in music-making, technology, and wearable design. The sessions were delivered by two artists from CM, accompanied by Nattana Chansawas (ILoveNoName), as a design and stylist consultant. We delivered six workshops of different length and focus, all of which culminated in the final performance, part of the PGVIS symposium, which took place in August 2022.
Below, a timeline of the events taking place in August 2022, during which our participants designed, coded, created and showcased their wearable musical instruments
Timeline
Please note: the following paragraphs focus on activities 1, 6, and 7
E-Wearables, a brief introduction
E-Wearables made their debut in the maker labs with Arduino’s Lilypads, which became commercially available in 2007. Since then, numerous companies produced hardware and add-ons, but we opted to use Adafruit’s Circuit Playground Express (CPE).
The two main factors for this choice are:
- The richness of embedded sensors: accelerometer, thermometer, microphone, speaker, RGB sensor, among others.
- The ease of use: while the LilyPad required a knowledge of the Arduino IDE and its language, the CPE uses a block-based coding interface, which is very child friendly and can be easily operated by complete beginners.
A typical e-Wearable instrument consists of a main piece of hardware, like the Circuit Playground Express, which can be stitched to a garment using standard thread, and connected via linear circuits to a variety of components by using conductive thread, consisting of woven metal fibres.
Once the garment is set up and tested, one can use Microsoft Makecode with block-based coding, or opt for JAVAscript or Python. The numerous built-in sensors allow makers to measure light, sound, temperature and acceleration, among others. Once these numeric data are collected, they can easily be transformed into sounds via direct correlation (the brighter the light, the higher the pitch) or more complex action-reaction relationships. For example, an arpeggiator may change its chord based on the orientation of the device: Cmaj if tilted forward, Fmaj to the left, D7 backwards, and G7 when tilted right.
Musical events can also be triggered by switches, such as buttons made of conductive fabric, or pressure sensors designed with velostat [5] and felt. Alternatively, one may create generative compositions triggered by thresholds or gates, such as a change in tempo connected to a jump, or an alternation of the rhythmic pattern determined by the cardinal direction chosen by the performer, coded through the built-in compass.
Microsoft MakeCode Circuit Playground Express [6]
The Digital Audio Workstation: Bandlab.com
We chose Bandlab.com as our digital audio workstation (DAW) because of its ease of use, stability, browser- and app-based nature, which enable all users to access remotely and collaborate in real time, as well as to continue working from the comfort of their home, after project completion. MIDI functionality and microphone accessibility is also included, but we were mostly interested in ASCII support, where keyboard keystrokes - programmable via the Circuit Playground Express - can be transformed into sound.
Letters on the keyboard (Bandlab.com)
At the simplest level, the students can code circuit playground to press the letters to play music on Bandlab. Both Microsoft MakeCode and Bandlab are free browser-based softwares. Therefore the participants can access them from anywhere, and they can modify them from their homes whenever they want. This accessibility is one of the most important factors in software selection. They do not require any extra installation on a PC or Mac; both software usages are adaptable to other similar DAW or coding platforms. Even if participants don’t have the hardware at home, they can keep creating by adapting the skills after the workshops.
Materials:
The students need to use different types of thread and fabric according to purpose. We prepared various sizes of colourful felts, plastic buttons, and cotton threads for the main structures. To create inputs for notes, they need to use conductive threads, conductive fabrics and tapes. We prepared Neopxel LED for additional light decoration according to the students’ design ideas, which could be triggered by sensors.
Activity 1: Wearable instruments for everybody
This workshop was delivered over three days (3rd, 4th, and 5th of August, 10 am - 3 pm). Lessons were delivered in English. The participants were students and lecturers from Silpakorn University’s faculty of Decorative Arts and Chulalongkorn University. Two CM’s artists and Ms Chansawas led all sessions. This workshop aimed to design accessible wearable instruments for disabled young people.
Day 1: Prototype and Coding
Morning: The participants learned how the circuit playground works and explored materials as conductors and insulators. They then stitched together five note prototypes and tested them.
Afternoon: The participants stitched Extension Neopixel LED and tested them. They engaged in a basic coding activity to give voice and light to their instrument and to understand the sensors.
The participants thoroughly enjoyed creating prototypes, each with their own specific design ideas to develop. They applied their skills and knowledge to explore the potential of the products they created. For example, in video sample 1, the light was triggered by the loudness of the sound.
For most of the participants, it was their very first time coding. They found the basic coding activity somewhat challenging, not only because it was their first time, but also because it was introduced at the end of a long day, and it was conducted in English.”
Prototype with five notes input & LED
Day 2: Design
Morning: All participants developed their wearables based on the prototype structure (5 touch notes and minimum of 1 LED). We reminded them that the targeted users would be blind children. A student from Chulalongkorn University (Student A) joined and made his prototype.
Afternoon: Participants started thinking about how their instruments should work. We prepared the codes for each instrument by following their idea. Most of the main structure was completed by the end of the day. Participants shared design ideas in a small showcase.
Participants were trying an error for design to find the solution. Some switched from 3D design ideas to 2D. They designed the shape and simultaneously thought about what features they wanted to code (e.g. Superhero cape). A small showcase allowed all participants to reflect on their designs.
Students starting creating their instruments.
Day 3: Test and troubleshoot, showcase
Morning: Finalising. Stitching completed by the morning.
Started testing instruments. Modified and fixed the design to avoid short circuits.
Afternoon: Using Bandlab: participants selected the voice for their instruments and composed a short song. We had a short performance at the end of the session to share each other’s work.
Short circuits are a common trouble for wearables design. Some of the participants needed to re-do the stitching of five inputs. Student A designed a hairband to focus on the music by coding.
Round up
One of the key features of this workshop was ‘making a product for the user’. Especially because our user target was disabled children, participants needed to be more aware of the texture and touch of the materials, which had to be easy to wear and play with. They designed the instruments not only as a creative endeavour but also for someone they had never met to use. Most participants were students of the decorative art faculty, so they were familiar with designing and stitching. However, working with a circuit was not so straightforward for them.
Conversely, student A who joined on day two was more familiar with coding. He dedicated more time to coding than designing his instrument.
Having the time to create a prototype on day one gave the participants an understanding of what is possible, and what could be the challenge for their creation. We placed great importance on assimilation time of new skills and knowledge.
Activity 1 participants & their works
Activity 6: E-plushies for the performance
Six BA students, two MA students, and a professor from Chulalongkorn University joined the session with her children. Most of the students worked in pairs or teams. This workshop aimed to design plushies musical instruments for the final performance at the conference. All three tutors delivered the workshops over two days.
On day one, in the morning, the participants used Bandlab and played sound with the Mac keyboard to familiarise themselves with it and find their favourite sounds. They used Circuit playground prototypes from Activity 1 to learn how they worked.
In the afternoon, they started to code with D minor Pentatonic. In parallel, they designed and developed performance ideas.
Activity 6: Day 1: E-plushie Jonathan the squid
In this activity, the students worked in pairs to develop their instruments and performance.The roles were clearly defined: One worked on the design, and the other developed the instrument’s code.
On day two of the workshop, the students started testing instruments and troubleshooting. Two groups developed stories for the performances.
Jonathan the squid was the video game character. Every time it rotated, arpeggio was triggered. This group developed a controller-shaped instrument which madevideo game sound effects.
This group developed a face hugger-shaped instrument.
They created a short acting performance about a futuristic coffee machine in honour of the vending machine in PGVIM.
Feedback
The average enjoyment of the workshops was 9.3/10.
Below are some of the feedbacks from our participants:
“I love the energy of the teacher (always encourage student to play around and have fun). The material that the workshop provide is more than enough. The activity is challenging and leave participant with a lot to think and to work during the class. I really really love this class. I will definitely apply things I learn to develop my class for sure!! Thank you with all my heart!”
“Really fun and informative. Especially for those who hasn’t experience coding before. The sound makes it much more fun than enacting virtual object in the monitor. It’s good to bridge the physical and virtual world together!”
“This is very new to me. Changed my perspective on how music can be made and how it can be performed. We were making electronic devices but it felt like creating a craft work. Thank you.”
“Meet people who I didn’t know before and have worked togeter know about I don’t know anything about, so good.”
“It was a very exciting new experience. I usually take netry and general music classes. But when I tried the workshop, I did something very unique and fun.”
Conclusion
In activity 1, we let the participants prioritise the design of the wearable structure over its codes. We created most of the code by discussing with the students what they wanted to create. At the same time, we also made time for them to learn how the devices worked. Even though they didn’t code their wearables, we thought it was important for them to understand how the technology worked. To avoid short circuits, the conductive threads should not cross over each other. Envisaging how their instruments produced sounds led them to deciding which input should be on which side of the wearable. Such limitations of the technology can be a catalyst for their creativity. Enforcing their expertise (decorative arts) through an inspiring new technology led them to create a variety of musical instruments. At the showcase on the last day, the participants gave a short performance.
Activity 6 was designed for the BA and MA students to work in a team to create a short performance with their creation. The students shared their roles and developed performance ideas freely. Through the creation process, the students’ creativity got unleashed by the idea of “making a musical instrument”.
Even though we introduced a technology to allow the creation of innovative products, sometimes the idea of having to make a musical instrument could limit the students’ adventurous creativity. As we can see from the feedback, creating an instrument from scratch inspired the students to explore their performance possibilities freely. Our intention was not for them to play Mozart or Beethoven with the instruments they created, but to make something new and to enjoy making it. This process of learning and creation fosters a positive attitude towards learning new things. Troubleshooting grows resilience. We strongly believe such learning experiences respond to transformative competencies, especially “Creating new value”.
“Creating new value requires critical thinking and creativity in finding different approaches to solving problems and collaboration with others to find solutions to complex problems. In evaluating whether their solutions work or not, students may need agility in trying out new ideas and may need to be able to manage risks associated with these new ideas.” [7]
Breaking stereotypes is not easy. Through the Pomposa project, the participants and the students collaboratively learned and created new instruments and performances; They tackled challenges together with peers; they shared their ideas with other groups at the showcases. The attitude to celebrate new ideas and transform them into their own is an important competence to live in an unforeseen future. With the STEAM approach, creation becomes a fun learning experience. There is no right way to play music with a brand new instrument you just created. The Pomposa project enables everyone to be creative, to leverage their artistic passion, and discover sides of music-making that were as unexpected as enjoyable.
[1] Yakman, G. (2008). STEAM education: An overview of creating a model of integrative education. In Pupils’attitudes towards technology (PATT-19) conference: Research on technology, innovation, design & engineering teaching, Salt Lake City, Utah, USA.
[2] The Organisation for Economic Co-operation and Development
[3] OECD(2016) “Conceptual framework key competencies 2030”
[4] OECD(2018) “The Future of Education and skills 2030”
[5] Velostat is a thin electrical bagging material known for its piezoresistive properties. Its unique characteristics make it a desirable choice for in-socket pressure sensing applications.
[6] Adafruit website (https://www.google.com/url?q=https://learn.adafruit.com/46973&sa =D&source=docs&ust=1675961567342761&usg=AOvVaw0yrqM6NwgCNAuTXzzxoNUJ)
References:
Department for Education (2021). Model music curriculum: Key stage 1 to 3. Department for Education (2014). National curriculum in England. Leighton, S., & Mitchell, P. (2016). A new STEAM age challenging the STEAM agenda in
research. The Culture Capital Exchange.
OECD(2016) “Conceptual framework key competencies 2030”(https://www.oecd.
org/education/2030-project/about/documents/E2030-CONCEPTUAL-
FRAMEWORK-KEY-COMPETENCIES-FOR-2030.pdf)
OECD(2018) “The Future of Education and Skills 2030”(
OECD(2019) Future of Education and Skills 2030 Concept Note
Yakman, G. (2008). STEAM education: An overview of creating a model
of integrative education. In Pupils’ attitudes towards technology (PATT-19) conference: Research on technology, innovation, design & engineering teaching, Salt Lake City, Utah, USA.