Jaana Okulov

2013-2018

An interactive art installation, an emotional artificial intelligence logic, and a research project.

Olento Collective members: Jani Hietanen, Mira Hyvönen, Jaana Okulov, Ismo Torvinen, Pat Nav, Mari Ljokkoi, Max Hannus, Collaborators: Katri Saarikivi and Valtteri Wikström from NEMO research group, Heikki Ketoharju, Samuli Kaipiainen, Anne Naukkarinen, Veera Hirvaskero.

The Olento Collective consists of a small team of artists and creative individuals dedicated to the development of Olento. It is our ambition to produce an artificial intelligence (Olento) capable of learning via a modicum of multi-modal information streams, with intent to deploy iterations of it in environments ranging from academic to artistic. Olento’s logic is based on aesthetic motion, reacting and investing into the dialogue of expressions generated during its encounters. Our hope is, that in time, any one iteration of Olento would then develop the ability to express itself based on the data and experiences it takes in, utilizing them in a variety of ways as it exists in virtual space. Olento in effect would then work to fluidly mirror our inner experience as it is expressed outward: intimacy and affection can be felt in a soft, caressing touch; belligerence and hostility in tense, sharp trajectories and vicious dynamism.

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

From a book chapter: Quantifying Qualia, Okulov 2023.

The Olento collective positions itself in the contemporary art feld but uses also scientifc methods to engage the area of interest. During the years 2013–2018, The Olento Collective finalized multiple installations that translated bodily and emotional data into colours, soundscapes, and digital forms, and gathered research data from around 600 people to inform Olento’s development. The Olento Collective’s work will demonstrate how artistic and scientifc research can be combined and how, through artistic research, information can be studied nonverbally.

In the years 2013–2016, the Olento Collective’s work concentrated on the affectivity of digital forms, first covering the linguistic descriptions of emotions in relation to aesthetics, proceeding into nonverbal comparisons between bodily sensations and 3D objects, and later moving to translations of facial expressions into 3D objects. In 2014, the collective completed a research questionnaire in an art exhibition entitled Reality Hunger. This exhibition was held in Fafa Gallery, Helsinki, and curated by Vesa-Pekka Rannikko. A total of 631 answers were gathered from 61 exhibition visitors about their interpretations of emotional qualities. The visitors were asked to define the basic emotions happiness, sadness, disgust, fear, surprise, and anger (Ekman, 1992) with aesthetic words. The words collected were open coded into 11 word-pairs that included opposing dynamics: small/large, thin/thick, short/long, light/heavy, horizontal/vertical, straight/curved, sharp/dull, symmetric/asymmetric, distinct/fuzzy, soft/hard, and open/closed. 

The results showed that basic emotions were described somewhat consistently, but artistically more relevant became how the bodily dynamics manifested themselves in the descriptions. The bodily dynamics were developed into a sculptural logic to create an artwork for the Young Artists Exhibition held in Kunsthalle Helsinki, 2015. The exhibition visitors were able to sculpt their current emotions from Olento’s digital mass with slide buttons changing or moving a 3D-form (in thickness–thinness, sharpness–roundness axis, and with buttons that lifted–sank, inflated–shrank, twisted–released, closed–opened, bent down–bent up the object), its colour, and its surface material. After sculpting, the visitors answered how their bodies felt in relation to the finished object. They could choose from different areas of the body that related to Nummenmaa Laboratory’s research on bodily maps of emotions (2014), after which an artificial agent (Neural network and the Boids) learning the embodiment organized the 3D objects into a virtual space where other visitors’ objects were also floating. Around 1000 objects were sculpted and drifting in the space with their distinct behaviours: some sought company, some avoided it, others did not care. The Olento Collective members noted how intuitively people were able to sculpt their emotions and how easily the bodily dynamics were made readable from the objects: thus, aesthetics seemed to have a very direct relationship to bodily sensations. (Olento Collective, 2015).

In 2016, the Olento Collective collaborated with the NEMO research group (which specializes in digitalization of emotions (Katri Saarikivi and Valtteri Wikström et al., 2016)) from the Department of Neurocognitive Sciences at the University of Helsinki to create an interactive installation for the Sideways and Pixelache Festivals in Helsinki. Olento was taught to associate digital forms with facial expressions by a person reacting to the changes in aesthetic dynamics of a digital object with facial gestures (open source computer vision and generative algorithm). After the algorithm was taught, the digital object tried to imitated the facial expressions with its form, surface material, and color; for example, when seeing a bright smile, it lifted its body, reached higher, grew sharper and glowed in yellow, still remaining abstract enough to avoid clear anthropomorphism. Olento also translated facial expressions of both a festival audience listening to a band in Sideways Festival and another audience looking at an Olento installation in Pixelache Festival into aesthetic dynamics of a digital object, which again influenced the soundscape. The Olento Collective attempted to triangulate affective research data nonverbally: from the music, played by a band, through an audience’s facial expressions into a soundscape of Olento, so that the qualia in live music would become explicit in Olento’s soundscape and the affective content comparable in sound and facial expression. Although the experiment did not enable systematic analysis in the end, a nonverbal research method was established. The way facial expressions were taught reactively in relation to digital aesthetics was a new approach for the collective in experimenting with machine learning.

In the years 2017–2018, the collective investigated the affective relationship between digitally drawn line, colour, bodily movement, and singing, with small research programs, and in an artistic manner with multimodal translation, whereby a singer and a dancer tried to develop a mutual affective language through improvisation. In these experiments, the collective noticed similarities between people’s behaviours: many of the expressions in relation to visual or auditory stimuli were individually distinctive, but certain patterns seemed to appear in people’s expression (Olento Collective, 2017). For example, the wellstudied multimodal correlation of pitch and brightness manifested itself in the research settings (Marks & Mulvenna, 2013), but other unstudied correlations also emerged; for example, a saturated lime-yellow colour was often said to be “the fastest” colour when colour was studied by reacting to it with a stylus pen (Olento Collective, 2017). 

For qualia research, the most relevant insights of the Olento Collective related to the attentional features in affective perception. The collective noticed how especially dynamic features in stimuli, such as changes in a sound or an image were strongly attended and most affectively expressed. Affective information, in Olento Collective’s research, seemed to be in relation to dynamic change (Olento Collective, 2018). The research programs seemed to work as a means to study in more detail how distinct modalities operated in expression. For example, colour often seemed to distract the felt affective consistency in comparison to another modality if the colour was interpreted to express a contradicting qualia in relation to the motion of a drawn line, so that a singer singing the qualia of the drawing explicitly noted that “there are too many affective stimuli to follow with singing” and also confused the machine expression so that it “just felt unintuitive” for the person observing the behaviour (Olento Collective, 2016, 2017). What remains to be studied in more detail is the overall cross-modal correlations between a wider range of modalities in order to map them into a coherent qualia space, where affecting features in different modalities would match the topology.