CODEX VIRTUALIS _

INTERSPECIFICS AT SETI X AI
PROJECT PROPOSAL

Multidimensional Cellular Automata using glsl for modeling

 

Codex Virtualis is an artistic research framework oriented towards the generation of an evolving taxonomic collection of hybrid bacterial-AI organisms. With a subtle echo to the endosymbiotic (a) theory, we propose a symbolic formulation of a style transfer machine learning environment as a host, in which to merge bacterial/archaea time-lapse microscopy footage along with multidimensional cellular automata computational models (b) as endosymbionts, all under the orchestration of an autonomous generative non-adversarial network architecture (c). We aim as a result, to encounter novel algorithmically-driven aesthetic representations, tagged with a unique morphotype and genotype like encoding, and articulated around a speculative narrative encompassing unconventional origins of life on earth and elsewhere.

The project stands on the idea of cooperation to expand on the concept of intelligence by including machine and non-human agents into its configuration. And intends to articulate new schemes for the social imaginary to picture life outside the planet, and to better appreciate life on earth.

How can complexity outburst from simple dynamical systems when represented using convolutional neural networks?

What forms can arise from training creative algorithms with pattern-forming bacteria in spatially explicit environments and extremophile morphology modeling?

Two possible outcomes for presenting this project, which will be determined by the Covid-19 situation are: a. A site-specific immersive visual installation. b. An interactive online web/mobile site/app. In both cases displaying the complete organism collection, extensive process description, and DIY tutorials for people to replicate in a citizen science manner.

a. https://en.wikipedia.org/wiki/Symbiogenesis b. https://arxiv.org/abs/1809.02942 c.https://openaccess.thecvf.com/content_CVPR_2020/papers/Aich_Non-Adversarial_Video_Synthesis_With_Learned_Priors_CVPR_2020_paper.pdf

Our motivation when applying for this residency resides in the possibility to incorporate our in-progress art & science project Speculative Communications [a], with research on extremophiles and complexity modeling studied at Seti. We are planning to create an image database by combining our collected visual data from bacterial colony growth with extremophile visual data and mathematical models of unicellular life forms collected at Seti, to pre-train a generative non-adversarial network, through style transfer interpolated from the biotic database and the self-organizing autonomous patterns into a continuous cellular automata. We are open for suggestions on other input sources to explore.
1. Map to represent the training model and the feedback loop created with the computer vision system.
We seek to activate a self-generative system, Artificial Intelligence, and algorithmic approximation for generating virtual organism models based on morphology and rule modeling. The system includes a sensorial element, via Computer Vision, which increases its ability to influence the training results by extracting data (morphology, mobility, etc) from each emerging generation. A feedback loop that might add on or remove rules from the cellular automata depending on the behavior of the Generative non -adversarial network. Stacking parameters into a continuous flow of integrative rules to stimulate evolution into the organism models Some of the essential research questions that we would like to tackle with this project are: how can we generate automated systems focused on the creation of new-to-nature organism models that inhabit the threshold between the biotic and the virtual domain?
2. Computer vision algorithms allow us to extract and encode visual, morphological and behavioral information. By following the correspondence between phenotype expression and the genotype-like encoding, we expect to direct some specific features of virtual organisms.
The novelty of this development resides in the cross-pollination of algorithms to stimulate a self-generative process and the possibility of influencing a generative non-adversarial network through the interaction between our computer vision (sensorial system) and the behavior of the cellular automata.
3. (a) bacterial behavior mosaic. (b) continuous cellular automata. (a) algorithmic thresholds for latent space.
We believe this system facilitates not only exploring emergencies between biotic and virtual but, over time, it can also be conditioned to create more specific experiments where we can address questions related to coevolutionary processes. As well as possible explorations on the environmental conditioning on the cellular automata to analyze interactions coming from extremophile emergencies. The three primordial elements of the full system are described in the following flowchart.

  1. The sources: from which we take characteristics biotic and virtual.
  2. The feedback: to retro control the cellular automata by integration of new rules.
  3. The virtual life generated: represented in a taxonomic way.
4 System model v.01.
In Interspecifics we have always admired the endeavour the institute has undertaken, as an independent organization, in exploring the origin and nature of life in the universe. We have been inspired by its non-anthropocentric model of thought, committed on ‘moving away from [our] equating life with human life and humans as being the pinnacle of evolution’ (a), into broadening our understanding of intelligence with the inclusion of other –existing and yet to be found– organisms, both outside and on this planet.

We have referenced Seti as an influence for our own artistic research on several occasions, such as in the creation of our fictional organization called Siti (Search for intraterrestrial intelligence) in which we studied bacterial patterns as a form of non-human communication, a project later called Speculative Communications (b). We understand Seti’s existence as a form of resistance, and as an important influence for future generations, outside and within the mainstream scientific community.

For this particular project we aim to gather data information from three specific, although not exclusively, lines of research:

  • Extremophiles and how ‘by studying conditions under which life can thrive, we hope to understand life elsewhere – where to look for it and what it could look like’  or simply to hypothesize if it would be possible for life to evolve and survive on other planetary bodies’ (c).
  • Complexity modelling and ‘how can the communication methods [and morphology] of other terrestrial species help us to recognize [and model] intelligence that might exist elsewhere in the cosmos’ (d).
  • Complex life in other planets and how ‘numerous articles have reported weird anomalies in the atmosphere of Mars, from an outburst of methane in June 2019 to patterns in oxygen concentrations that cannot be explained by any known, atmospheric or surface, processes on the Red Planet’ (e).
We hope this piece will contribute to Seti’s mission on knowledge dissemination of scientific discovery and public engagement

In terms of the on-sites residencies, only two of the members of the collective will travel to each place. The others will be working on a remote way to help accomplish the goals of the development. We normally divide our team into research areas, so we can work in parallel all processes.  Expenses are divided likewise and for this specific piece are almost concentrated in research, programming and web development.
4 week SETI residency:
  • Interviews with astrobiologists on extremophiles.
  • Interviews on possible SETI databases
  • Interviews on models and complexity
  • Start arrangements on GAN environment
  • Start arrangements of the Cellular Automatas
  • Programming Computer vision systems
  • Gather databases
6 week AI LAB residency:
  • Discussion of model with AI LAB team
  • Model automatization
  • Start training proces
  • Model adjustments
  • Start definition process for final outcome
  • Start programming for virtual environment
  • First stage result presentation
12 week work after the residency 
  • Finish model adjustments
  • Training continues
  • Programming model for classification
  • Finish programming of the virtual environment
  • Final result presentation either web format or physical installation.
Areas to develop:
  • GAN Environment: Python, Tensorflow, Cuda
  • Cellular automata: Python, Openframeworks, C++
  • ComputerVision: Python, OpenCV
  • Installation setup: Touch Designer, Python
  • Web: Three.js, D3.js P5.js
Production budget:
  • GAN Environment:  1,400 €
  • Cellular automata: 1,400 €
  • ComputerVision: 1,400 €
  • Installation setup: 1,400 €
  • Website js react: 1,400 €
TOTAL 7000 € ( The other 3000 € from the artist fee will be used to cover a second member expenses to travel to SETI and Ars Electronica)
Interspecifics is an Independent artistic research bureau founded in Mexico City in 2013. We have focused our research on the use of sound and A.I., to explore patterns emerging from biosignals and the morphology of different living organisms as a potential form of non-human communication. With this aim, we have developed a collection of experimental research and education tools we call Ontological Machines. Our work is deeply shaped by the Latin American context where precarity enables creative action and ancient technologies meet cutting-edge forms of production. Our current lines of research are shifting towards exploring the hard problem of consciousness and the close relationship between mind and matter, where magic appears to be fundamental. Sound remains our interface to the universe. 

www.interspecifics.cc

  Members:

Leslie García. Tijuana Mx, 1984. Former associate researcher, Nano Laboratory Nucleus, Fine Arts School, UFRJ, Rio de Janeiro. Former artistic researcher Media Environments department, Bauhaus University Weimar. Fellow of the National Fund for Culture and the Arts of Mexico. A former fellow of the DAAD artist in Berlin. She is an electronic sound artist and digital media developer. She explores the fusion process between art and technology, using techniques such as electronic prototyping, audio production, net.art, hardware development, and biological interfaces. She is a co-founder and artistic director of bio-art collective Interspecifics, founder of electronic media collective DreamAddictive, and co-founder of Astrovandalistas. http://lessnullvoid.cc/content/

Paloma López Ramírez. Guadalajara Mx, 1979. Graduated in Communication Sciences, MA in Cultural & Creative Entrepreneurship. Independent researcher founder of Interspecifics focused on bridging artistic exploration methodologies with experimental scientific practices. She has studied potential non-human forms of communication by transducing biosignals from bacteria and slime mold into sound and within the morphology of pattern-forming bacteria. A former fellow of the European network ECAS (European Cities for Advanced Sound), and Edgar Varèse 2022 chair professor at the Technical University of Berlin.

Emmanuel Anguiano Hernández. Puebla Mx, 1984. MSc by National Institute of Astrophysics, Optics and Electrónics. His work focuses on natural language processing, specifically on statistical language and communication modeling. Worked at Language Sciences Laboratory from 2010 to 2015 developing user profiling algorithms for social networks and publishing articles on specialized journals and conferences as Advances on Artificial Intelligence and Iberoamer- ican Society for Artificial Intelligence Conference. Currently part of Interspecifics, Art Science and Technology Collective, and Binocular Studio developing human computer interfaces, and interactive multimedia experi- ences. www.binocularstudio.xyz/u/emme

Felipe Rebolledo Carvajal. Santiago Ch, 1983. Graduated in Architecture. He has studied in Lighting and Industrial Design, the latter being the main focus of his professional work. Parallel to this, he is currently studying new forms of data visualization through the exploration of audiovisual pieces.He has been involved in various new media and art-sci- ence-technology projects, exhibitions, and residencies. He is currently part of the Interspecifics collective, based in Mexico City.

Carles Tardío Pi Barcelona Sp,  1985. Graduate in Physics, MsC in Cognitive Systems and Interactive Media and PhD in Music Tech- nology. He is actually focused in the study of non-chemcial communication mechanisms and pattern for- mation dynamics of bacterial colonies at Synthetic and Systems Biology Laboratory at Centro de Ciencias Genómicas and Instituto de Investigaciones en Matemáticas y Sistemas at UNAM. He is also a researcher within the Art, Science and Complexity Group at Centro de Ciencias de la Complejidad UNAM, where he ex- plores emergent phenomena properties within the sonic domains. He has been involved in several new media and art-science projects, exhibitions and residencies and he is actually co-leading the Laboratorio de Medios Inestables in Mexico City. (https://carlestapi.hotglue.me/)

Maro Pebo. Mexico City, 1987. PhD in Creative Media (HK) ,MA in Gender Studies (IT) and BA in Art History (MX). Specializes in the intersections of art, science, and biotechnology. Her current interest lies in microorganisms’ culture and a microbial posthuman turn. Senior lecturer in Moist Media at the DeTao Masters Academy SIVA Shanghai. Maro has presented her research internationally including at ISEA, EVA, ISCMA and Media Art Histories, co-curated the Open Systems salon, the HK Leonardo Art Science Rendezvous and was involved in the Mexican Pavilion of the 56th Venice Biennale. Currently working on fostering post-anthropocentric microbiology literacy in society.

MEMORIA ESFÉRICA [ 2019 ]

Instalación odoro-comestible que activa la memoria por medio de la exposición olfativa a tierra húmeda. La pieza sintetiza el petricor: el olor que desprende la tierra cuando la lluvia cae en suelo seco. En el proceso, ciertas plantas exudan un aceite absorbido por rocas arcillosas, que al entrar en contacto con la lluvia, es liberado junto con el compuesto geosmina—subproducto metabólico de ciertas actinobacterias—, la fuente del distintivo olor.

En un cuarto oscuro que minimiza la estimulación del resto de los sentidos, un aparato de destilación por arrastre con vapor desdobla los hidrolatos del petricor, que al contacto con el aire, se impregnan en un gel capaz de solidificarse. A su lado, una bioimpresora produce esferas de este material con el propósito de ser ingeridas por el espectador. Al momento de introducirse en la boca, la esfera se desintegra, libera el olor y viaja a través de la faringe hasta el bulbo olfativo, que tiene dos conexiones directas a con á reas del cerebro fuertemente implicadas con las emociones y la memoria, que además no se activan de forma tan inmediata con información visual, auditiva o táctil: la amígdala y el hipotálamo. Al finalizar la experiencia, se le pide a cada espectador compartir esa memoria que generó el aroma y se produce un registro vivo de la pieza.

Pieza comisionada por ivan edeza para la muestra ” sinestencia olfativa” CDMX

[ REPOSITORY ]  [ PHOTO LIBRARY ]



RELATED WORK:

TOPOLOGÍAS DEL DESEO
OPEN SENSORY SUBSTITUTION
DIMENSIONS
ACTION POTENTIAL

 

 

TOPOLOGIES OF DESIRE [ 2016 ]

Topologies of Desire is a technochamanistic performance created with brain readings from a group of therapist and shamans converted into three-dimensional disks. A laser system designed to scan them reads the topol- ogy on the data discs and converts them in to sound. The performance explores the psychoacoustic use of sound to provoke extrasensory experiences based on binaural rhythms and musical structures from Mexican indige- nous traditions.

Presented at INDEX MUAC (UNIVERSITARY MUSEUM OF COMTEMPORARY ART) ESPACIO X CCE ( CULTURAL CENTER OF SPAIN IN MEXICO) AND ALUMNOS47  MOBILE LIBRARY

[ PROJECT SITE ]  [ REPOSITORY ]  [ PHOTO LIBRARY ]



RELATED WORK:

ENTRAINMENT
OPEN SENSORY SUBSTITUTION
DIMENSIONS
ACTION POTENTIAL

DIMENSIONS [ 2015 ]

Dimensions is a brain activity sonification and visualisation system using topographic values from energy pointers. The main technique is programmed in OpenFrameworks, Supercollider and Puredata to map every electrode from an EEG headset and represent the dominant frequencies searching for possible power, phase or trigger correlations.

This system enables the user to be aware of the displacement of energies moving on the brain. One can notice correlation such as symmetry on electrodes of the same order, same frequency domains in all electrodes, or asymmetrical correlation on left and right side of the brain.

Second price at the Hack the Brain congress, WAAG SOCIETY, AMSTERDAM.

[ REPOSITORY ]



RELATED WORK:

TOPOLOGIES OF DESIRE
ENTRAINMENT
OPEN SENSORY SUBSTITUTION
ACTION POTENTIAL

OPEN SENSORY SUBSTITUTION [ 2015 ]

The term sensory substitution refers to a non-invasive technique introduced by neuroscientist Paul Bach-y-Rita in the 60’s who attends restoring senses damaged using hybrid interfaces brain-machine to redirect certain characteristics of a sensory modality to another and using the adaptive capacities of the brain to replace perceptual categories. He suggests that while the brain and its connection with the environment operate properly, peripheral sensory systems are secondary in the exercise of global cognition.

The aim of the prototype of sensory substitution is to allow the brain to create a mapping of space through sound and electrotáctiles sensations. And thus consider expanding human capabilities and challenge the Aristotelian conception that postulates five senses with specific functions clearly separated from each other.

In this project we are using OpenCV to detect blobs from a video camara attached to the Raspberry Pi, this detections is later tranform into sound using SuperCollider and SC.VO.3.1 for python and electrical impulses using GPIOS.


[ REPOSITORY ]  [ PHOTO LIBRARY ] [ LABORATORY ]



RELATED WORK:

TOPOLOGIES OF DESIRE
ENTRAINMENT
DIMENSIONS
ACTION POTENTIAL

ACTION POTENTIAL [ 2014 ]

Action potential is a research project on Neuroesthetics, a recent subdiscipline of empirical aesthetics, that studies the possible relationships existing between art perception and the bioelectric signals produced in the brain in terms of active listening and observation.

The first experiments took place during the commemorative exhibition of poet and art critic Octavio Paz at Palacio de Bellas Artes in México. The experiment consisted in recording the bioelectrical activity of 20 volunteers of different ages and backgrounds under active observation of one of the art pieces, active listening of a text by Octavio Paz concerning the piece of art and the two stimuli simultaneously, using a wireless brain-computer interface. Once the sampling was completed we converted the signals in to sound using the OSC protocol in Pure Data, and in to computer graphics through Processing with the de- sign of an algorithm for data interpretation.

Action potential is a multidisciplinary approach intended to assist the exploration of patterns during the data analysis under the aesthetic experience. In collaboration with Erick Fernando Gonzaález Castañeda and Alejandro Antonio Torres García directed by Luis Villaseñor Pineda Ph.D and Carlos Reyes García Ph.D of the National Institute of Astrophysics, optics and electronics (INAOE) for interpretive advice and spectral analysis of the recorded samplings.


[ PROJECT SITE ]  [ REPOSITORY ]



RELATED WORK:

TOPOLOGIES OF DESIRE
ENTRAINMENT
OPEN SENSORY SUBSTITUTION
DIMENSIONS