Menu

Art and AI – Outbreaks from the Grid

November 5, 2020

A team from UVM gets help from the Northeast Cybterteam to develop an AI-enabled tool for creating digital artworks.

Reporting by Helen Hill for MGHPCC

Jenn Karson is an artist who works with concepts of mapping —the organization of space—as well as scoring—the organization of time. Her artworks take the form of intermedia, sculpture, sound works, programming, and installation. She is also a lecturer at the University of Vermont’s Department of Art and Art History and oversees the College of Engineering and Mathematical Sciences digital fabrication lab (the UVM CEMS FabLab).

In 2018 Karson found herself creating a series of collages exploring ideas of visual hierarchy. Originally inspired by the architecture of the Old Mill tower on the UVM campus – the octagon forming its base and the eight triangles forming the facets of its roof – Karson began to realize the images she was creating, and the decisions that led to them, felt a lot like some kind of algorithm at work in her head.

Dancing Stars: Old Mill Tower
Jenn Karson, 2018
24″ x 18″
mixed media

Building on this idea, Karson began musing on what machine learning might bring to her process and so her UVM Art and AI Initiative was born.

“We are exploring approaches to artistic image production, comparing the results of StyleGAN and Genetic Algorithms,” explains Karson. “More broadly, we are exploring emerging artistic practices with Machine Learning and AI while referencing an artistic lineage to artists such as Wassily Kandinsky, John Cage, and Yoko Ono; artists who employ(ed) instructions and systems in their non-digital artworks. For example, Kandinsky distinguished systems and developed a science of aesthetics with the basic elements of point, line, and plane; Cage used the oracle ‘I Ching‘ like a computer to inform his compositional decisions; Ono writes poetic scores that turn her audience into active participants when they follow a series of imaginative instructions. Through this ongoing research and practice, we intend to join the larger conversation about art and AI and design new curricula for UVM undergraduate students.”

Work began in February 2020 with a team that has so far included five  UVM students: two graduate students in data science, an undergraduate mechanical engineering student, and two undergraduate art minors, as well a  UVM alumna who did her undergraduate degree in Studio Art and is now at Parsons School of Design.  

StyleGAN is a generative adversarial network (GAN), an unsupervised machine learning tool able to automatically discover and learn patterns in input data in order to generate new examples with characteristics indistinguishable from those of the original.  Developed by NVIDIA it is notable for its disturbing effectiveness when used to generate  lifelike artificial faces

Recognizing it as a fascinating tool for communicating the otherwise intangible, latent aesthetic information or “style” encoded in a piece of artwork, the team, after first digitizing the collection of physical “source” artworks Karson had created, initially turned to a genetic algorithm as a tool to expand the number of training images.

Based on Processingan open-source language, and development environment built on top of the Java programming language, the team tailored their genetic algorithm (based on one built by Daniel Shiffman) to evolve the arrangement of individual shapes, pulled randomly from the original dataset, allowing a user to flag “pleasing” combinations according to their own preference. The manually chosen selections are then added to the initial training set allowing the algorithm to continuously learn the user’s taste while also growing the collection of artworks informing the algorithm.

Once armed with a sufficiently large dataset, the expanded image library can be used to train the GAN to create more and more compositions.

The genetic algorithm is small enough to run on a laptop but evolving hundreds of images, through thousands of steps required by the StyleGAN2 experiments takes a lot more computing muscle and for this, the team uses UVM’s Vermont Advanced Computing Core (VACC) and RunwayML but look forward to writing their own bespoke GANs in the future. “Our focus is not necessarily on conventional imitation of the original artworks but on the creation of novel images: Trying to find a compromise between physical and digital practice in art,” explained Ethan Davis, a computer science masters student who has a background in fine art .

StyleGAN2 has yielded some delightful surprises for the team in terms of their response to many of the images that emerge.

“Something really interesting happens with images starting to look almost lifelike, like vegetation or even cellular structures,” Karson says. “Other things which, as an artist, are really engaging and interesting visually are the latent spacewalks which seem to be suggestive of something almost multi-dimensional.”

She says, “I am delighting in how machine learning has helped me to move into this more organic space.”

To date, the Art and AI project has been featured in the Art, Robot, and Virtual Creature Gallery at ALife 2020  and the 2020 Mind Candy Small File Media Festival. 

The team is currently in conversation with the Fleming Museum of Art at UVM about some kind of exhibition and is also collaborating with choreographer Paula Higa to film a short film this month (to be presented at dance festivals) that incorporates the images made over the summer as projections and digitally fabricated objects.

Finally, the team will be sharing their work as part of a public online series with the Vermont Studio Center on December 17  

In the meantime, you can hear Jenn (and the students she has been working with) speak in more depth about this project in their ALife video in which she highlights the “artificial life” emergent in the AI-generated images.

The team is particularly grateful for advice and support from  Northeast Cyberteams mentor Adrian Del Maestro a professor at the University of Tennessee, Knoxville.

 

About the Researcher

Jennifer Karson

Jenn Karson is a lecturer at the University of Vermont’s Department of Art and Art History and the director of the College of Engineering and Mathematical Sciences digital fabrication lab (the UVM CEMS FabLab). Additionally, she is a curator of exhibitions, programs, and curricula that cross the disciplines of the arts, technology, fashion, and architecture. She received her MFA in Design + Technology from the San Francisco Art Institute and has artworks exhibited in the United States, Canada, and Europe.

Links

Northeast Cyberteam

studiojuju.com

Previous Post:
Next Post:

Research projects

A Future of Unmanned Aerial Vehicles
Yale Budget Lab
Volcanic Eruptions Impact on Stratospheric Chemistry & Ozone
The Rhode Island Coastal Hazards Analysis, Modeling, and Prediction System
Towards a Whole Brain Cellular Atlas
Tornado Path Detection
The Kempner Institute – Unlocking Intelligence
The Institute for Experiential AI
Taming the Energy Appetite of AI Models
Surface Behavior
Studying Highly Efficient Biological Solar Energy Systems
Software for Unreliable Quantum Computers
Simulating Large Biomolecular Assemblies
SEQer – Sequence Evaluation in Realtime
Revolutionizing Materials Design with Computational Modeling
Remote Sensing of Earth Systems
QuEra at the MGHPCC
Quantum Computing in Renewable Energy Development
Pulling Back the Quantum Curtain on ‘Weyl Fermions’
New Insights on Binary Black Holes
NeuraChip
Network Attached FPGAs in the OCT
Monte Carlo eXtreme (MCX) – a Physically-Accurate Photon Simulator
Modeling Hydrogels and Elastomers
Modeling Breast Cancer Spread
Measuring Neutrino Mass
Investigating Mantle Flow Through Analyses of Earthquake Wave Propagation
Impact of Marine Heatwaves on Coral Diversity
IceCube: Hunting Neutrinos
Genome Forecasting
Global Consequences of Warming-Induced Arctic River Changes
Fuzzing the Linux Kernel
Exact Gravitational Lensing by Rotating Black Holes
Evolution of Viral Infectious Disease
Evaluating Health Benefits of Stricter US Air Quality Standards
Ephemeral Stream Water Contributions to US Drainage Networks
Energy Transport and Ultrafast Spectroscopy Lab
Electron Heating in Kinetic-Alfvén-Wave Turbulence
Discovering Evolution’s Master Switches
Dexterous Robotic Hands
Developing Advanced Materials for a Sustainable Energy Future
Detecting Protein Concentrations in Assays
Denser Environments Cultivate Larger Galaxies
Deciphering Alzheimer’s Disease
Dancing Frog Genomes
Cyber-Physical Communication Network Security
Avoiding Smash Hits
Analyzing the Gut Microbiome
Adaptive Deep Learning Systems Towards Edge Intelligence
Accelerating Rendering Power
ACAS X: A Family of Next-Generation Collision Avoidance Systems
Neurocognition at the Wu Tsai Institute, Yale
Computational Modeling of Biological Systems
Computational Molecular Ecology
Social Capital and Economic Mobility
All Research Projects

Collaborative projects

ALL Collaborative PROJECTS

Outreach & Education Projects

See ALL Scholarships
100 Bigelow Street, Holyoke, MA 01040