MICHAEL HASEY // AI & ARCHITECTURE

Neural Style Transfer // The Building

year: 2020

type: research

GitHub: click for source code

“The rise of Artificial Intelligence in recent years have posed a challenge to the architecture community. How will this novel technology impact our profession?

… [This] might be the first genuinely 21st century design technique as it questions the role of the sole genius, perpetuated by the postmodern era, and proposes a conversation between the creativity and ingenuity of both, mind and machine.”

- Matias del Campo, architect and pioneer of contemporary technologies in architectural production

This research explores a recent breakthrough in A.I. technology called “Neural Style Transfer”, a complex algorithm that allows any image to be re-created in an infinite number of new ways and styles. By taking two images - a content image and a style reference image (such as an artwork by a famous painter), the neural style transfer algorithm “blends” them together and produces a resultant output image that appears to be both the content image and the style reference image at the same time. Though the baseline content and underlying geometric organization of this new image matches the original content image, the re-styled output image appears to be created in its own unique style, allowing us to reinterpret images in ways that we may have never considered or imagined before through traditional means.

What does this mean for design?

By applying this interpretive approach to architecture, neural style transfer algorithms give us the agency to accurately and rapidly visualize buildings in multiple new and novel ways. Instead of taking days or weeks to manually create just a handful of new designs in the traditional manner, this tool could provide the means to produce hundreds within a fraction of time.

This new ability to quickly re-interpret baseline geometries and designs will drastically change and improve the design process by allowing us to rapidly visualize our ideas and approach a satisfactory, and potentially better final design faster. By exploring these new tools, approaches to computer-human design thinking and craft, we may open the doors to new design approaches which will undoubtably improve architecture and built futures in ways in which we could have never imagined.

Screen Shot 2020-07-19 at 12.25.38 PM.png

Visualizing the neural style transfer process

The core idea consists in optimizing a custom loss function by gradient descent, with this loss being computed based on features extracted from intermediate layers of the neural style transfer algorithm (shown left) , which is fed a content image, a style image and finally the image to optimize (output image), which is initially random noise.

Our research focus

Within this research project, we conducted a series of experiments to determine how Neural Style Transfer can be manipulated and applied to produce convincing new buildings in novel styles while maintaining the baseline geometries and composition of the original content image.

In particular, we tried to determine the best way to re-imagine the panorama hotel, an iconic building in the heart of Slovakia designed by Zdeněk Řihák in 1967. This building was chosen due to its clear geometric composition and recognizable cantilevered expressions. A building with strong features was preferable in order to easily compare it to the resultant output building images and understand how new styles have been applied.

Screen Shot 2020-07-20 at 12.01.18 PM.png

Our results

By conducting a series of trial runs using various settings, filter weights, content and style images, we successfully determined the best method to allow this transformation to occur in a clear and convincing way. See below for the specific metrics, filter weights, and iteration counts that we’ve found to be most effective in creating successful output images.

Screen Shot 2020-07-20 at 11.42.14 AM.png

Screen Shot 2020-07-20 at 11.42.38 AM.png

Screen Shot 2020-07-20 at 11.47.05 AM.png

Screen Shot 2020-07-20 at 12.15.02 PM.png

Screen Shot 2020-07-20 at 12.17.33 PM.png

Screen Shot 2020-07-20 at 12.17.17 PM.png

Screen Shot 2020-07-20 at 12.17.01 PM.png

Screen Shot 2020-07-20 at 12.17.54 PM.png

Complexity Experiments

Increasingly complex styles were applied to the baseline content image in order to observe how the neural network understood and applied these gestures in new output building designs.

Screen Shot 2020-07-20 at 11.44.25 AM.png

Screen Shot 2020-07-20 at 11.44.18 AM.png

Screen Shot 2020-07-20 at 12.32.31 PM.png

Screen Shot 2020-07-20 at 12.32.50 PM.png

Screen Shot 2020-07-20 at 12.18.48 PM.png

Screen Shot 2020-07-20 at 12.18.15 PM.png